User-agent: * Disallow: /my/ Disallow: /user/ Disallow: /search Disallow: /*? Disallow: /*& Allow: /*.js Allow: /*.css Allow: /*.jpg Allow: /*.gif Allow: /*.png Crawl-delay: 3 Sitemap: https://avero.by/sitemap.2332742.xml.gz User-agent: msnbot Crawl-delay: 10 User-agent: MJ12bot Crawl-delay: 20 User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: /