User-agent: * Disallow: /news/go.php Disallow: /it/go.php User-agent: bingbot Crawl-delay: 120 User-agent: SMTBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: Yandex Disallow: / User-agent: linkdexbot Disallow: / User-agent: Proximic Disallow: / User-agent: EasouSpider Disallow: / User-agent: ChinasoSpider Disallow: / User-agent: Diffbot Disallow: / User-agent: CrystalSemanticsBot Disallow: / User-agent: DotBot Disallow: / User-agent: wotbox Disallow: / User-agent: netseer Disallow: / User-agent: Insitesbot Disallow: / User-agent: nbot Disallow: / User-agent: memoryBot Disallow: / User-agent: SiteExplorer Disallow: / User-agent: GrapeshotCrawler Disallow: / User-agent: Yahoo! Slurp Disallow: / User-agent: SemrushBot Disallow: / User-agent: MBCrawler Disallow: / User-agent: YaK Disallow: / User-agent: YandexBot Disallow: / User-agent: Uptimebot Disallow: / User-agent: Auto Spider Disallow: /