##ACAP version=1.0 # Conventional policies... User-agent: * Disallow: /sendarticle/ Disallow: /Users/ Disallow: /users/ Disallow: /*/print$ Disallow: /email/ Disallow: /contactus/ Disallow: /share/ Disallow: /websearch Disallow: /*?commentpage= Disallow: /whsmiths/ Disallow: /external/overture/ Disallow: /*?showallcomments=true User-agent: SETOOZBOT/0.30 Disallow: / User-agent: Dow Jones Searchbot Disallow: / User-agent: TurnitinBot/2.1 Disallow: / User-agent: SETOOZBOT/0.30 Disallow: / User-agent: msnbot/2.0b Disallow: / User-agent: Dow Jones Searchbot Disallow: / User-agent: discobot/1.1 Disallow: / User-agent: WebDataCentreBot/1.0 Disallow: / User-agent: WebDataCentreBot/1.0 Disallow: / User-agent: MJ12bot/v1.3.2 Disallow: / User-agent: YandexBot/3.0 Disallow: / User-agent: MJ12bot/v1.3.2 Disallow: / User-agent: SBIder/Nutch-1.0-dev Disallow: / User-agent: Nutch-1.0 Disallow: / User-agent: research-scan-bot/Nutch-1.0 Disallow: / User-agent: 91.85.205.218 Disallow: / User-agent: Microsoft URL Control Disallow: / User-agent: SaladSpoon/ShopSalad 1.0 Disallow: / User-agent: msnbot/2.0b Disallow: / User-agent: taptubot Disallow: / User-agent: cuil.com Disallow: / User-agent: spbot Disallow: / User-agent: Amazon AWS Disallow: / User-agent: Alexa Disallow: / User-agent: seoprofiler.com/bot/ Disallow: / User-agent: twiceler/robot Disallow: / User-agent: TurnitinBot/2.1 Disallow: / User-agent: DotBot/1.1 Disallow: / User-agent: CatchBot/1.0 Disallow: / User-agent: aiHitBot Disallow: / User-agent: snprtz Disallow: / User-agent: DotBot/1.1; Disallow: / User-agent: Sogou-Test-Spider/4.0 Disallow: / User-agent: crawler5018 Disallow: / User-agent: CCBot/1.0 Disallow: / User-agent: Gigabot/3.0 Disallow: / User-agent: 142ob.scansafe.net Disallow: / User-agent: spider27 Disallow: / User-agent: dech092 Disallow: / User-agent: taptubot Disallow: / User-agent: msnbot/2.0bt Disallow: / User-agent: Speedy Spider Disallow: / User-agent: envolk Disallow: / User-agent: psbot Disallow: / User-agent: mmcrawler/Nutch Disallow: / User-agent: Chen Li/Nutch Disallow: / User-agent: MySpace Disallow: / User-agent: sbider21 Disallow: / User-agent: bot3 Disallow: / User-agent: Exabot/3.0 Disallow: / User-agent: crawl-5c.cuil.com Disallow: / User-agent: panscient Disallow: / User-agent: Yandex Disallow: / User-agent: ScoutJet Disallow: / User-agent: LinkWalker Disallow: / User-agent: BPImageWalker Disallow: / User-agent: SheenBot Disallow: / User-agent: natcrawlbloc03 Disallow: / User-agent: DotBot Disallow: / User-agent: Baiduspider Disallow: / User-agent: Sogou web spider Disallow: / User-agent: Voilabot Disallow: / User-agent: YoudaoBot Disallow: / User-agent: Yahoo! Slurp Disallow: / User-agent: Googlebot-Mobile Crawl-delay: 1 Disallow: /sendarticle/ Disallow: /Users/ Disallow: /*/print$ Disallow: /email/ Disallow: /contactus/ Disallow: /share/ Disallow: /websearch Disallow: /*?commentpage= Disallow: /whsmiths/ Disallow: /external/overture/ Disallow: /*?showallcomments=true # ACAP policies... ACAP-crawler: * ACAP-disallow-crawl: /sendarticle/ ACAP-disallow-crawl: /Users/ ACAP-disallow-crawl: /*/print$ ACAP-disallow-crawl: /email/ ACAP-disallow-crawl: /contactus/ ACAP-disallow-crawl: /share/ ACAP-disallow-crawl: /websearch ACAP-disallow-crawl: /*?commentpage= ACAP-disallow-crawl: /whsmiths/ ACAP-disallow-crawl: /external/overture/ ACAP-disallow-crawl: /*?showallcomments=true