User-agent: ia_archiver Disallow: / User-agent: archive.org_bot Disallow: / User-agent: ia_archiver-web.archive.org Disallow: / User-agent: Download Demon Disallow: / User-agent: DownloadDirect.1.0  Disallow: / User-agent: Download Druid Disallow: / User-agent: Download Ninja 3.0  Disallow: / User-agent: Download Master  Disallow: / User-agent: Download Express Disallow: / User-agent: Download Wonder  Disallow: / User-agent: Download.exe(1.1) Disallow: / User-agent: EyeCatcher Disallow: / User-agent: Download-Tipp Linkcheck Disallow: / User-agent: Nederland.zoek Disallow: / User-agent: msnbot Disallow: / User-agent: Teoma Disallow: / User-agent: Scooter Disallow: / User-agent: uTorrent/1500  Disallow: / User-agent: 123spider-Bot Disallow: / User-agent: 8484 Boston Project v 1.0  Disallow: / User-agent: AbachoBOT  Disallow: / User-agent: ABCdatos BotLink/5.xx.xxx#BBL  Disallow: / User-agent: BotLink Disallow: / User-agent: About/0.1libwww-perl/5.47  Disallow: / User-agent: Libby_1.1 Disallow: / User-agent: Ace Explorer  Disallow: / User-agent: Acme.Spider Disallow: / User-agent: Accoona-AI-Agent Disallow: / User-agent: Activeworlds  Disallow: / User-agent: MFHttpScan  Disallow: / User-agent: ahoy Disallow: / User-agent: ia_archiver  Disallow: / User-agent: IAArchiver Disallow: / User-agent: Anonymizer/1.1  Disallow: / User-agent: Anonymouse Disallow: / User-agent: antibot-V1.1.5/i586-linux-2.2  Disallow: / User-agent: A-Online Disallow: / User-agent: Sqworm Disallow: / User-agent: Asahina-Antenna Disallow: / User-agent: aretha Disallow: / User-agent: ArchitextSpider Disallow: / User-agent: AskAboutOil/0.06-rcp Disallow: / User-agent: Teoma Disallow: / User-agent: ASpider Disallow: / User-agent: atSpider/1.0  Disallow: / User-agent: AWeb  Disallow: / User-agent: AVSearch-1.0 Disallow: / User-agent: Azureus Disallow: / User-agent: BackStreet Disallow: / User-agent: BanBots Disallow: / User-agent: Barca Disallow: / User-agent: spider.batsch.com  Disallow: / User-agent: bigbrother Disallow: / User-agent: Big Brother Disallow: / User-agent: Custom Disallow: / User-agent: SBL-BOT Disallow: / User-agent: BlackWidow Disallow: / User-agent: Naamah Disallow: / User-agent: Ukonline Disallow: / User-agent: bluefish Disallow: / User-agent: borg-bot/0.9 Disallow: / User-agent: parallelContextFocusCrawler1.1parallelContextFocusCrawler1.1  Disallow: / User-agent: CE-Preload  Disallow: / User-agent: cg-eye Disallow: / User-agent: Checkbot Disallow: / User-agent: CheckLinks Disallow: / User-agent: BullsEye  Disallow: / User-agent: Bulkfeeds Disallow: / User-agent: CheckWeb  Disallow: / User-agent: CJ Spider Disallow: / User-agent: CipinetBot Disallow: / User-agent: OliverPerry  Disallow: / User-agent: Claymont Disallow: / User-agent: CMC/0.01 Disallow: / User-agent: confuzzledbot Disallow: / User-agent: ContactBot Disallow: / User-agent: ContentSmartz  Disallow: / User-agent: CoolBot Disallow: / User-agent: CoralWebPrx Disallow: / User-agent: carleson Disallow: / User-agent: CrawlConvera Disallow: / User-agent: Crawllybot Disallow: / User-agent: CreativeCommons/0.06-dev Disallow: / User-agent: CrocCrawler Disallow: / User-agent: Cuasarbot Disallow: / User-agent: CyberSpyder Disallow: / User-agent: Cyberdog Disallow: / User-agent: DataparkSearch Disallow: / User-agent: DataSpear Disallow: / User-agent: Talkro Web-Shot/1.0 Disallow: / User-agent: dds explorer Disallow: / User-agent: dbDig Disallow: / User-agent: DeepIndex  Disallow: / User-agent: online link validator Disallow: / User-agent: del.icio.us-thumbnails/1.0 Mozilla/5.0 Disallow: / User-agent: desertrealm, desert realm Disallow: / User-agent: DevComponents.com Disallow: / User-agent: blindekuh Disallow: / User-agent: dienstspider Disallow: / User-agent: Digimarc WebReader Disallow: / User-agent: Pompos Disallow: / User-agent: grabber Disallow: / User-agent: DISCo Pump x.x  Disallow: / User-agent: disco/Nutch-0.9 Disallow: / User-agent: DittoSpyder  Disallow: / User-agent: Dr.Web ® Disallow: / User-agent: DragonBot Disallow: / User-agent: Dragonfly Disallow: / User-agent: eCatch Disallow: / User-agent: EBrowse Disallow: / User-agent: ebiness Disallow: / User-agent: Dumbot Disallow: / User-agent: Maintenance Disallow: / User-agent: elfbot/1.0 Disallow: / User-agent: ELinks Disallow: / User-agent: Emacs-w3/v[0-9\.]+ Disallow: / User-agent: EmailSiphon  Disallow: / User-agent: EmailWolf Disallow: / User-agent: EmeraldShield.com WebBot  Disallow: / User-agent: Search/1.0 Disallow: / User-agent: Enterprise_Search/1.0  Disallow: / User-agent: Speedy Spider Disallow: / User-agent: envolk Disallow: / User-agent: EroCrawler  Disallow: / User-agent: e-sense Disallow: / User-agent: ESISmartSpider  Disallow: / User-agent: ExactSearch  Disallow: / User-agent: eseek-larbin Disallow: / User-agent: ExactSeek Crawler Disallow: / User-agent: Exabot Disallow: / User-agent: Exabot-Images/1.0  Disallow: / User-agent: Excalibur Disallow: / User-agent: Execrawl Disallow: / User-agent: ArchitextSpider  Disallow: / User-agent: Arachnoidea Disallow: / User-agent: EvaalSE Disallow: / User-agent: ExtractorPro  Disallow: / User-agent: Express WebPictures Disallow: / User-agent: Expired Domain Sleuth  Disallow: / User-agent: The Expert HTML Source Viewer Disallow: / User-agent: Extreme Picture Finder  Disallow: / User-agent: Factbot 1.09 (see http://www.factbites.com/webmasters.php)  Disallow: / User-agent: FaEdit/2.0.x  Disallow: / User-agent: FairAd Client  Disallow: / User-agent: FARK.com link verifier  Disallow: / User-agent: fastbot Disallow: / User-agent: fastcrawler Disallow: / User-agent: FavIconizer  Disallow: / User-agent: favo.eu Disallow: / User-agent: FavOrg  Disallow: / User-agent: Favorites Sweeper Disallow: / User-agent: Feed Seeker Bot Disallow: / User-agent: Feed::Find/0.0x  Disallow: / User-agent: FeedChecker Disallow: / User-agent: FeedForAll rss2html.php v2  Disallow: / User-agent: BlogMap Disallow: / User-agent: ESI Disallow: / User-agent: NewsGator FetchLinks Disallow: / User-agent: fetch libfetch Disallow: / User-agent: FileHeap! file downloader Disallow: / User-agent: FileHound Disallow: / User-agent: JobSpider_BA Disallow: / User-agent: Finjan-prefetch  Disallow: / User-agent: Firefly Disallow: / User-agent: KIT-Fireball/2.0  Disallow: / User-agent: Fish-Search-Robot Disallow: / User-agent: firstsbot  Disallow: / User-agent: Flapbot Disallow: / User-agent: fly/6.01 libwww/4.0D  Disallow: / User-agent: flyindex.net 1.0/http://www.flyindex.net  Disallow: / User-agent: Freecrawl Disallow: / User-agent: FreeFind.com-SiteSearchEngine Disallow: / User-agent: FFC Trap Door Spider  Disallow: / User-agent: FreshNotes Disallow: / User-agent: FSurf15a 01  Disallow: / User-agent: Full Web Bot 0416B  Disallow: / User-agent: Full Web Bot 0516B  Disallow: / User-agent: Full Web Bot 2816B  Disallow: / User-agent: FunnelWeb-1.0 Disallow: / User-agent: FuseBulb Disallow: / User-agent: FyberSpider Disallow: / User-agent: geckobot  Disallow: / User-agent: generic_crawler Disallow: / User-agent: genieBot Disallow: / User-agent: GeoBot Disallow: / User-agent: GetBot  Disallow: / User-agent: getbot Disallow: / User-agent: GetRight/4.5xx  Disallow: / User-agent: GetRight Disallow: / User-agent: GetURL.rexx v1.05 Disallow: / User-agent: GetWeb Disallow: / User-agent: MicroBaz  Disallow: / User-agent: gigabaz Disallow: / User-agent: Gigabot Disallow: / User-agent: GigabotSiteSearch Disallow: / User-agent: GNODSPIDER  Disallow: / User-agent: Go!Zilla Disallow: / User-agent: mogimogi Disallow: / User-agent: moget Disallow: / User-agent: ichiro Disallow: / User-agent: Goofer Disallow: / User-agent: Big Fish Disallow: / User-agent: Go-Ahead-Got-It Disallow: / User-agent: GrapeFX Disallow: / User-agent: grapnel Disallow: / User-agent: monkeyagent  Disallow: / User-agent: GreatNews Disallow: / User-agent: GreenBrowser  Disallow: / User-agent: gridwell Disallow: / User-agent: griffon Disallow: / User-agent: GrigorBot Disallow: / User-agent: Gromit Disallow: / User-agent: GSiteCrawler Disallow: / User-agent: Guestbook Auto Submitter  Disallow: / User-agent: gulper Disallow: / User-agent: Gungho Disallow: / User-agent: harvest Disallow: / User-agent: Harvest-NG Disallow: / User-agent: hambot Disallow: / User-agent: HappyFunBot Disallow: / User-agent: AITCSRobot/1.1 Disallow: / User-agent: HiDownload  Disallow: / User-agent: Hippias Disallow: / User-agent: Hitwise Disallow: / User-agent: HomePageSearch Disallow: / User-agent: Homerbot Disallow: / User-agent: nt Hometown Spider Pro Disallow: / User-agent: Honda-Search Disallow: / User-agent: HooWWWer Disallow: / User-agent: htdig Disallow: / User-agent: HPL/Nutch Disallow: / User-agent: HTMLParser Disallow: / User-agent: Html Link Disallow: / User-agent: pitkow Disallow: / User-agent: HTML2JPG Disallow: / User-agent: HTTP Retriever  Disallow: / User-agent: ParaSite Disallow: / User-agent: HTTP::Lite  Disallow: / User-agent: HTTPResume  Disallow: / User-agent: httpunit Disallow: / User-agent: Hybrid Disallow: / User-agent: HyperEstraier Disallow: / User-agent: hyperdecontextualizer Disallow: / User-agent: irobot Disallow: / User-agent: i1searchbot Disallow: / User-agent: iajabot Disallow: / User-agent: iaskspider2 Disallow: / User-agent: iaskspider  Disallow: / User-agent: LotusDiscovery Disallow: / User-agent: t IBM_Planetwide Disallow: / User-agent: almaden Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: BlogSearch Disallow: / User-agent: BlogzIce Disallow: / User-agent: icsbot-0.1  Disallow: / User-agent: iearthworm Disallow: / User-agent: IlTrovatore Disallow: / User-agent: ilial/Nutch Disallow: / User-agent: image.kapsi.net/1.0 Disallow: / User-agent: ImageVisu/ Disallow: / User-agent: IndexTheWeb Disallow: / User-agent: Industry Program Disallow: / User-agent: vb wininet  Disallow: / User-agent: infoConveraCrawler Disallow: / User-agent: InfoFly Disallow: / User-agent: INFOMINE Disallow: / User-agent: DataFountains Disallow: / User-agent: Informant Disallow: / User-agent: Sidewinder Disallow: / User-agent: InfoSeek Robot 1.0 Disallow: / User-agent: Ultraseek  Disallow: / User-agent: Infoseek Sidewinder Disallow: / User-agent: InfoSpiders Disallow: / User-agent: INGRID/0.1 Disallow: / User-agent: slurp Disallow: / User-agent: URL_Spider_Pro Disallow: / User-agent: InnerpriseBot Disallow: / User-agent: Rational SiteCheck Disallow: / User-agent: inspectorwww Disallow: / User-agent: IAGENT/1.0 Disallow: / User-agent: Blogpulse Disallow: / User-agent: Internet Cruiser Robot Disallow: / User-agent: sitecheck.internetseer.com  Disallow: / User-agent: IRLbot Disallow: / User-agent: Iron33 Disallow: / User-agent: IsraeliSearch/1.0 Disallow: / User-agent: iSiloX Disallow: / User-agent: Irvine Disallow: / User-agent: IrssiUrlLog Disallow: / User-agent: istarthere Disallow: / User-agent: isurf Disallow: / User-agent: iVia Disallow: / User-agent: IWAgent Disallow: / User-agent: Jakarta Commons-HttpClient Disallow: / User-agent: Jambot Disallow: / User-agent: ideare Disallow: / User-agent: SearchTone2.0 Disallow: / User-agent: JavaBee Disallow: / User-agent: Jayde Disallow: / User-agent: JBot Disallow: / User-agent: jBrowser Disallow: / User-agent: JCheckLinks Disallow: / User-agent: jcrawler Disallow: / User-agent: JetCar  Disallow: / User-agent: Jetbot Disallow: / User-agent: Jigsaw Disallow: / User-agent: jobo Disallow: / User-agent: Jobot/0.1alpha libwww-perl/4.0 Disallow: / User-agent: JoeBot/x.x Disallow: / User-agent: JordoMedia Disallow: / User-agent: Journster Disallow: / User-agent: J-PHONE Disallow: / User-agent: JRTwine Disallow: / User-agent: jumpstation Disallow: / User-agent: Jyxobot Disallow: / User-agent: KakleBot Disallow: / User-agent: Kapere Disallow: / User-agent: Katipo/1.0 Disallow: / User-agent: Kazehakase Disallow: / User-agent: KDD-Explorer Disallow: / User-agent: Kenjin Disallow: / User-agent: EasyDL Disallow: / User-agent: KFSW-Bot Disallow: / User-agent: kilroy Disallow: / User-agent: kinjabot Disallow: / User-agent: Klondike Disallow: / User-agent: KIT-Fireball Disallow: / User-agent: K-Meleon Disallow: / User-agent: KnowItAll Disallow: / User-agent: Knowledge Disallow: / User-agent: Ksbot Disallow: / User-agent: ko_yappo_robot Disallow: / User-agent: Mariner/5.1b Disallow: / User-agent: Kontiki Disallow: / User-agent: Krugle Disallow: / User-agent: kuloko-bot Disallow: / User-agent: kulturarw3 Disallow: / User-agent: KummHttp/1.1 Disallow: / User-agent: label-grabber Disallow: / User-agent: Lachesis  Disallow: / User-agent: lanshanbot Disallow: / User-agent: LapozzBot/1.4 Disallow: / User-agent: larbin Disallow: / User-agent: IPiumBot Disallow: / User-agent: lawinfo-crawler Disallow: / User-agent: lcabotAccept Disallow: / User-agent: LECodeChecker Disallow: / User-agent: LeechGet Disallow: / User-agent: LetsCrawl Disallow: / User-agent: legs Disallow: / User-agent: LexiBot Disallow: / User-agent: Mata Hari/2.00   Disallow: / User-agent: LNSpiderguy  Disallow: / User-agent: libcurl-agent/1.0  Disallow: / User-agent: LibertyW Disallow: / User-agent: ActiveBookmark Disallow: / User-agent: libWeb/clsHTTP Disallow: / User-agent: Liferea Disallow: / User-agent: LightningDownload Disallow: / User-agent: Lincoln State Web Browser  Disallow: / User-agent: Link Valet Disallow: / User-agent: Linkidator Disallow: / User-agent: link_check3 Disallow: / User-agent: LinkAlarm Disallow: / User-agent: Linkbot  Disallow: / User-agent: LinkCheck Disallow: / User-agent: BeebwareDirectory Disallow: / User-agent: LinkLint-checkonly Disallow: / User-agent: linknzbot  Disallow: / User-agent: AgentName/0.1 Disallow: / User-agent: LinkPimpin  Disallow: / User-agent: Links Disallow: / User-agent: Links4US-Crawler Disallow: / User-agent: LinkScan Disallow: / User-agent: LinksManager Disallow: / User-agent: LinkSonar Disallow: / User-agent: LinkWalker  Disallow: / User-agent: ListBidBot Disallow: / User-agent: LjSEEK Picture-Bot Disallow: / User-agent: lmspider Disallow: / User-agent: LMQueueBot/0.2  Disallow: / User-agent: Llaut/1.0 Disallow: / User-agent: LocalcomBot Disallow: / User-agent: LocalBot/1.0 Disallow: / User-agent: Lockon Disallow: / User-agent: Lockstep Disallow: / User-agent: logo_gif_crawler Disallow: / User-agent: NetResearchServer Disallow: / User-agent: Seeker.lookseek.com  Disallow: / User-agent: MantraAgent  Disallow: / User-agent: MARTINI  Disallow: / User-agent: smartwit.com  Disallow: / User-agent: luchs Disallow: / User-agent: Lunascape  Disallow: / User-agent: Lycos/x.x Disallow: / User-agent: Mac Finder Disallow: / User-agent: macworm Disallow: / User-agent: MagicWML/1.0 Disallow: / User-agent: Mag-Net  Disallow: / User-agent: MagpieRSS Disallow: / User-agent: Magpie/1.0 Disallow: / User-agent: MJ12bot Disallow: / User-agent: mammoth Disallow: / User-agent: MapoftheInternet Disallow: / User-agent: OSSProxy Disallow: / User-agent: Marvin v0.3  Disallow: / User-agent: marvin Disallow: / User-agent: MasterSeek  Disallow: / User-agent: SurfMaster  Disallow: / User-agent: mattie Disallow: / User-agent: Spider/maxbot.com Disallow: / User-agent: maxomobot/dev-20051201 Disallow: / User-agent: McBot/5.001 Disallow: / User-agent: MediaCrawler-1.0 Disallow: / User-agent: mediafox Disallow: / User-agent: Spinne Disallow: / User-agent: MegaSheep Disallow: / User-agent: Mercator Disallow: / User-agent: Mercator-Scrub Disallow: / User-agent: MerzScope Disallow: / User-agent: RRC (crawler_admin@bigfoot.com)  Disallow: / User-agent: flunky  Disallow: / User-agent: Metaeuro Disallow: / User-agent: MetaGer_PreChecker0.1  Disallow: / User-agent: MetagerBot/0.8-dev Disallow: / User-agent: MetaGer-LinkChecker  Disallow: / User-agent: XRL Disallow: / User-agent: Mqbot Disallow: / User-agent: Metaspinner Disallow: / User-agent: metatagsdir Disallow: / User-agent: MSRBOT Disallow: / User-agent: MFC_Tear_Sample  Disallow: / User-agent: MicrosoftPrototypeCrawler Disallow: / User-agent: Intelix Disallow: / User-agent: MindCrawler Disallow: / User-agent: Mindjet Disallow: / User-agent: miniRank/1.6 Disallow: / User-agent: minibot  Disallow: / User-agent: MiracleAlphaTest  Disallow: / User-agent: ExperimentalHenrytheMiragoRobot  Disallow: / User-agent: HeinrichderMiragoRobot  Disallow: / User-agent: HenrytheMiragoRobot  Disallow: / User-agent: HenriLeRobotMirago Disallow: / User-agent: Missauga Locate Disallow: / User-agent: Missouri College Browse  Disallow: / User-agent: Mister Pix II Disallow: / User-agent: Misterbot-Nutch/0.7.1 Disallow: / User-agent: Miva Disallow: / User-agent: Felix - Mixcat Crawler (+http://mixcat.com)  Disallow: / User-agent: Morris - Mixcat Disallow: / User-agent: Mizzu Labs 2.2  Disallow: / User-agent: MnogoSearch Disallow: / User-agent: udmsearch Disallow: / User-agent: Mo College 1.9  Disallow: / User-agent: moget Disallow: / User-agent: moiNAG Disallow: / User-agent: MojeekBot Disallow: / User-agent: MOMspider/1.00 libwww-perl/0.40 Disallow: / User-agent: monogol Disallow: / User-agent: Monster Disallow: / User-agent: MoonBrowser Disallow: / User-agent: Moreoverbot Disallow: / User-agent: holmes Disallow: / User-agent: Motor Disallow: / User-agent: Motoricerca-Robots.txt-Checker/1.0 Disallow: / User-agent: mozDex Disallow: / User-agent: Mozi!  Disallow: / User-agent: Mouse-House/7.4 Disallow: / User-agent: msnbot-media/1.0 Disallow: / User-agent: multiBlocker Disallow: / User-agent: msnbot Disallow: / User-agent: multicrawler Disallow: / User-agent: MultiText Disallow: / User-agent: muncher Disallow: / User-agent: muninn Disallow: / User-agent: MuscatFerret Disallow: / User-agent: MVAClient  Disallow: / User-agent: MwdSearch Disallow: / User-agent: Snappy/1.1 Disallow: / User-agent: myDaemon  Disallow: / User-agent: UIowaCrawler/1.0  Disallow: / User-agent: Mylinea.com Disallow: / User-agent: NP Disallow: / User-agent: aipbot Disallow: / User-agent: NPBot Disallow: / User-agent: naoFavicon4IE/1.xx  Disallow: / User-agent: NASA Search 1.0  Disallow: / User-agent: NationalDirectory-WebSpider Disallow: / User-agent: dloader Disallow: / User-agent: NABOT/5.0  Disallow: / User-agent: NaverBot-1.0 Disallow: / User-agent: Cowbot Disallow: / User-agent: nabot_1.0  Disallow: / User-agent: NavissoBot  Disallow: / User-agent: NDSpider/1.5 Disallow: / User-agent: Nebullabot/2.2 Disallow: / User-agent: NEC Research Agent Disallow: / User-agent: NEC-MeshExplorer Disallow: / User-agent: NEC-Hayek Disallow: / User-agent: :robot Disallow: / User-agent: Francis Disallow: / User-agent: NetAnts/1.2x  Disallow: / User-agent: netcarta Disallow: / User-agent: NETCOMplete Disallow: / User-agent: NetinfoBot Disallow: / User-agent: NetLookout Disallow: / User-agent: Netluchs/0.8-dev Disallow: / User-agent: NetMechanic Disallow: / User-agent: NetNoseCrawler Disallow: / User-agent: Netprospector Disallow: / User-agent: NetPumper/x.xx  Disallow: / User-agent: NetScoop Disallow: / User-agent: NSPlayer/10.0.0.xxxx WMFSDK/10.0  Disallow: / User-agent: NetWhatCrawler/0.06-dev Disallow: / User-agent: SmartDownload Disallow: / User-agent: NetZippy  Disallow: / User-agent: NeuralBot/0.2  Disallow: / User-agent: newscan-online Disallow: / User-agent: newsearchengine Disallow: / User-agent: NetNewsWire Disallow: / User-agent: NewsGatorOnline Disallow: / User-agent: findlinks/x.xxx Disallow: / User-agent: NextopiaBOT Disallow: / User-agent: NG-Search Disallow: / User-agent: NHSEWalker/3.0 Disallow: / User-agent: nicebot  Disallow: / User-agent: NICO Disallow: / User-agent: Nikita the Spider Disallow: / User-agent: Nitro Disallow: / User-agent: NITLE Blog Spider Disallow: / User-agent: Noago Spider  Disallow: / User-agent: Nocilla Disallow: / User-agent: NokodoBot/1.x Disallow: / User-agent: Gulliver Disallow: / User-agent: gulliver Disallow: / User-agent: Nomad-V2.x Disallow: / User-agent: noxtrumbot/1.0 Disallow: / User-agent: noyona_0_1  Disallow: / User-agent: Nsauditor/1.x  Disallow: / User-agent: CheckUrl  Disallow: / User-agent: nttdirectory_robot/0.9 Disallow: / User-agent: DoCoMo Disallow: / User-agent: Nucleus SiteList LinkChecker/1.1  Disallow: / User-agent: nuSearch Spider Disallow: / User-agent: Nutch  Disallow: / User-agent: explorersearch Disallow: / User-agent: NutchCrawler Disallow: / User-agent: obidos-bot Disallow: / User-agent: ObjectsSearch Disallow: / User-agent: Occam Disallow: / User-agent: Octora Beta Bot - www.octora.com  Disallow: / User-agent: Octopus  Disallow: / User-agent: Jabot Disallow: / User-agent: Offline Explorer 1.*  Disallow: / User-agent: JetBrains Disallow: / User-agent: OmniExplorer_Bot/1.0x Disallow: / User-agent: Online24-Bot Disallow: / User-agent: Slarp/0.1  Disallow: / User-agent: onCHECK Linkchecker Disallow: / User-agent: OntoSpider/1.0 Disallow: / User-agent: OpenISearch Disallow: / User-agent: Openbot/3.0+ Disallow: / User-agent: Openbot Disallow: / User-agent: DigOut4U  Disallow: / User-agent: OpenTaggerBot Disallow: / User-agent: OpenTextSiteCrawler Disallow: / User-agent: OpenWebSpider Disallow: / User-agent: OpidooBOT Disallow: / User-agent: Oracle iMTCrawler  Disallow: / User-agent: OrangeSpider  Disallow: / User-agent: Oracle Ultra Search  Disallow: / User-agent: Orbsearch/1.0 Disallow: / User-agent: Orca Disallow: / User-agent: Orbiter/T-2.0 Disallow: / User-agent: OutfoxBot Disallow: / User-agent: OutfoxMelonBot Disallow: / User-agent: Overture-WebCrawler Disallow: / User-agent: OWR_Crawler Disallow: / User-agent: ozelot Disallow: / User-agent: ShopWiki/1.0 ( +http://www.shopwiki.com/)  Disallow: / User-agent: Shoula.com Crawler 2.0  Disallow: / User-agent: DatenBot Disallow: / User-agent: virus_detector Disallow: / User-agent: SietsCrawler Disallow: / User-agent: sift Disallow: / User-agent: Siigle Orumcex Disallow: / User-agent: SimBot Disallow: / User-agent: SimpleFavPanel/1.2  Disallow: / User-agent: Simpy Disallow: / User-agent: Argus/1.1 Disallow: / User-agent: asterias/2.0  Disallow: / User-agent: Sirketcebot Disallow: / User-agent: ssearcher100 Disallow: / User-agent: SiteSpider Disallow: / User-agent: site-valet Disallow: / User-agent: SiteBar Disallow: / User-agent: SiteRecon+(xx)  Disallow: / User-agent: SBIder Disallow: / User-agent: SiteSucker Disallow: / User-agent: SiteTaggerBot Disallow: / User-agent: SiteTech-Rover Disallow: / User-agent: SiteTruth Disallow: / User-agent: +SitiDi.net/SitiDiBot Disallow: / User-agent: GhostRouteHunter Disallow: / User-agent: Skampy Disallow: / User-agent: SLCrawler/2.0 Disallow: / User-agent: skymob Disallow: / User-agent: Sleek Spider/1.2 Disallow: / User-agent: Sleipnir  Disallow: / User-agent: silk/1.0  Disallow: / User-agent: Slider_Search_v1-de  Disallow: / User-agent: SlySearch Disallow: / User-agent: ESI Disallow: / User-agent: SmiffyDCMetaSpider/1.0  Disallow: / User-agent: Snapbot/1.0  Disallow: / User-agent: Snarfer/0.x.x Disallow: / User-agent: snooper Disallow: / User-agent: SnoopRob/x.x  Disallow: / User-agent: Snoopy v1.xx  Disallow: / User-agent: sna Disallow: / User-agent: SnykeBot Disallow: / User-agent: nsyght.com/Nutch-0.9 Disallow: / User-agent: SocSciBot ()  Disallow: / User-agent: SoftHypermarketFileCheckBot Disallow: / User-agent: Softizerbot Disallow: / User-agent: sohu-search  Disallow: / User-agent: solbot Disallow: / User-agent: Program Shareware 1.0.2  Disallow: / User-agent: MusicWalker2.0 Disallow: / User-agent: Deep Link Calculator v1.0  Disallow: / User-agent: Space Bison Disallow: / User-agent: SpeedDownload Disallow: / User-agent: speedfind ramBot xtreme 8.1  Disallow: / User-agent: speedy Disallow: / User-agent: Sphider  Disallow: / User-agent: Spida/0.1  Disallow: / User-agent: Spinne/2.0  Disallow: / User-agent: spider_monkey Disallow: / User-agent: SpiderBot/1.0 Disallow: / User-agent: SpiderKU/0.x  Disallow: / User-agent: spiderline Disallow: / User-agent: SpiderMan Disallow: / User-agent: SpiderMonkey/7.0x Disallow: / User-agent: SpiderView Disallow: / User-agent: Spock Crawler Disallow: / User-agent: sportsuchmaschine.de-Robot Disallow: / User-agent: sproose/0.1-alpha Disallow: / User-agent: spry Disallow: / User-agent: SQ Webscanner  Disallow: / User-agent: squidclam  Disallow: / User-agent: Squid-Prefetch  Disallow: / User-agent: Sqworm Disallow: / User-agent: SSurf15a Disallow: / User-agent: Stamina/1.4  Disallow: / User-agent: Star Downloader  Disallow: / User-agent: stat statcrawler@gmail.com  Disallow: / User-agent: Steeler Disallow: / User-agent: Strategic Board Bot Disallow: / User-agent: Mulder, VCR-1.0  Disallow: / User-agent: suchbaer Disallow: / User-agent: suchbot  Disallow: / User-agent: Suchknecht.at-Robot  Disallow: / User-agent: CoolBot  Disallow: / User-agent: suchpadbot Disallow: / User-agent: suke Disallow: / User-agent: Sunrise Disallow: / User-agent: suntek Disallow: / User-agent: SuperBot Disallow: / User-agent: Robot@SuperSnooper.Com  Disallow: / User-agent: SURF   Disallow: / User-agent: Submission Spider Disallow: / User-agent: SurveyBot Disallow: / User-agent: sven Disallow: / User-agent: SWB/V1.4 Disallow: / User-agent: Swooglebot/2.0. Disallow: / User-agent: SWSBot-Images Disallow: / User-agent: sygol Disallow: / User-agent: SygolBot Disallow: / User-agent: Sylera Disallow: / User-agent: !Susie Disallow: / User-agent: SyncBot  Disallow: / User-agent: SyncIT/x.x  Disallow: / User-agent: Syndirella/0.91pre  Disallow: / User-agent: SynoBot  Disallow: / User-agent: Syntryx ANT Scout Chassis Pheromone Disallow: / User-agent: Szukacz Disallow: / User-agent: Onet.pl SA- Disallow: / User-agent: Agent-SharewarePlazaFileCheckBot/2.0+ Disallow: / User-agent: SharewarePlazaFileCheckBot/1.0+ Disallow: / User-agent: ShopWiki/1.0 ( +http://www.shopwiki.com/)  Disallow: / User-agent: Shoula.com Crawler 2.0  Disallow: / User-agent: DatenBot Disallow: / User-agent: virus_detector Disallow: / User-agent: SietsCrawler Disallow: / User-agent: sift Disallow: / User-agent: Siigle Orumcex Disallow: / User-agent: SimBot Disallow: / User-agent: SimpleFavPanel/1.2  Disallow: / User-agent: Simpy Disallow: / User-agent: Argus/1.1 Disallow: / User-agent: asterias/2.0  Disallow: / User-agent: Sirketcebot Disallow: / User-agent: ssearcher100 Disallow: / User-agent: SiteSpider Disallow: / User-agent: site-valet Disallow: / User-agent: SiteBar Disallow: / User-agent: SiteRecon+(xx)  Disallow: / User-agent: SBIder Disallow: / User-agent: SiteSucker Disallow: / User-agent: SiteTaggerBot Disallow: / User-agent: SiteTech-Rover Disallow: / User-agent: SiteTruth Disallow: / User-agent: +SitiDi.net/SitiDiBot Disallow: / User-agent: GhostRouteHunter Disallow: / User-agent: Skampy Disallow: / User-agent: skymob Disallow: / User-agent: SLCrawler/2.0 Disallow: / User-agent: Sleek Spider/1.2 Disallow: / User-agent: Sleipnir  Disallow: / User-agent: silk/1.0  Disallow: / User-agent: Slider_Search_v1-de  Disallow: / User-agent: SlimBrowser  Disallow: / User-agent: SlySearch Disallow: / User-agent: ESI Disallow: / User-agent: SmiffyDCMetaSpider/1.0  Disallow: / User-agent: Snapbot/1.0  Disallow: / User-agent: Snarfer/0.x.x Disallow: / User-agent: snooper Disallow: / User-agent: SnoopRob/x.x  Disallow: / User-agent: Snoopy v1.xx  Disallow: / User-agent: sna Disallow: / User-agent: SnykeBot Disallow: / User-agent: nsyght.com/Nutch-0.9 Disallow: / User-agent: SoftHypermarketFileCheckBot Disallow: / User-agent: Softizerbot Disallow: / User-agent: sohu-search  Disallow: / User-agent: solbot Disallow: / User-agent: Program Shareware 1.0.2  Disallow: / User-agent: MusicWalker2.0 Disallow: / User-agent: Deep Link Calculator v1.0  Disallow: / User-agent: Space Bison Disallow: / User-agent: SpeedDownload Disallow: / User-agent: speedfind ramBot xtreme 8.1  Disallow: / User-agent: speedy Disallow: / User-agent: Sphider  Disallow: / User-agent: Spida/0.1  Disallow: / User-agent: Spinne/2.0  Disallow: / User-agent: spider_monkey Disallow: / User-agent: SpiderBot/1.0 Disallow: / User-agent: SpiderKU/0.x  Disallow: / User-agent: spiderline Disallow: / User-agent: SpiderMan Disallow: / User-agent: SpiderMonkey/7.0x Disallow: / User-agent: SpiderView Disallow: / User-agent: Spock Crawler Disallow: / User-agent: sportsuchmaschine.de-Robot Disallow: / User-agent: sproose/0.1-alpha Disallow: / User-agent: spry Disallow: / User-agent: SQ Webscanner  Disallow: / User-agent: squidclam  Disallow: / User-agent: Squid-Prefetch  Disallow: / User-agent: Sqworm Disallow: / User-agent: SSurf15a Disallow: / User-agent: Stamina/1.4  Disallow: / User-agent: Star Downloader  Disallow: / User-agent: stat statcrawler@gmail.com  Disallow: / User-agent: Steeler Disallow: / User-agent: Strategic Board Bot Disallow: / User-agent: Mulder, VCR-1.0  Disallow: / User-agent: suchbaer Disallow: / User-agent: suchbot  Disallow: / User-agent: Suchknecht.at-Robot  Disallow: / User-agent: CoolBot  Disallow: / User-agent: suchpadbot Disallow: / User-agent: suke Disallow: / User-agent: Sunrise Disallow: / User-agent: suntek Disallow: / User-agent: SuperBot Disallow: / User-agent: Robot@SuperSnooper.Com  Disallow: / User-agent: SURF   Disallow: / User-agent: Submission Spider Disallow: / User-agent: SurveyBot Disallow: / User-agent: sven Disallow: / User-agent: SWB/V1.4 Disallow: / User-agent: Swooglebot/2.0. Disallow: / User-agent: SWSBot-Images Disallow: / User-agent: SygolBot Disallow: / User-agent: sygol Disallow: / User-agent: Sylera Disallow: / User-agent: !Susie Disallow: / User-agent: SyncBot  Disallow: / User-agent: SyncIT/x.x  Disallow: / User-agent: Syndirella/0.91pre  Disallow: / User-agent: SynoBot  Disallow: / User-agent: Syntryx ANT Scout Chassis Pheromone Disallow: / User-agent: Szukacz Disallow: / User-agent: Onet.pl SA- Disallow: / User-agent: tach_bw Disallow: / User-agent: LinkProver Disallow: / User-agent: tags2dir.com/0.8 Disallow: / User-agent: Tagword Disallow: / User-agent: Tagyu Agent/1.0  Disallow: / User-agent: TAMU_CS_IRL_CRAWLER/1.0  Disallow: / User-agent: tarantula Disallow: / User-agent: TargetYourNews.com Disallow: / User-agent: targetblaster.com/0.9k  Disallow: / User-agent: tarspider Disallow: / User-agent: TCDBOT/Nutch-0.8 Disallow: / User-agent: dlw3robot Disallow: / User-agent: TechBOT Disallow: / User-agent: TECOMAC-Crawler/0.x  Disallow: / User-agent: Tecomi Bot Disallow: / User-agent: Teleport Pro/1.2x(.1xxx)  Disallow: / User-agent: templeton Disallow: / User-agent: Teoma Disallow: / User-agent: Teradex Mapper Disallow: / User-agent: TeragramCrawler  Disallow: / User-agent: terraminds-bot/1.0 Disallow: / User-agent: Spider.TerraNautic.net Disallow: / User-agent: TerrawizBot Disallow: / User-agent: JubiiRobot/version# Disallow: / User-agent: NorthStar Disallow: / User-agent: w3index Disallow: / User-agent: perignator Disallow: / User-agent: python Disallow: / User-agent: tkwww Disallow: / User-agent: webmoose Disallow: / User-agent: TheSuBot/0.1 Disallow: / User-agent: thumbshots-de-Bot Disallow: / User-agent: Sqworm/2.9.89-BETA Disallow: / User-agent: TimelyWeb Disallow: / User-agent: titan Disallow: / User-agent: titin Disallow: / User-agent: tivraSpider Disallow: / User-agent: TJG/Spider  Disallow: / User-agent: TJvMultiHttpGrabber Disallow: / User-agent: Tkensaku Disallow: / User-agent: topiclink Disallow: / User-agent: toCrawl/UrlDispatcher  Disallow: / User-agent: T-Online Browser  Disallow: / User-agent: Topodia/1.2-dev Disallow: / User-agent: TOPOS robot Disallow: / User-agent: traazibot/testengine Disallow: / User-agent: Trailfire-bot Disallow: / User-agent: Trampelpfad-Spider  Disallow: / User-agent: trendtech Disallow: / User-agent: tricosMetaCheck Disallow: / User-agent: TSurf15a 11  Disallow: / User-agent: TulipChain Disallow: / User-agent: TurnitinBot Disallow: / User-agent: Turnpike Emporium LinkChecker/0.1  Disallow: / User-agent: Tutorial Crawler Disallow: / User-agent: Twiceler-0.9 http://www.cuill.com/twiceler/robot.html  Disallow: / User-agent: Twisted PageGetter  Disallow: / User-agent: Twotrees Reactive Filter Disallow: / User-agent: Willow Internet Crawler by Twotrees Disallow: / User-agent: TygoBot  Disallow: / User-agent: TygoProwler  Disallow: / User-agent: Superdownloads Spiderman  Disallow: / User-agent: UCmore  Disallow: / User-agent: ucsd Disallow: / User-agent: UdmSearch Disallow: / User-agent: UDM  Disallow: / User-agent: Mackster Disallow: / User-agent: UKWizz/Nutch-0.8.1 Disallow: / User-agent: lwp-trivial Disallow: / User-agent: UnChaos Disallow: / User-agent: Under the Rainbow Disallow: / User-agent: Contact  Disallow: / User-agent: UofTDB_experiment Disallow: / User-agent: updated Disallow: / User-agent: UptimeBot Disallow: / User-agent: urlck Disallow: / User-agent: URI::Fetch/0.06  Disallow: / User-agent: Search+  Disallow: / User-agent: URL Spider Pro Disallow: / User-agent: URLBase/6.x  Disallow: / User-agent: URLBlaze  Disallow: / User-agent: URLGetFile  Disallow: / User-agent: Vakes Disallow: / User-agent: USyd-NLP-Spider Disallow: / User-agent: Valkyrie libwww-perl Disallow: / User-agent: VayalaCreep Disallow: / User-agent: mylinkcheck/1.02  Disallow: / User-agent: k2spider  Disallow: / User-agent: vspider  Disallow: / User-agent: versus Disallow: / User-agent: verticrawl Disallow: / User-agent: VeryGoodSearch.com.DaddyLongLegs  Disallow: / User-agent: verzamelgids.nl - Networking4all Bot/x.x  Disallow: / User-agent: Vespa Crawler  Disallow: / User-agent: AlkalineBOT Disallow: / User-agent: NCSA Beta 1 Disallow: / User-agent: Victoria Disallow: / User-agent: Matrix Disallow: / User-agent: VisBot/2.0 Disallow: / User-agent: Visicom Toolbar  Disallow: / User-agent: Vision Research Lab image spider at vision.ece.ucsb.edu  Disallow: / User-agent: vision-search Disallow: / User-agent: Skywalker Disallow: / User-agent: VLC media player Disallow: / User-agent: VSE Disallow: / User-agent: VMBot Disallow: / User-agent: void-bot Disallow: / User-agent: KE_1.0/2.0 Disallow: / User-agent: Vortex Disallow: / User-agent: Voyager Disallow: / User-agent: Everest-Vulcan Inc./0.1 Disallow: / User-agent: VWbot_K Disallow: / User-agent: VWBOT/Nutch-0.9-dev Disallow: / User-agent: wapspider Disallow: / User-agent: W3SiteSearch Crawler_v1.1 Disallow: / User-agent: W3CLineMode/5.4.0 libwww/5.x.x  Disallow: / User-agent: W3C-checklink Disallow: / User-agent: w3m/0.x.xx  Disallow: / User-agent: W3M2 Disallow: / User-agent: w3mir Disallow: / User-agent: wadaino.jp-crawler Disallow: / User-agent: appie Disallow: / User-agent: appie Disallow: / User-agent: crawlpaper Disallow: / User-agent: SurferF3 1/0  Disallow: / User-agent: WannaBe (Macintosh; PPC)  Disallow: / User-agent: WapOnWindows Disallow: / User-agent: Watchfire WebXM 1.0  Disallow: / User-agent: WAVcheck Disallow: / User-agent: Wavefire Disallow: / User-agent: Waypath Disallow: / User-agent: Waypath Scout v2.x Disallow: / User-agent: WDG_Validator Disallow: / User-agent: webbandit Disallow: / User-agent: webcore Disallow: / User-agent: UtilMind Disallow: / User-agent: WebTrends Disallow: / User-agent: WebVulnCrawl.blogspot.com/1.0 libwww-perl/5.803  Disallow: / User-agent: WebWatcherMonitor/2.01  Disallow: / User-agent: wombat Disallow: / User-agent: web2express.org/Nutch-0.9-dev Disallow: / User-agent: WebAlta Crawler/1.2.1 Disallow: / User-agent: pulseBot Disallow: / User-agent: WebarooBot Disallow: / User-agent: WebBandit/1.0 Disallow: / User-agent: web-bekannt Disallow: / User-agent: Web-Bot V1.03  Disallow: / User-agent: WebCatcher Disallow: / User-agent: Webclipping.com  Disallow: / User-agent: webcollage/1.xx  Disallow: / User-agent: W3C-WebCon/5.x.x libwww/5.x.x  Disallow: / User-agent: WebCopier Disallow: / User-agent: webcopy Disallow: / User-agent: WebCorp/1.0  Disallow: / User-agent: webcrawl.net  Disallow: / User-agent: WebDownloader Disallow: / User-agent: Webdup Disallow: / User-agent: WebFetcher Disallow: / User-agent: WebFilter Robot Disallow: / User-agent: WebFindBot Disallow: / User-agent: webfoot Disallow: / User-agent: Webglimpse Disallow: / User-agent: webGobbler Disallow: / User-agent: webhack  Disallow: / User-agent: WebImages Disallow: / User-agent: T-H-U-N-D-E-R-S-T-O-N-E Disallow: / User-agent: IIITBOT Disallow: / User-agent: weblayers Disallow: / User-agent: WebLight Disallow: / User-agent: Weblog Attitude Diffusion Disallow: / User-agent: wlm Disallow: / User-agent: webmeasurement-bot Disallow: / User-agent: WebMiner Disallow: / User-agent: webmirror Disallow: / User-agent: WeBoX Disallow: / User-agent: WebPix Disallow: / User-agent: webquest Disallow: / User-agent: WebRACE Disallow: / User-agent: WebRankSpider Disallow: / User-agent: WebReaper Disallow: / User-agent: Web-Robot Disallow: / User-agent: webs Disallow: / User-agent: savvybot/0.2  Disallow: / User-agent: WebSearch.COM.AU/3.0.1 Disallow: / User-agent: WebSearchBench WebCrawler Disallow: / User-agent: WSB WebCrawler Disallow: / User-agent: Website Explorer Disallow: / User-agent: Website eXtractor  Disallow: / User-agent: WebsiteWorth Disallow: / User-agent: websnarf Disallow: / User-agent: webspider Disallow: / User-agent: Webspinne Disallow: / User-agent: Websquash Disallow: / User-agent: WebStat/1.0 (Unix; beta; 20040314)  Disallow: / User-agent: Webster Disallow: / User-agent: webster-internet.de pad browser  Disallow: / User-agent: WOLP Disallow: / User-agent: WebStripper Disallow: / User-agent: WebTrafficExpress Disallow: / User-agent: WebVac Disallow: / User-agent: WebVal Disallow: / User-agent: Webverzeichnis Disallow: / User-agent: webwalk Disallow: / User-agent: WebWalker Disallow: / User-agent: SiteWinder  Disallow: / User-agent: Utopia WebWasher Disallow: / User-agent: WebWatch Disallow: / User-agent: ScollSpider Disallow: / User-agent: WebZIP Disallow: / User-agent: Wells Search II  Disallow: / User-agent: WEP Search Disallow: / User-agent: WFARC  Disallow: / User-agent: wget Disallow: / User-agent: Whatsup Disallow: / User-agent: whatUseek_winona Disallow: / User-agent: winona Disallow: / User-agent: WhizBang! Lab  Disallow: / User-agent: DaviesBot Disallow: / User-agent: whowhere Disallow: / User-agent: Wildsoft Surfer  Disallow: / User-agent: WincerSong Disallow: / User-agent: WinampMPEG Disallow: / User-agent: msnbot-Products/1.0 Disallow: / User-agent: WebFetch  Disallow: / User-agent: TeamSoft WinInet Component  Disallow: / User-agent: WinkBot Disallow: / User-agent: WinPodder Disallow: / User-agent: WinWAP Disallow: / User-agent: WIRE Disallow: / User-agent: NetSprint Disallow: / User-agent: Vagabondo Disallow: / User-agent: WISEbot Disallow: / User-agent: ZyBorg Disallow: / User-agent: WiseWire-Spider2  Disallow: / User-agent: wish-project Disallow: / User-agent: WordChampBot  Disallow: / User-agent: woriobot Disallow: / User-agent: WWWeasel Robot Disallow: / User-agent: wanderer Disallow: / User-agent: worm Disallow: / User-agent: WorldLight  Disallow: / User-agent: WorQmada Disallow: / User-agent: Wotbox Disallow: / User-agent: wume_crawler Disallow: / User-agent: Wusage/x.0@boutell.com  Disallow: / User-agent: West Wind Internet Protocols 4.xx  Disallow: / User-agent: Wwlib/Linux  Disallow: / User-agent: WWSBOT Disallow: / User-agent: MediaSearch/0.1  Disallow: / User-agent: www4mail/2.x libwww-FM/2.14 Disallow: / User-agent: WWWC Disallow: / User-agent: WWWC Disallow: / User-agent: WWW-Mechanize Disallow: / User-agent: wwwster Disallow: / User-agent: WWWOFFLE Disallow: / User-agent: Xaldon WebSpider  Disallow: / User-agent: egothor Disallow: / User-agent: Xenu Disallow: / User-agent: Xerka WebBot Disallow: / User-agent: XGET Disallow: / User-agent: xine Disallow: / User-agent: xirq Disallow: / User-agent: XMLSlurp Disallow: / User-agent: Project XP5 [2.03.07-111203]  Disallow: / User-agent: MIIxpc/4.2  Disallow: / User-agent: Search-Engine-Studio  Disallow: / User-agent: SiteXpert  Disallow: / User-agent: cosmos Disallow: / User-agent: Y!TunnelPro  Disallow: / User-agent: yacy Disallow: / User-agent: Yahoo-Blogs/v3.9 Disallow: / User-agent: PostFavorites  Disallow: / User-agent: Yahoo! Mindset  Disallow: / User-agent: Yahoo-MMAudVid Disallow: / User-agent: Y!OASIS Disallow: / User-agent: Yahoo Pipes 1.0  Disallow: / User-agent: YahooSeeker Disallow: / User-agent: YahooFeedSeeker Disallow: / User-agent: YahooSeeker/CafeKelsa-dev Disallow: / User-agent: Y!J-BSC Disallow: / User-agent: Wir sind die Borg Disallow: / User-agent: WebAuto Disallow: / User-agent: Yandex/1.01.001 (compatible; Win16; I)  Disallow: / User-agent: yarienavoir.net Disallow: / User-agent: spider.yellopet.com Disallow: / User-agent: Findexa Disallow: / User-agent: yggdrasil/Nutch-0.9 Disallow: / User-agent: RssReader Disallow: / User-agent: YodaoBot Disallow: / User-agent: suzuran  Disallow: / User-agent: yoogliFetchAgent/0.1  Disallow: / User-agent: yoono Disallow: / User-agent: YottaCars_Bot Disallow: / User-agent: YottaShopping_Bot Disallow: / User-agent: GulperBot Disallow: / User-agent: Z-Add Link Checker Disallow: / User-agent: Zao-Crawler  Disallow: / User-agent: ZeBot_lseek.net Disallow: / User-agent: Zearchit  Disallow: / User-agent: zedzo.digest Disallow: / User-agent: Zend_Http_Client  Disallow: / User-agent: zerxbot Disallow: / User-agent: Zbot Disallow: / User-agent: Zeus Disallow: / User-agent: Zeus ThemeSite Viewer Webster Pro V2.9 Win32  Disallow: / User-agent: Zeusbot Disallow: / User-agent: Ziggy Disallow: / User-agent: ZipppBot Disallow: / User-agent: ZIPPPCVS Disallow: / User-agent: Zippy Disallow: / User-agent: Zoo Tycoon 2 Client Disallow: / User-agent: ZoomSpider Disallow: / User-agent: zspider Disallow: / User-agent: Alexibot Disallow: / User-agent: Pasiphae Disallow: / User-agent: Aqua_Products Disallow: / User-agent: BackDoorBot Disallow: / User-agent: BackDoorBot/1.0 Disallow: / User-agent: Black.Hole Disallow: / User-agent: BlackWidow Disallow: / User-agent: BlowFish Disallow: / User-agent: BlowFish/1.0 Disallow: / User-agent: Bookmark search tool Disallow: / User-agent: Bot mailto:craftbot@yahoo.com Disallow: / User-agent: BotALot Disallow: / User-agent: BotRightHere Disallow: / User-agent: BuiltBotTough Disallow: / User-agent: Bullseye Disallow: / User-agent: Bullseye/1.0 Disallow: / User-agent: BunnySlippers Disallow: / User-agent: Cegbfeieh Disallow: / User-agent: CheeseBot Disallow: / User-agent: CherryPicker Disallow: / User-agent: CherryPickerElite/1.0 Disallow: / User-agent: CherryPickerSE/1.0 Disallow: / User-agent: ChinaClaw Disallow: / User-agent: Copernic Disallow: / User-agent: CopyRightCheck Disallow: / User-agent: Crescent Disallow: / User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 Disallow: / User-agent: Custo Disallow: / User-agent: DISCo Disallow: / User-agent: DISCo Pump 3.0 Disallow: / User-agent: DISCo Pump 3.2 Disallow: / User-agent: DISCoFinder Disallow: / User-agent: DittoSpyder Disallow: / User-agent: Download Demon Disallow: / User-agent: Download Demon/3.2.0.8 Disallow: / User-agent: Download Demon/3.5.0.11 Disallow: / User-agent: EirGrabber Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: EroCrawler Disallow: / User-agent: Express WebPictures Disallow: / User-agent: Express WebPictures (www.express-soft.com) Disallow: / User-agent: ExtractorPro Disallow: / User-agent: EyeNetIE Disallow: / User-agent: FairAd Client Disallow: / User-agent: Flaming AttackBot Disallow: / User-agent: FlashGet Disallow: / User-agent: FlashGet WebWasher 3.2 Disallow: / User-agent: Foobot Disallow: / User-agent: FrontPage Disallow: / User-agent: FrontPage [NC,OR] Disallow: / User-agent: Gaisbot Disallow: / User-agent: GetRight Disallow: / User-agent: GetRight/2.11 Disallow: / User-agent: GetRight/3.1 Disallow: / User-agent: GetRight/3.2 Disallow: / User-agent: GetRight/3.3 Disallow: / User-agent: GetRight/3.3.3 Disallow: / User-agent: GetRight/3.3.4 Disallow: / User-agent: GetRight/4.0.0 Disallow: / User-agent: GetRight/4.1.0 Disallow: / User-agent: GetRight/4.1.1 Disallow: / User-agent: GetRight/4.1.2 Disallow: / User-agent: GetRight/4.2 Disallow: / User-agent: GetRight/4.2b (Portuguxeas) Disallow: / User-agent: GetRight/4.2c Disallow: / User-agent: GetRight/4.3 Disallow: / User-agent: GetRight/4.5 Disallow: / User-agent: GetRight/4.5a Disallow: / User-agent: GetRight/4.5b Disallow: / User-agent: GetRight/4.5b2 Disallow: / User-agent: GetRight/4.5b1 Disallow: / User-agent: GetRight/4.5b3 Disallow: / User-agent: GetRight/4.5b6 Disallow: / User-agent: GetRight/4.5b7 Disallow: / User-agent: GetRight/4.5c Disallow: / User-agent: GetRight/4.5d Disallow: / User-agent: GetRight/4.5e Disallow: / User-agent: GetRight/5.0beta1 Disallow: / User-agent: GetRight/5.0beta2 Disallow: / User-agent: GetWeb! Disallow: / User-agent: Go!Zilla Disallow: / User-agent: Go!Zilla (www.gozilla.com) Disallow: / User-agent: Go!Zilla 3.3 (www.gozilla.com) Disallow: / User-agent: Go!Zilla 3.5 (www.gozilla.com) Disallow: / User-agent: Go-Ahead-Got-It Disallow: / User-agent: GrabNet Disallow: / User-agent: Grafula Disallow: / User-agent: HMView Disallow: / User-agent: HTTrack Disallow: / User-agent: HTTrack 3.0 Disallow: / User-agent: HTTrack [NC,OR] Disallow: / User-agent: Harvest Disallow: / User-agent: Harvest/1.5 Disallow: / User-agent: Image Stripper Disallow: / User-agent: Image Sucker Disallow: / User-agent: Indy Library Disallow: / User-agent: Indy Library [NC,OR] Disallow: / User-agent: InfoNaviRobot Disallow: / User-agent: InterGET Disallow: / User-agent: Internet Ninja Disallow: / User-agent: Internet Ninja 4.0 Disallow: / User-agent: Internet Ninja 5.0 Disallow: / User-agent: Internet Ninja 6.0 Disallow: / User-agent: Iron33/1.0.2 Disallow: / User-agent: JOC Web Spider Disallow: / User-agent: JennyBot Disallow: / User-agent: JetCar Disallow: / User-agent: Kenjin Spider Disallow: / User-agent: Kenjin.Spider Disallow: / User-agent: Keyword Density/0.9 Disallow: / User-agent: Keyword.Density Disallow: / User-agent: LNSpiderguy Disallow: / User-agent: LeechFTP Disallow: / User-agent: LexiBot Disallow: / User-agent: LinkScan/8.1a Unix Disallow: / User-agent: LinkScan/8.1a.Unix Disallow: / User-agent: LinkWalker Disallow: / User-agent: LinkextractorPro Disallow: / User-agent: MIDown tool Disallow: / User-agent: MIIxpc Disallow: / User-agent: MIIxpc/4.2 Disallow: / User-agent: MSIECrawler Disallow: / User-agent: Mass Downloader Disallow: / User-agent: Mass Downloader/2.2 Disallow: / User-agent: Mata Hari Disallow: / User-agent: Mata.Hari Disallow: / User-agent: Microsoft URL Control Disallow: / User-agent: Microsoft URL Control - 5.01.4511 Disallow: / User-agent: Microsoft URL Control - 6.00.8169 Disallow: / User-agent: Microsoft.URL Disallow: / User-agent: Mister PiX Disallow: / User-agent: Mister PiX version.dll Disallow: / User-agent: Mister Pix II 2.01 Disallow: / User-agent: Mister Pix II 2.02a Disallow: / User-agent: Mister.PiX Disallow: / User-agent: NICErsPRO Disallow: / User-agent: NPBot Disallow: / User-agent: NPbot Disallow: / User-agent: NearSite Disallow: / User-agent: Navroad Disallow: / User-agent: Net Vampire Disallow: / User-agent: Net Vampire/3.0 Disallow: / User-agent: NetAnts Disallow: / User-agent: NetAnts/1.10 Disallow: / User-agent: NetAnts/1.23 Disallow: / User-agent: NetAnts/1.25 Disallow: / User-agent: NetAnts/1.24 Disallow: / User-agent: NetMechanic Disallow: / User-agent: NetSpider Disallow: / User-agent: NetZIP Disallow: / User-agent: NetZip Downloader 1.0 Win32(Nov 12 1998) Disallow: / User-agent: NetZip-Downloader/1.0.62 (Win32; Dec 7 1998) Disallow: / User-agent: NetZippy+(http://www.innerprise.net/usp-spider.asp) Disallow: / User-agent: Octopus Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Offline Explorer/1.2 Disallow: / User-agent: Offline Explorer/1.4 Disallow: / User-agent: Offline Explorer/1.6 Disallow: / User-agent: Offline Explorer/1.7 Disallow: / User-agent: Offline Explorer/2.0 Disallow: / User-agent: Offline Explorer/1.9 Disallow: / User-agent: Offline Explorer/2.3 Disallow: / User-agent: Offline Explorer/2.1 Disallow: / User-agent: Offline Explorer/2.4 Disallow: / User-agent: Offline Explorer/2.5 Disallow: / User-agent: Offline Navigator Disallow: / User-agent: Offline.Explorer Disallow: / User-agent: Openbot Disallow: / User-agent: Openfind Disallow: / User-agent: Openfind data gatherer Disallow: / User-agent: Oracle Ultra Search Disallow: / User-agent: PageGrabber Disallow: / User-agent: Papa Foto Disallow: / User-agent: PerMan Disallow: / User-agent: ProPowerBot/2.14 Disallow: / User-agent: ProWebWalker Disallow: / User-agent: Python-urllib Disallow: / User-agent: QueryN Metasearch Disallow: / User-agent: QueryN.Metasearch Disallow: / User-agent: RMA Disallow: / User-agent: Radiation Retriever 1.1 Disallow: / User-agent: ReGet Disallow: / User-agent: RealDownload Disallow: / User-agent: RealDownload/4.0.0.40 Disallow: / User-agent: RealDownload/4.0.0.41 Disallow: / User-agent: RealDownload/4.0.0.42 Disallow: / User-agent: RepoMonkey Disallow: / User-agent: RepoMonkey Bait & Tackle/v1.01 Disallow: / User-agent: SlySearch Disallow: / User-agent: SiteSnagger Disallow: / User-agent: SmartDownload Disallow: / User-agent: SmartDownload/1.2.76 (Win32; Apr 1 1999) Disallow: / User-agent: SmartDownload/1.2.77 (Win32; Aug 17 1999) Disallow: / User-agent: SmartDownload/1.2.77 (Win32; Feb 1 2000) Disallow: / User-agent: SmartDownload/1.2.77 (Win32; Jun 19 2001) Disallow: / User-agent: SpankBot Disallow: / User-agent: Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux Disallow: / User-agent: SuperBot Disallow: / User-agent: SuperBot/3.0 (Win32) Disallow: / User-agent: SuperBot/3.1 (Win32) Disallow: / User-agent: SuperHTTP Disallow: / User-agent: SuperHTTP/1.0 Disallow: / User-agent: Surfbot Disallow: / User-agent: Szukacz/1.4 Disallow: / User-agent: Teleport Disallow: / User-agent: Teleport Pro Disallow: / User-agent: Teleport Pro/1.29 Disallow: / User-agent: Teleport Pro/1.29.1590 Disallow: / User-agent: Teleport Pro/1.29.1634 Disallow: / User-agent: Teleport Pro/1.29.1718 Disallow: / User-agent: Teleport Pro/1.29.1820 Disallow: / User-agent: Teleport Pro/1.29.1847 Disallow: / User-agent: TeleportPro Disallow: / User-agent: Telesoft Disallow: / User-agent: The Intraformant Disallow: / User-agent: The.Intraformant Disallow: / User-agent: TheNomad Disallow: / User-agent: TightTwatBot Disallow: / User-agent: Titan Disallow: / User-agent: True_Robot Disallow: / User-agent: True_Robot/1.0 Disallow: / User-agent: TurnitinBot Disallow: / User-agent: TurnitinBot/1.5 Disallow: / User-agent: URL Control Disallow: / User-agent: URL_Spider_Pro Disallow: / User-agent: URLy Warning Disallow: / User-agent: URLy.Warning Disallow: / User-agent: VCI Disallow: / User-agent: VCI WebViewer VCI WebViewer Win32 Disallow: / User-agent: VoidEYE Disallow: / User-agent: WWW-Collector-E Disallow: / User-agent: WWWOFFLE Disallow: / User-agent: Web Image Collector Disallow: / User-agent: Web Sucker Disallow: / User-agent: Web.Image.Collector Disallow: / User-agent: WebAuto Disallow: / User-agent: WebAuto/3.40 (Win98; I) Disallow: / User-agent: WebBandit Disallow: / User-agent: WebBandit/3.50 Disallow: / User-agent: WebCapture 2.0 Disallow: / User-agent: WebCopier Disallow: / User-agent: WebCopier v.2.2 Disallow: / User-agent: WebCopier v2.5 Disallow: / User-agent: WebCopier v2.6 Disallow: / User-agent: WebCopier v2.7a Disallow: / User-agent: WebCopier v2.8 Disallow: / User-agent: WebCopier v3.0 Disallow: / User-agent: WebCopier v3.0.1 Disallow: / User-agent: WebCopier v3.2 Disallow: / User-agent: WebCopier v3.2a Disallow: / User-agent: WebEMailExtrac.* Disallow: / User-agent: WebEnhancer Disallow: / User-agent: WebGo IS Disallow: / User-agent: WebLeacher Disallow: / User-agent: WebReaper Disallow: / User-agent: WebReaper [info@webreaper.net] Disallow: / User-agent: WebReaper [webreaper@otway.com] Disallow: / User-agent: WebReaper v9.7 - www.webreaper.net Disallow: / User-agent: WebReaper v9.1 - www.otway.com/webreaper Disallow: / User-agent: WebFetch Disallow: / User-agent: WebGo IS Disallow: / User-agent: WebLeacher Disallow: / User-agent: WebReaper Disallow: / User-agent: WebReaper [info@webreaper.net] Disallow: / User-agent: WebReaper [webreaper@otway.com] Disallow: / User-agent: WebReaper v9.1 - www.otway.com/webreaper Disallow: / User-agent: WebReaper v9.7 - www.webreaper.net Disallow: / User-agent: WebReaper v9.8 - www.webreaper.net Disallow: / User-agent: WebReaper vWebReaper v7.3 - www,otway.com/webreaper Disallow: / User-agent: WebSauger Disallow: / User-agent: WebSauger 1.20b Disallow: / User-agent: WebSauger 1.20j Disallow: / User-agent: WebSauger 1.20k Disallow: / User-agent: WebStripper Disallow: / User-agent: WebStripper/2.03 Disallow: / User-agent: WebStripper/2.10 Disallow: / User-agent: WebStripper/2.12 Disallow: / User-agent: WebStripper/2.13 Disallow: / User-agent: WebStripper/2.15 Disallow: / User-agent: WebStripper/2.16 Disallow: / User-agent: WebStripper/2.19 Disallow: / User-agent: WebWhacker Disallow: / User-agent: WebZIP Disallow: / User-agent: WebZIP/3.65 (http://www.spidersoft.com) Disallow: / User-agent: WebZIP/2.75 (http://www.spidersoft.com) Disallow: / User-agent: WebZIP/3.80 (http://www.spidersoft.com) Disallow: / User-agent: WebZIP/4.0 (http://www.spidersoft.com) Disallow: / User-agent: WebZIP/4.21 Disallow: / User-agent: WebZIP/4.1 (http://www.spidersoft.com) Disallow: / User-agent: WebZIP/4.21 (http://www.spidersoft.com) Disallow: / User-agent: WebZIP/5.0 Disallow: / User-agent: WebZIP/5.0 (http://www.spidersoft.com) Disallow: / User-agent: WebZIP/5.0 PR1 (http://www.spidersoft.com) Disallow: / User-agent: WebZip Disallow: / User-agent: WebmasterWorldForumBot Disallow: / User-agent: Website Quester Disallow: / User-agent: WebZip/4.0 Disallow: / User-agent: Website Quester - www.asona.org Disallow: / User-agent: Website Quester - www.esalesbiz.com/extra/ Disallow: / User-agent: Website eXtractor Disallow: / User-agent: Website eXtractor (http://www.asona.org) Disallow: / User-agent: Website.Quester Disallow: / User-agent: Webster Pro Disallow: / User-agent: Webster.Pro Disallow: / User-agent: Wget Disallow: / User-agent: Wget/1.5.2 Disallow: / User-agent: Wget/1.5.3 Disallow: / User-agent: Wget/1.6 Disallow: / User-agent: Wget/1.7 Disallow: / User-agent: Wget/1.8 Disallow: / User-agent: Wget/1.8.1 Disallow: / User-agent: Wget/1.8.1+cvs Disallow: / User-agent: Wget/1.8.2 Disallow: / User-agent: Wget/1.9-beta Disallow: / User-agent: Widow Disallow: / User-agent: Xaldon WebSpider Disallow: / User-agent: Xaldon WebSpider 2.5.b3 Disallow: / User-agent: Zeus Disallow: / User-agent: Zeus 11389 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 11652 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 18018 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 26378 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 30747 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 32297 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 39206 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 41641 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 44238 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 51070 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 51674 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 51837 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 63567 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 6694 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 71129 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 82016 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 82900 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 84842 Webster Pro V2.9 Win32 Disallow: / User-agent: Zeus 90872 Webster Pro V2.9 Win32 Disallow: / User-agent: asterias Disallow: / User-agent: b2w/0.1 Disallow: / User-agent: cosmos Disallow: / User-agent: eCatch Disallow: / User-agent: eCatch/3.0 Disallow: / User-agent: hloader Disallow: / User-agent: httplib Disallow: / User-agent: humanlinks Disallow: / User-agent: ia_archiver Disallow: / User-agent: larbin Disallow: / User-agent: larbin (samualt9@bigfoot.com) Disallow: / User-agent: larbin samualt9@bigfoot.com Disallow: / User-agent: larbin_2.6.2 (kabura@sushi.com) Disallow: / User-agent: larbin_2.6.2 (larbin2.6.2@unspecified.mail) Disallow: / User-agent: larbin_2.6.2 (listonATccDOTgatechDOTedu) Disallow: / User-agent: larbin_2.6.2 (vitalbox1@hotmail.com) Disallow: / User-agent: larbin_2.6.2 kabura@sushi.com Disallow: / User-agent: larbin_2.6.2 larbin2.6.2@unspecified.mail Disallow: / User-agent: larbin_2.6.2 larbin@correa.org Disallow: / User-agent: larbin_2.6.2 listonATccDOTgatechDOTedu Disallow: / User-agent: larbin_2.6.2 vitalbox1@hotmail.com Disallow: / User-agent: libWeb/clsHTTP Disallow: / User-agent: lwp-trivial Disallow: / User-agent: lwp-trivial/1.34 Disallow: / User-agent: moget Disallow: / User-agent: moget/2.1 Disallow: / User-agent: pavuk Disallow: / User-agent: pcBrowser Disallow: / User-agent: psbot Disallow: / User-agent: searchpreview Disallow: / User-agent: spanner Disallow: / User-agent: suzuran Disallow: / User-agent: tAkeOut Disallow: / User-agent: toCrawl/UrlDispatcher Disallow: / User-agent: turingos Disallow: / User-agent: webfetch/2.1.0 Disallow: / User-agent: wget Disallow: / allow: / User-agent: spanner Disallow: / User-agent: suzuran Disallow: / User-agent: tAkeOut Disallow: / User-agent: toCrawl/UrlDispatcher Disallow: / User-agent: turingos Disallow: / User-agent: webfetch/2.1.0 Disallow: / User-agent: wget Disallow: /