List of all Crawlers

008

008 is the user-agent used by 80legs, a web crawling service provider. 80legs allows its users to design and run custom web crawls.

Click on any string to get more details

008 0.83

• Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html) Gecko/2008032620

ABACHOBot

Abacho's spider. German based portal and search engine. Has localized versions in the following countries: Austria, Switzerland, France, UK, Spain, Italy, Sweden and Turkey.

Click on any string to get more details

ABACHOBot

• ABACHOBot

Accoona-AI-Agent

Accoona's webcrawler

Click on any string to get more details

Accoona-AI-Agent 1.1.2

• Accoona-AI-Agent/1.1.2 (aicrawler at accoonabot dot com)
• Accoona-AI-Agent/1.1.2

Accoona-AI-Agent 1.1.1

• Accoona-AI-Agent/1.1.1 (crawler at accoona dot com)

AddSugarSpiderBot

Click on any string to get more details

AddSugarSpiderBot

• Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0) AddSugarSpiderBot www.idealobserver.com

AnyApexBot

Crawler for the web directory AnyApex

Click on any string to get more details

AnyApexBot 1.0

• Mozilla/5.0 (compatible; AnyApexBot/1.0; +http://www.anyapex.com/bot.html)

Arachmo

Japanese Crawler. Seems to be a download tool. Here's some information in japanese. If you can translate than, please let me know

Click on any string to get more details

Arachmo

• Mozilla/4.0 (compatible; Arachmo)

B-l-i-t-z-B-O-T

Crawler for the German search engine tricus. Spiders German, Dutch, Swiss and Austrian websites. Same as BlitzBOT

Click on any string to get more details

B-l-i-t-z-B-O-T

• Mozilla/4.0 (compatible; B-l-i-t-z-B-O-T)

Baiduspider

Crawler for the chinese search engine Baidu

Click on any string to get more details

Baiduspider 2.0

• Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Baiduspider

• Baiduspider+(+http://www.baidu.com/search/spider_jp.html)
• Baiduspider+(+http://www.baidu.com/search/spider.htm)
• BaiDuSpider

BecomeBot

Become crawler. Shopping related portal

Click on any string to get more details

BecomeBot 3.0

• Mozilla/5.0 (compatible; BecomeBot/3.0; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

BecomeBot 2.3

• Mozilla/5.0 (compatible; BecomeBot/2.3; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

BeslistBot

Dutch shopping portal

Click on any string to get more details

BeslistBot 1.0

• Mozilla/5.0 (compatible; BeslistBot; nl; BeslistBot 1.0; http://www.beslist.nl/

BillyBobBot

Click on any string to get more details

BillyBobBot 1.0

• BillyBobBot/1.0 (+http://www.billybobbot.com/crawler/)

Bimbot

Unknown crawler, gives no information. IP address belongs to Backbone Communications Inc. (BBCOM). Provides converged data and voice services

Click on any string to get more details

Bimbot 1.0

• Bimbot/1.0

Bingbot

Bot for Microsofts Bing search engine

Click on any string to get more details

Bingbot 2.0

• Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
• Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)

BlitzBOT

Crawler for the German search engine tricus. Spiders German, Dutch, Swiss and Austrian websites. Same as B-l-i-t-z-B-O-T

Click on any string to get more details

BlitzBOT

• Mozilla/4.0 (compatible; BlitzBot)
• [email protected] (Mozilla compatible)
• [email protected] (Mozilla compatible)

boitho.com-dc

Boitho's Web Crawler, a distributed crawler that downloads web pages to build the database used by Boitho.com to search in. To allow volunteers to donate their superfluous bandwidth and idle CPU time, they have developed a distributed crawler, like seti@home and Grub. That way people can install a program on their computers and help them with the crawling.

Click on any string to get more details

boitho.com-dc 0.85

• boitho.com-dc/0.85 ( http://www.boitho.com/dcbot.html )

boitho.com-dc 0.83

• boitho.com-dc/0.83 ( http://www.boitho.com/dcbot.html )

boitho.com-dc 0.82

• boitho.com-dc/0.82 ( http://www.boitho.com/dcbot.html )

boitho.com-dc 0.81

• boitho.com-dc/0.81 ( http://www.boitho.com/dcbot.html )

boitho.com-dc 0.79

• boitho.com-dc/0.79 ( http://www.boitho.com/dcbot.html )

boitho.com-robot

This is an old version of Boitho's boitho.com-dc. It was a more traditional webrobot, run on computers controlled by Boitho, while boitho.com-dc is a distributed crawler run on the computers of volunteers.
The boitho.com-robot isn抰 in use any more.

Click on any string to get more details

boitho.com-robot 1.1

• boitho.com-robot/1.1

boitho.com-robot 1.0

• boitho.com-robot/1.0

btbot

btbot's search engine for bittorrents, ringtones for cell phones, friends and extraterrestrial intelligence

Click on any string to get more details

btbot 0.4

• btbot/0.4 (+http://www.btbot.com/btbot.html)

CatchBot

Web crawler for Catch, the online division of Reed Business Information Australia

Click on any string to get more details

CatchBot 2.0

• CatchBot/2.0; +http://www.catchbot.com

CatchBot 1.0

• CatchBot/1.0; +http://www.catchbot.com
• CatchBot/1.0; http://www.catchbot.com

Cerberian Drtrs

Click on any string to get more details

Cerberian Drtrs 3.2

• Mozilla/4.0 (compatible; Cerberian Drtrs Version-3.2-Build-1)
• Mozilla/4.0 (compatible; Cerberian Drtrs Version-3.2-Build-0)

Charlotte

Charlotte is a spider created by Searchme, Inc. in Mountain View, CA

Click on any string to get more details

Charlotte 1.1

• Mozilla/5.0 (compatible; Charlotte/1.1; http://www.searchme.com/support/)

Charlotte 1.0t

• Mozilla/5.0 (compatible; Charlotte/1.0t; http://www.searchme.com/support/)

Charlotte 1.0b

• Mozilla/5.0 (compatible; Charlotte/1.0b; http://www.searchme.com/support/)
• Mozilla/5.0 (compatible; Charlotte/1.0b; http://www.betaspider.com/)

Charlotte 0.9t

• Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.11) Gecko/20080109 (Charlotte/0.9t; http://www.searchme.com/support/) (Charlotte/0.9t; http://www.searchme.com/support/)
• Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.11) Gecko/20080109 (Charlotte/0.9t; http://www.searchme.com/support/)
• Mozilla/5.0 (compatible; Charlotte/0.9t; http://www.searchme.com/support/)
• Mozilla/5.0 (compatible; Charlotte/0.9t; +http://www.searchme.com/support/)

ConveraCrawler

ConveraCrawler is an experimental web crawler under development since April 2004. ConveraCrawler is owned and operated by Convera Corporation

Click on any string to get more details

ConveraCrawler 0.9e

• ConveraCrawler/0.9e (+http://ews.converasearch.com/crawl.htm)

ConveraCrawler 0.9d

• ConveraCrawler/0.9d (+http://www.authoritativeweb.com/crawl)
• ConveraCrawler/0.9d ( http://www.authoritativeweb.com/crawl)

ConveraCrawler 0.9

• ConveraCrawler/0.9 (+http://www.authoritativeweb.com/crawl)

cosmos

Crawler from xyleme which indexes XML content on the web.

Click on any string to get more details

cosmos 0.9

• cosmos/0.9_([email protected])

Covario IDS

Proprietary crawler used as part of Covario's Organic Search Insight solution

Click on any string to get more details

Covario IDS 1.0

• Covario-IDS/1.0 (Covario; http://www.covario.com/ids; support at covario dot com)

DataparkSearch

Open source web-based search engine released under the GNU General Public License and designed to organize search within a website, group of websites, intranet or local system. DataparkSearch consists of two parts. The first part is indexing mechanism (indexer). Indexer walks over html hypertext references and stores found words and new references into database. The second part is web CGI front-end to provide search using data collected by indexer.

Click on any string to get more details

DataparkSearch 4.37

• DataparkSearch/4.37-23012006 ( http://www.dataparksearch.org/)

DataparkSearch 4.36

• DataparkSearch/4.36 ( http://www.dataparksearch.org/)

DataparkSearch 4.35

• DataparkSearch/4.35-02122005 ( http://www.dataparksearch.org/)
• DataparkSearch/4.35 ( http://www.dataparksearch.org/)

DiamondBot

Crawler for Claria (formerly Gator). Adware company

Click on any string to get more details

DiamondBot

• DiamondBot

Discobot

Discobot is the experimental web crawler for Discovery Engine

Click on any string to get more details

Discobot 1.0

• Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html)

Dotbot

Click on any string to get more details

Dotbot 1.1

• Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, [email protected])

Dotbot 1.0.1

• DotBot/1.0.1 (http://www.dotnetdotcom.org/#info, [email protected])

EmeraldShield.com WebBot

Crawls domains as part of a spam and web filtration services. If a site is determined to contain questionable, or objectionable content it will be added to a blocklist. Ignores the robots.txt file

Click on any string to get more details

EmeraldShield.com WebBot

• EmeraldShield.com WebBot (http://www.emeraldshield.com/webbot.aspx)

envolk[ITS]spider

envolk search engine spider [ITS] Internet Tracking Spider(TM)

Click on any string to get more details

envolk[ITS]spider 1.6

• envolk[ITS]spider/1.6 (+http://www.envolk.com/envolkspider.html)
• envolk[ITS]spider/1.6 ( http://www.envolk.com/envolkspider.html)

EsperanzaBot

Web Crawler of Esperanza Consulting LTD

Click on any string to get more details

EsperanzaBot

• EsperanzaBot(+http://www.esperanza.to/bot/)

Exabot

Exava shopping search engine, belongs now to Become

Click on any string to get more details

Exabot 2.0

• Exabot/2.0

FAST Enterprise Crawler

Product of the norvegian company Fast. Part of their FAST ProPublish solution for gathering, processing and delivering reference material to online and offline users.

Click on any string to get more details

FAST Enterprise Crawler 6

• FAST Enterprise Crawler 6 used by Schibsted ([email protected])
• FAST Enterprise Crawler 6 / Scirus [email protected]; http://www.scirus.com/srsapp/contactus/
• FAST Enteprise Crawler/6 (www dot fastsearch dot com)

FAST-WebCrawler

Crawler for the Fast search engine

Click on any string to get more details

FAST-WebCrawler 3.8

• FAST-WebCrawler/3.8 (atw-crawler at fast dot no; http://fast.no/support/crawler.asp)

FAST-WebCrawler 3.7

• FAST-WebCrawler/3.7/FirstPage (atw-crawler at fast dot no;http://fast.no/support/crawler.asp)
• FAST-WebCrawler/3.7 (atw-crawler at fast dot no; http://fast.no/support/crawler.asp)

FAST-WebCrawler 3.6

• FAST-WebCrawler/3.6/FirstPage (atw-crawler at fast dot no;http://fast.no/support/crawler.asp)
• FAST-WebCrawler/3.6 (atw-crawler at fast dot no; http://fast.no/support/crawler.asp)
• FAST-WebCrawler/3.6

FAST-WebCrawler 3.x

• FAST-WebCrawler/3.x Multimedia (mm dash crawler at fast dot no)
• FAST-WebCrawler/3.x Multimedia

FDSE robot

Search engine of Fluid Dynamics Software Corporation

Click on any string to get more details

FDSE robot

• Mozilla/4.0 (compatible: FDSE robot)

FindLinks

A project of the Automated Speech Processing Group at the Institute of Computer Science at Universit盲t Leipzig.

Click on any string to get more details

FindLinks 2.0.1

• findlinks/2.0.1 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.6-beta6

• findlinks/1.1.6-beta6 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.6-beta4

• findlinks/1.1.6-beta4 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.6-beta1

• findlinks/1.1.6-beta1 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.5-beta7

• findlinks/1.1.5-beta7 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.4-beta1

• findlinks/1.1.4-beta1 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.3-beta9

• findlinks/1.1.3-beta9 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.3-beta8

• findlinks/1.1.3-beta8 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.3-beta6

• findlinks/1.1.3-beta6 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.3-beta4

• findlinks/1.1.3-beta4 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.3-beta2

• findlinks/1.1.3-beta2 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.3-beta1

• findlinks/1.1.3-beta1 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.2-a5

• findlinks/1.1.2-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.1-a5

• findlinks/1.1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.1-a1

• findlinks/1.1.1-a1 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1.1

• findlinks/1.1.1 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1-a9

• findlinks/1.1-a9 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1-a8

• findlinks/1.1-a8 (+http://wortschatz.uni-leipzig.de/findlinks/)
• findlinks/1.1-a8 ( http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1-a7

• findlinks/1.1-a7 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1-a5

• findlinks/1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1-a4

• findlinks/1.1-a4 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1-a3

• findlinks/1.1-a3 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.1

• findlinks/1.1 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.06

• findlinks/1.06 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.0.9

• findlinks/1.0.9 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.0.8

• findlinks/1.0.8 (+http://wortschatz.uni-leipzig.de/findlinks/)

FindLinks 1.0

• findlinks/1.0 (+http://wortschatz.uni-leipzig.de/findlinks/)

FurlBot

Furl's crawler. Furl is a social bookmark service from LookSmart

Click on any string to get more details

FurlBot Furl Search 2.0

• Mozilla/4.0 compatible FurlBot/Furl Search 2.0 (FurlBot; http://www.furl.net; [email protected])

FyberSpider

FyberSearch web crawler

Click on any string to get more details

FyberSpider

• FyberSpider (+http://www.fybersearch.com/fyberspider.php)
• FyberSpider

g2crawler

g2crawler : Gnutella2Crawler codename Aenea. Not in use anymore.

Click on any string to get more details

g2crawler

• g2Crawler [email protected]

Gaisbot

Gais - Global Area Information Servers - Search enginge crawler of the National Chung Cheng University Taiwan

Click on any string to get more details

Gaisbot 3.0+

• Gaisbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/robot.php)

Gaisbot 3.0

• Gaisbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/robot.php)
• Gaisbot/3.0 ([email protected]; http://gais.cs.ccu.edu.tw/robot.php)

GalaxyBot

Browser for Galaxy Classifieds, a searchable directory.

Click on any string to get more details

GalaxyBot 1.0

• GalaxyBot/1.0 (http://www.galaxy.com/galaxybot.html)

genieBot

Web-indexing robot of GenieKnows Local Search Engine

Click on any string to get more details

genieBot

• genieBot (http://64.5.245.11/faq/faq.html)
• genieBot ((http://64.5.245.11/faq/faq.html))

Gigabot

Gigablast's indexing agent

Click on any string to get more details

Gigabot 3.0

• Gigabot/3.0 (http://www.gigablast.com/spider.html)

Gigabot 2.0

• Gigabot/2.0/gigablast.com/spider.html
• Gigabot/2.0 (http://www.gigablast.com/spider.html)
• Gigabot/2.0

Gigabot 1.0

• Gigabot/1.0

Girafabot

Click on any string to get more details

Girafabot

• Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; Girafabot [girafa.com])
• Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; Girafabot; girafabot at girafa dot com; http://www.girafa.com)
• Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; Girafabot; girafabot at girafa dot com; http://www.girafa.com)

Googlebot

Click on any string to get more details

Googlebot 2.1

• Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
• Googlebot/2.1 (+http://www.googlebot.com/bot.html)
• Googlebot/2.1 (+http://www.google.com/bot.html)

Googlebot-Image

Google's image crawler

Click on any string to get more details

Googlebot-Image 1.0

• Googlebot-Image/1.0

GurujiBot

Indian search engine

Click on any string to get more details

GurujiBot 1.0

• Mozilla/5.0 GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html)
• Mozilla/5.0 GurujiBot/1.0 ( http://www.guruji.com/en/WebmasterFAQ.html)
• Mozilla/5.0 (compatible; GurujiBot/1.0; +http://www.guruji.com/en/WebmasterFAQ.html)
• GurujiBot/1.0 (+http://www.guruji.com/WebmasterFAQ.html)
• GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html)

HappyFunBot

Crawler for Happy Fun Search

Click on any string to get more details

HappyFunBot 1.1

• HappyFunBot/1.1 ( http://www.happyfunsearch.com/bot.html)

hl_ftien_spider

Web Crawler from China. IP addresses belong to Qipusi Technology Ltd and Rongzhengwuye-ltd from Tjanjin city

Click on any string to get more details

hl_ftien_spider 1.1

• hl_ftien_spider_v1.1

hl_ftien_spider

• hl_ftien_spider

Holmes

Sherlock Holmes is a open source universal search engine. The URL can be added by the user. Often used to spam your logfiles

Click on any string to get more details

Holmes 3.9

• holmes/3.9 (someurl.co.cc)

Holmes 3.12.4

• holmes/3.12.4 (http://morfeo.centrum.cz/bot)

Holmes 3.12.3

• holmes/3.12.3 (http://morfeo.centrum.cz/bot)

Holmes 3.12.2

• holmes/3.12.2 (http://morfeo.centrum.cz/bot)

Holmes 3.12.1

• holmes/3.12.1 (http://morfeo.centrum.cz/bot)

htdig

Crawler of the ht://Dig Group's software package, a system for indexing and searching a finite (not necessarily small) set of sites or intranet. It is not meant to replace any of the many internet-wide search engines. htdig retrieves HTML documents using the HTTP protocol.

Click on any string to get more details

htdig 3.1.6

• htdig/3.1.6 ([email protected])
• htdig/3.1.6 ([email protected])

htdig 3.1.5

• htdig/3.1.5 ([email protected])
• htdig/3.1.5 (root@localhost)
• htdig/3.1.5 ([email protected])
• htdig/3.1.5

iaskspider

Bot for iAsk , chinese search engine from Sina.com

Click on any string to get more details

iaskspider 2.0

• iaskspider/2.0(+http://iask.com/help/help_index.html)

iaskspider

• iaskspider

ia_archiver

Alexa Web crawler

Click on any string to get more details

ia_archiver 8.9

• ia_archiver/8.9 (Windows NT 3.1; en-US;)
• ia_archiver/8.9 (Windows 3.9; en-US;)
• ia_archiver/8.9 (Linux 1.0; en-US;)

ia_archiver 8.8

• ia_archiver/8.8 (Windows XP 7.2; en-US;)
• ia_archiver/8.8 (Windows XP 3.0; en-US;)

ia_archiver 8.2

• ia_archiver/8.2 (Windows 7.6; en-US;)

ia_archiver 8.1

• ia_archiver/8.1 (Windows 2000 1.9; en-US;)

ia_archiver 8.0

• ia_archiver/8.0 (Windows 2.4; en-US;)

ia_archiver

• ia_archiver

iCCrawler

ICCrawler is ICCenter's specialized web-crawling robot. Currently they are collecting only job offers from company sites. Those job offers are getting listed at ICjobs

Click on any string to get more details

iCCrawler

• iCCrawler (http://www.iccenter.net/bot.htm)

ichiro

Japanese Webcrawler for Goo

Click on any string to get more details

ichiro 4.0

• ichiro/4.0 (http://help.goo.ne.jp/door/crawler.html)

ichiro 3.0

• ichiro/3.0 (http://help.goo.ne.jp/door/crawler.html)

ichiro 2.0

• ichiro/2.0+(http://help.goo.ne.jp/door/crawler.html)
• ichiro/2.0 ([email protected])
• ichiro/2.0 (http://help.goo.ne.jp/door/crawler.html)

igdeSpyder

Crawler for the russian IGDE commercial search engine

Click on any string to get more details

igdeSpyder

• igdeSpyder (compatible; igde.ru; +http://igde.ru/doc/tech.html)

IRLbot

IRL-crawler is a Texas A&M University research project sponsored in part by the National Science Foundation that investigates algorithms for mapping the topology of the Internet and discovering the various parts of the web. The crawler downloads random web pages (text only) and follows certain links to find other websites.

Click on any string to get more details

IRLbot 3.0

• IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler/)
• IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)

IRLbot 2.0

• IRLbot/2.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)
• IRLbot/2.0 (+http://irl.cs.tamu.edu/crawler)
• IRLbot/2.0 ( http://irl.cs.tamu.edu/crawler)

IssueCrawler

Govcom.org Foundation's web bot. Locates and visualizes networks on the Web. The Issue Crawler is used by NGOs and other researchers to answer questions about specific networks and effective networking more generally. You also may do in-depth research with the software. You need an account to use it.

Click on any string to get more details

IssueCrawler

• IssueCrawler

Jaxified Bot

Click on any string to get more details

Jaxified Bot

• Jaxified Bot (+http://www.jaxified.com/crawler/)

Jyxobot

Czech Webcrawler for Jyxo

Click on any string to get more details

Jyxobot 1

• Jyxobot/1

KoepaBot

Click on any string to get more details

KoepaBot

• Mozilla/5.0 (compatible; KoepaBot BETA; http://www.koepa.nl/bot.html)

L.webis

Crawler developed at the Institute of Informatics and Telematics (IIT), of the National Research Council (CNR) of Italy, in Pisa

Click on any string to get more details

L.webis 0.87

• L.webis/0.87 (http://webalgo.iit.cnr.it/index.php?pg=lwebis)

LapozzBot

Hungarian bot. Spiders for the Lapozz search engine.
躣v鰖l鰉 !?!

Click on any string to get more details

LapozzBot 1.4

• LapozzBot/1.4 (+http://robot.lapozz.com)

Larbin

Multi-purpose web crawler

Click on any string to get more details

Larbin 5.0

• Mozilla/5.0 [email protected]

Larbin 2.6.3

• larbin_2.6.3 [email protected]
• larbin_2.6.3 [email protected]
• larbin_2.6.3 [email protected]
• larbin_2.6.3 [email protected]
• larbin_2.6.3 [email protected]
• larbin_2.6.3 [email protected]
• larbin_2.6.3 [email protected]
• larbin_2.6.3 ([email protected])
• larbin_2.6.3 ([email protected])
• larbin_2.6.3 ([email protected])
• larbin_2.6.3 ([email protected])

Larbin 2.6.2

• larbin_2.6.2 [email protected]
• larbin_2.6.2 [email protected]
• larbin_2.6.2 listonATccDOTgatechDOTedu
• larbin_2.6.2 [email protected]
• larbin_2.6.2 [email protected]
• larbin_2.6.2 [email protected]
• larbin_2.6.2 [email protected]
• larbin_2.6.2 ([email protected])
• larbin_2.6.2 ([email protected])
• larbin_2.6.2 ([email protected])
• larbin_2.6.2 ([email protected])

Larbin 2.6.1

• larbin_2.6.1 [email protected]

Larbin 2.5.0

• larbin_2.5.0 ([email protected])

Larbin xy250

• larbin_xy250 [email protected]

Larbin

• larbin_test [email protected]
• larbin_test ([email protected])
• LARBIN-EXPERIMENTAL [email protected]

LDSpider

LDSpider project aims to build a web crawling framework for the linked data web

Click on any string to get more details

LDSpider

• ldspider (http://code.google.com/p/ldspider/wiki/Robots)

LexxeBot

Bot for Lexxe Search Engine

Click on any string to get more details

LexxeBot 1.0

• LexxeBot/1.0 ([email protected])

Linguee Bot

Search engine for bilingual texts. Helps with translating common phrases into another language

Click on any string to get more details

Linguee Bot

• Linguee Bot (http://www.linguee.com/bot; [email protected])

LinkWalker

SEVENtwentyfour Inc Link Checker

Click on any string to get more details

LinkWalker 2.0

• LinkWalker/2.0

LinkWalker

• LinkWalker

lmspider

Collects text from the web as part of a research project at Scansoft (renamed Nuance) ,trying to use web documents to improve the linguistic models used in their speech recognition engine

Click on any string to get more details

lmspider

• lmspider [email protected]
• lmspider ([email protected])

lwp-trivial

lwp-trivial is the user-agent associated with the Perl code Module LWP::Simple

Click on any string to get more details

lwp-trivial 1.41

• lwp-trivial/1.41

lwp-trivial 1.38

• lwp-trivial/1.38

lwp-trivial 1.36

• lwp-trivial/1.36

lwp-trivial 1.35

• lwp-trivial/1.35

lwp-trivial 1.33

• lwp-trivial/1.33

mabontland

Crawler for the web directory mabontland

Click on any string to get more details

mabontland

• http://www.mabontland.com

magpie-crawler

Crawler for Brandwatch

Click on any string to get more details

magpie-crawler 1.1

• magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net)

Mediapartners-Google

Unregistered versions of opera prior to 8.5 contained advertising. To serve up relevant adverts based on what you are browsing Google provided these adverts.
More information

Click on any string to get more details

Mediapartners-Google 2.1

• Mediapartners-Google/2.1

MJ12bot

Majestic-12 Web Crawler

Click on any string to get more details

MJ12bot 1.2.4

• Mozilla/5.0 (compatible; MJ12bot/v1.2.4; http://www.majestic12.co.uk/bot.php?+)

MJ12bot 1.2.3

• Mozilla/5.0 (compatible; MJ12bot/v1.2.3; http://www.majestic12.co.uk/bot.php?+)

MJ12bot 1.0.8

• MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)

MJ12bot 1.0.7

• MJ12bot/v1.0.7 (http://majestic12.co.uk/bot.php?+)

MJ12bot 1.0.6

• MJ12bot/v1.0.6 (http://majestic12.co.uk/bot.php?+)

MJ12bot 1.0.5

• MJ12bot/v1.0.5 (http://majestic12.co.uk/bot.php?+)

Mnogosearch

Web search engine software for intranet and internet servers from Mnogosearch.org (a project of Lavtech)

Click on any string to get more details

Mnogosearch 3.1.21

• Mnogosearch-3.1.21

mogimogi

Unclear. The IP address belongs to Goo but they don't give any information about that bot. Goo itself uses ichiro for their search engine

Click on any string to get more details

mogimogi 1.0

• mogimogi/1.0

MojeekBot

MojeekBot (formerly Citenikbot) is the web crawler for the Mojeek search engine.

Click on any string to get more details

MojeekBot 2.0

• Mozilla/5.0 (compatible; MojeekBot/2.0; http://www.mojeek.com/bot.html)

MojeekBot 0.2

• MojeekBot/0.2 (archi; http://www.mojeek.com/bot.html)

Moreoverbot

Rssfeed bot

Click on any string to get more details

Moreoverbot 5.1

• Moreoverbot/5.1 ( http://w.moreover.com; [email protected]) Mozilla/5.0

Moreoverbot 5.00

• Moreoverbot/5.00 (+http://www.moreover.com; [email protected])
• Moreoverbot/5.00 (+http://www.moreover.com)

Morning Paper

Crawler for Boutell.com.

Click on any string to get more details

Morning Paper 1.0

• Morning Paper 1.0 (robots.txt compliant!)

msnbot

MSN (or Microsoft Service Network) Search Web Crawler

Click on any string to get more details

msnbot 2.1

• msnbot/2.1

msnbot 2.0b

• msnbot/2.0b

msnbot 1.1

• msnbot/1.1 (+http://search.msn.com/msnbot.htm)
• msnbot/1.1

msnbot 1.0

• msnbot/1.0 (+http://search.msn.com/msnbot.htm)

msnbot 0.9

• msnbot/0.9 (+http://search.msn.com/msnbot.htm)

msnbot 0.11

• msnbot/0.11 ( http://search.msn.com/msnbot.htm)

msnbot 0.1

• MSNBOT/0.1 (http://search.msn.com/msnbot.htm)

MSRBot

Microsoft Research web crawler

Click on any string to get more details

MSRBot

• MSRBOT (http://research.microsoft.com/research/sv/msrbot/)
• MSRBOT

MVAClient

I have no information about this one. The ip address belongs to Chunghwa Telecom Co.,Ltd. in Taiwan. It is blacklisted by SORBS. If you know anything about this bot please let me know

Click on any string to get more details

MVAClient

• MVAClient

mxbot

Crawler for Chainn

Click on any string to get more details

mxbot 1.0

• Mozilla/5.0 (compatible; mxbot/1.0; +http://www.chainn.com/mxbot.html)
• Mozilla/5.0 (compatible; mxbot/1.0; http://www.chainn.com/mxbot.html)

NetResearchServer

Spider for LOOP Improvements. Crawls the web by using the links found in the DMOZ Open Directory Project.

Click on any string to get more details

NetResearchServer 4.0

• NetResearchServer/4.0(loopimprovements.com/robot.html)

NetResearchServer 3.5

• NetResearchServer/3.5(loopimprovements.com/robot.html)

NetResearchServer 2.8

• NetResearchServer/2.8(loopimprovements.com/robot.html)

NetResearchServer 2.7

• NetResearchServer/2.7(loopimprovements.com/robot.html)

NetResearchServer 2.5

• NetResearchServer/2.5(loopimprovements.com/robot.html)

NetResearchServer

• NetResearchServer(http://www.look.com)

NetSeer Crawler

Click on any string to get more details

NetSeer Crawler 2.0

• Mozilla/5.0 (compatible; NetSeer crawler/2.0; +http://www.netseer.com/crawler.html; [email protected])

NewsGator

Click on any string to get more details

NewsGator 2.5

• NewsGator/2.5 (http://www.newsgator.com; Microsoft Windows NT 5.1.2600.0; .NET CLR 1.1.4322.2032)

NewsGator 2.0

• NewsGator/2.0 Bot (http://www.newsgator.com)

NG-Search

NG-Search is experimental searchengine with new semantic trials to list the most relevance words and groups around your query

Click on any string to get more details

NG-Search 0.9.8

• NG-Search/0.9.8 (http://www.ng-search.com)

NG-Search 0.86

• NG-Search/0.86 (+http://www.ng-search.com)
• NG-Search/0.86 ( http://www.ng-search.com)

nicebot

Click on any string to get more details

nicebot

• nicebot

noxtrumbot

Spanish search engine for Spanish and Portuguese pages. Belongs to TPI, Telef髇ica Publicidad e Informaci髇, S.A

Click on any string to get more details

noxtrumbot 1.0

• noxtrumbot/1.0 ([email protected])

Nusearch Spider

Crawls for the Nusearch search engine. Customizable search engine with some additional features like active bookmarks, and alternative result views.

Click on any string to get more details

Nusearch Spider

• Nusearch Spider (www.nusearch.com)
• nuSearch Spider (compatible; MSIE 4.01; Windows NT)

NutchCVS

Open source robot

Click on any string to get more details

NutchCVS 0.8-dev

• NutchCVS/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected])

NutchCVS 0.7.2

• NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected])

NutchCVS 0.7.1

• NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected])
• NutchCVS/0.7.1 (Nutch running at UW; http://crawlers.cs.washington.edu/; [email protected])

NutchCVS 0.7

• NutchCVS/0.7 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected])

NutchCVS 0.06-dev

• NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; [email protected])
• NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; [email protected])

NutchCVS 0.05

• NutchCVS/0.05 (Nutch; http://www.nutch.org/docs/en/bot.html; [email protected])

Nymesis

Click on any string to get more details

Nymesis 1.0

• Nymesis/1.0 (http://nymesis.com)

obot

German spider from Cobion, now part of Internet Security Systems. Scans the web for their clients looking for copyright infringement

Click on any string to get more details

obot

• Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; obot)

oegp

The IP address belongs to the Deutsche Telekom in Germany. They don't give any information about that crawler. IP address is blacklisted

Click on any string to get more details

oegp 1.3.0

• oegp v. 1.3.0

omgilibot

Click on any string to get more details

omgilibot 0.4

• omgilibot/0.4 +http://omgili.com

omgilibot 0.3

• omgilibot/0.3 +http://www.omgili.com/Crawler.html
• omgilibot/0.3 http://www.omgili.com/Crawler.html

OmniExplorer_Bot

New crawler for Omni-Explorer. Site not launched yet (February 06)

Click on any string to get more details

OmniExplorer_Bot 6.70

• OmniExplorer_Bot/6.70 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 6.65a

• OmniExplorer_Bot/6.65a (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 6.63b

• OmniExplorer_Bot/6.63b (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 6.62

• OmniExplorer_Bot/6.62 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 6.60

• OmniExplorer_Bot/6.60 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 6.47

• OmniExplorer_Bot/6.47 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 5.91c

• OmniExplorer_Bot/5.91c (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 5.28

• OmniExplorer_Bot/5.28 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 5.25

• OmniExplorer_Bot/5.25 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 5.20

• OmniExplorer_Bot/5.20 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 5.01

• OmniExplorer_Bot/5.01 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 4.80

• OmniExplorer_Bot/4.80 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot 4.32

• OmniExplorer_Bot/4.32 (+http://www.omni-explorer.com) WorldIndexer

OOZBOT

Click on any string to get more details

OOZBOT 0.20

• OOZBOT/0.20 ( -- ; http://www.setooz.com/oozbot.html ; agentname at setooz dot_com )

OOZBOT 0.17

• OOZBOT/0.17 (--; http://www.setooz.com/oozbot.html; pvvpr at iiit dot ac dot in)

Orbiter

Spider for DailyOrbit search engine. Visits only the homepage of a domain.

Click on any string to get more details

Orbiter

• Orbiter (+http://www.dailyorbit.com/bot.htm)

PageBitesHyperBot

Crawler for PageBites, a search engine for job openings and/or r閟um閟. You can also post your r閟um