A Brief, Incomplete, and Mostly Wrong History of Search Engines

An attempt to create a friendly, fairly comprehensive, history of search engines with a slight emphasis on web search.

Contributions Welcome

Contributions are appreciated. Please submit a pull request or open an issue.

Individuals in Search History

• See People of Search.

1990-1999

• 1992 - Tim Berners-Lee founds the The WWW Virtual Library (VLib), an early web directory.
  • It was curated by a group of experts in their respective fields.
• 1993 - May - O'Reilly Media launches Global Network Navigator (GNN), a web directory.
  • 1996 - AOL acquires and closes GNN.
• 1993 - June - Graham Spencer, Joe Kraus, Mark VanHaren, Ryan McIntyre, Ben Lutch, and Martin Reinfried work on Architext which will eventually become the search engine Excite.
• 1993 - September - Oscar Nierstrasz creates the first web search engine, W3Catalog.
  • Used existing lists of websites, did not crawl. Was too aggressive in its accessing of sites and caused performance slowdowns.
• 1993 - October/November - Martijn Koster creates Aliweb.
  • No crawler, requires sites to be submitted to the engine for inclusion.
• 1993 - December - Jonathan Fletcher creates JumpStation which utilizes a crawler, indexer, and search interface - the three components that make up modern web search.
  • 1994 - Unable to secure funds for the search engine, development is discontinued.
• 1994 - January - EINet Galaxy, a web directory, launches.
• 1994 - January - Jerry Yang and David Filo co-found Yahoo Directory.
  • Includes human created descriptions of each site.
  • 1995 - Yahoo Search is launched, allows searching of the directory, does not utilize a crawler.
• 1994 - January - Infoseek, founded by Steve Kirsch launches.
  • 1995 - Netscape makes Infoseek the default search engine in their browser.
  • 1999 - Disney acquires Infoseek.
  • 2001 - Disney closes Infoseek.
• 1994 - February - Martijn Koster proposes the Robots Exclusion Standard.
• 1994 - March - Oliver McBryan's World-Wide Web Worm is released.
  • Supports use of regular expressions in queries.
  • 1997 - Acquired by GoTo.com.
• 1994 - April - Brian Pinkerton's WebCrawler is released.
  • Allows querying to return results from the full-text of pages.
  • 1995 - AOL acquires WebCrawler.
  • 1997 - Excite acquires WebCrawler from AOL.
  • 2001 - Excite changes WebCrawler from using its own engine to pulling Excite's results.
• 1994 - July - Michael Loren Mauldin's Lycos is released.
  • Offered prefix matching and word proximity in ranking.
  • Origins at Carnegie Mellon University.
  • 2001 - Lycos switches from internal search index to using Fast.
• 1995 - Microsoft releases MSN.
• 1995 - MetaCrawler launches.
  • Created by Aaron Collins and Oren Etzioni at University of Washington.
  • 2000 - Acquired by InfoSpace.
  • 2014 - Is merged with Zoo.
  • 2017 - Relaunches as own search engine.
• 1995 - LookSmart is released as a web directory.
  • Founded by Evan Thornley and Tracy Ellery.
  • 1999 - Microsoft agrees to use LookSmart results for five years.
  • 2000 - Both Juno Online Services and AltaVista begin using LookSmart.
  • 2000 - October - LookSmart acquires Zeal, a web directory.
  • 2002 - LookSmart begins charging per click for sites in its directory, rapidly loses popularity.
  • 2002 - LookSmart acquires WiseNut, a search engine.
  • 2003 - October - Microsoft will not renew its agreement with LookSmart.
• 1995 - October - Excite officially launches.
  • 2001 - October - Excite files for bankruptcy, InfoSpace will eventually acquire Excite.
• 1995 - December - Altavista is launched.
  • Allows natural language queries, ability to add/delete one's own sites within 24 hours.
  • 2013 - Yahoo closes AltaVista.
• 1996 - Robin Li create the RankDex site-scoring algorithm.
  • It used hyperlinks to rank sites.
• 1996 - March - Naveen Jain founds InfoSpace.
• 1996 - Excite acquires Magellan.
• 1996 - Dogpile launches as a meta search engine.
• 1996 - January-March - Larry Page and Sergey Brin begin work on Google's predecessor, BackRub.
  • 1998 - Google search becomes available at google.com.
• 1996 - May - Wired releases its HotBot search engine.
• 1996 - October - Gary Culliss and Steven Yang work at MIT on a "popularity engine" which becomes Direct Hit Technologies search engine.
  • Uses previous search selection stats to determine rank.
  • 1998 - August - Direct Hit Technologies releases their search engine.
• 1996 - MetaGer is lanched by Dr. Wolfgang Sander-Beuermann as a research project at the University of Hanover and the regional computing center of Lower Saxony.
• 1997 - FAST is founded, Arne Halaas and John M. Lervik are two of the core contributors, the latter becomes CEO.
  • 2003 - FAST sells its search related business to Overture.
• 1997 - April - Ask Jeeves (Ask.com) is released.
  • Uses natural language for queries, ranks links by popularity.
  • Also used human editors to improve search rsults.
  • 2005 - March - IAC acquires Ask Jeeves.
• 1997 - September - Arkady Volozh and Ilya Segalovich launch Yandex, a Russion search engine.
• 1998 - Jan-Willem Tusveld launches Vinden.nl, internationally will become ZapMeta.
• 1998 - Bill Gross launches GoTo, which will become Overture.
  • 2003 - Yahoo buys Overture.
• 1998 - David Bodnick founds Ixquick.
  • 2016 - Ixquick is merged with Startpage, a sibling project.
• 1998 - June - Rich Skrenta and Bob Truel launch Gnuhoo which is later named the Open Directory Project/DMoZ.
  • Run by volunteers.
  • 1998 - November - Netscape acquires ODP/DMoZ.
• 1998 - July-September - MSN Search launches.
  • Uses Inktomi for its results, eventually changes name to Bing.
• 1998 - Looksmart acquires Zeal, a web directory.
• 1999 - May - Tor Egge launches AlltheWeb.
• 1999 - Bo Shu and Subhash Kak launch meta search engine Anvish.
  • Later merged with Solosearch.
• 1999 - Entireweb launches.
• 1999 - Doug Cutting writes Lucene, a popular open source search engine.
  • 2001 - Lucene joins Apache Software Foundation.
  • 2005 - Lucene becomes top-level project at Apache.
  • 2010 - Apache Solr becomes a sub-project of Lucene.

2000-2009

• 2000 - Francois Bourdoncle and Patrice Bertin found Exalead.
• 2000 - OpenCOLA, distributed search engine, launches.
• 2000 - April - Gene Kan and Steve Waterhouse launch InfraSearch, a distributed search engine.
  • It is acquired by Sun Microsystems.
• 2000 - Matt Wells founds Gigablast.
• 2000 - January - Robin Li and Li Hanyong launch Baidu, a Chinese search engine.
• 2000 - Google launches AdWords and Google Toolbar.
• 2000 - Teoma search engine is released.
  • Uses clustering to determine subject specific popularity.
• 2001 - Ask Jeeves acquires Teoma.
• 2001 - May - Magellan is closed down.
• 2003 - December - Michael Christen launches what will eventually become YaCy, a distributed search engine.
• 2003 - Amazon launches A9.com.
  • The technology behind Amazon.com
  • 2004 - A9 launches a web search engine.
  • 2008 - Web Search Engine is discontinued.
• 2002 - Zoeken.nl is launched, internationally will become ZapMeta.
• 2003 - Google launches AdSense.
• 2003 - Overture Services buys AltaVista and AlltheWeb.
• 2003 - Yahoo begins using its own crawler (Yahoo Slurp).
  • Previously, Yahoo! was using Google.
  • 2003 - Yahoo acquires Inktomi.
• 2003 - Doug Cutting and Mike Cafarella create Nutch, a popular open source web crawler.
  • From this will come MapReduce and Hadoop.
  • 2005 - Nutch joins Apache Incubator.
  • 2010 - Nutch becomes top level project at Apache Software Foundation.
  • 2014 - Common Crawl begins using Nutch for its crawls.
• 2003 - Mozdex is founded by Byron Miller. It is built on Apache Nutch and Lucene and uses the DMoZ directory as its seed list.
  • 2011 - Mozdex ceases to exist.
• 2004 - Marc Smith at the Sussex Innovation Centre launches Mojeek.
• 2004 - Vivisimo launches Clusty.
  • Started at Carnegie Mellon University.
  • IBM acquires Clusty.
  • 2010 - Clusty sold again to company now named Yippy.
• 2004 - Ask Jeeves acquires Interactive Search Holdings.
• 2004 - iZito becomes the international version of Vinden/Zoeken.
• 2004 - December - Google Suggest becomes a Google Labs feature.
• 2004 - Dr. Amanda Spink and Dr. Bernard J. Jansen* publish Web Search: Pubic Searching of the Web (Kluger Academic Publishers).
• 2004 - 2005 - MSN Search begins using its own indexer and crawler.
  • Previously, MSN was powered by LookSmart and Inktomi.
• 2005 - ZapMeta becomes another international version of Vinden/Zoeken.
• 2005 - January - nofollow attribute for links is introduced by Google, Yahoo!, and Microsoft to reduce spam.
• 2005 - Wolf Garbe launches Faroo, a distributed search engine.
• 2005 - October - Bill Gross launches Snap.
  • Offered detailed display of search volume.
  • Had sophisticated auto-completion, related terms surfacing.
• 2005 - December - Yahoo buys Del.icio.us.
• 2006 - Looksmart closes Zeal.
• 2006 - Powerset (co-founded by Barney Pell, Steve Newcomb, Lorenzo Thione) is developing a natural language search engine, it was based on research from Xerox PARC.
  • 2008 - July - Microsoft acquires Powerset.
• 2007 - Sundar Kundayam launches Zakta, a collaborative search engine.
  • As of 12/2019 the original Zakta collaborative search (now known as zGuides) is down "for maintenance."
  • Zakta Marketing is a paid offering.
• 2007 - December - Wikia Search becomes available in private pre-alpha.
  • 2009 - May - Wikia Search closes.
• 2008 - Google launches SearchWiki.
  • 2010 - Google shuts down SearchWiki.
• 2008 - January - Cuil is launched.
  • 2010 - September - Cuil closes.
• 2008 - September - DuckDuckGo launches.
  • Privacy-centric.
• 2008 - September - Jumper (later renamed to ApexKB), a collaborative search / knowledge management engine is announced.
  • 2009 - March - Stable release of Jumper is made available on SourceForge.
  • 2010 - November - Last release of Jumper is made available on SourceForge.
  • 2011 - September - Jumper Networks Inc closes.
• 2009 - June - Microsoft launches Bing.
• 2009 - December - Ecosia launches.

2010-2019

• 2010 - Shay Banon releases Elasticsearch under an Apache license.
• 2010 - September - Google launches Google Instant.
• 2010 - November - Blekko launches.
  • Uses slashtags to allow searches to filter results.
  • 2015 - IBM acquires Blekko and shuts it down.
• 2011 - Launch of SearchTeam, a collaborative search engine from Zakta.
  • Does not appear fully functional as of 12/2019.
• 2011 - June - Google, Yahoo, and Microsoft announced Schema.org.
  • Supports additional tags that sites can use to convey information to engines better.
• 2012 - January - MetaGer is transferred from the University of Hanover to SUMA-EV.
• 2012 - January - Google launches Search Plus Your World which integrates social data into one's query results.
• 2012 - May - Microsoft adds social data functionality to Bing.
• 2013 - July - Jean-Manuel Rozan, Eric Leandri, and Patrick Constant's Qwant is launched.
• 2013 - August - MetaGer launches English version.
  • 2013 - December - MetaGer integrates Tor into its offering.
  • 2014 - March - MetaGer integrates anonymous proxy into its offering.
• 2012 - May - Google releases Knowledge Graph.
• 2014 - Yahoo closes Yahoo! Directory.
• 2014 - Elastic begins offering commercial addons (later combined into a single package, X-Pack).
• 2015 - Amazon launches Amazon Elasticsearch Service and is sued by Elastic.
• 2016 - MetaGer releases it's source code under the GNU AGPL.
• 2017 - March - ODP/DMoZ is shut down.
• 2019 - Amazon launches Open Distro for Elasticsearch, similar to X-Pack.
• 2021 - Elastic discontinues Apache-licensed Elasticsearch, offering Elasticsearch under the more restrictive SSPL and Elastic License.
• 2021 - Amazon responds by forking the last Apache-licensed version of Elasticsearch under the name OpenSearch.

Information To Add

• Daniel Dreilinger at Colorado State University first to build a meta search engine.
• Eric Selberg at University of Washington improves on meta search engine.
• Inktomi

Bibliography

We've attempted to be somewhat comprehensive in our bibliography. Resources we found particularly helpful are marked with an *.

We should also acknowledge our indebtedness to James Iry for the title of this repository, a reference to their article A Brief, Incomplete, and Mostly Wrong History of Programming Languages.

General

• Joel Lee. 7 Search Engines That Rocked Before Google Even Exists. Mashable, 2015.
• Aaron Wall. Search Engine History. 2017.*
• Wikipedia
  • Search Engine*
  • Timeline of Web Search Engines*
  • Timeline of Google Search*
• Yazdan Mansourian. Book Review: Web Search-Public Searching of the Web. Webology, Vol. 1, No. 2 Dec. 2004.

Specific Search Engines / Technologies

• A9.com
• ApexKB
• Distributed Search Engine
• Ecosia
• Elasticsearch and OpenSearch
  • Radu Gheorghe. OpenSearch vs Elasticsearch: Which One Is Better to Use?. 3/2023.
• Exalead
• Excite
• Gigablast
• InfoSpace
• Inktomi
• JumpStation
• LookSmart
• Lucene
• MetaCrawler
• MetaGer
• MetaGer - About Us
• Microsoft Development Center Norway (FAST)
• Mojeek
• Mozdex
  • 2005 Snapshot
  • 2011 Snapshot
• Nutch
• Powerset
• Robots Exclusion Standard
• SearchTeam
• Startpage
• Qwant
• WebCrawler
• World Wide Web Worm
• YaCy
• Yippy
• Company Information and History. ZapMeta.