Search and you will find

In the early days of the web, Tim Berners-Lee maintained a list of websites on the CERN webserver. This was fine at the beginning of 1992, when there were only 26 websites. By the end of ‘94 there were more than 10,000.

The first search engine was developed before the web. ‘Archie’, short for archive, searched FTP sites on the Internet.

Archie

It was created by Alan Emtage (who I was delighted to discover went to school in Barbados with my neighbour) at McGill University Montreal in 1989. Of his world-changing invention he says, “I wrote a piece of code that gave birth to a multibillion-dollar industry. I didn’t make any money off of it, but I wouldn’t change anything.” Hmmmm, are you sure about that Alan?

Alan Emtage, inventor of the search engine

Other search engines followed but it wasn’t until 1995 that an engine emerged that crawled, indexed and ranked websites. By 1997, AltaVista was the most popular page on the Web and was handling more than twenty million queries a day.

A number of other search engines followed the AltaVista model. Excite, Infoseek, Lycos and Magellan (later purchased by Excite) all returned results based on keywords. By 1998 there were millions of websites. Search results deteriorated. To maintain quality, Yahoo! bucked the trend and used people to create their search directory.

A couple of Stanford students, Sergey Brin and Larry Page, saw an opportunity. Yahoo!’s model was unsustainable and relying on keywords wasn’t accurate enough but how else to measure it? They took their inspiration from the academic world they lived in. The value of an academic paper is judged by peer approval, or how many times the paper is referenced. On paper, those references appear as footnotes. On the web, they appear as links. They came up with the PageRank algorithm, ranking results based on the number of incoming links rather than the number of keywords. A beta site, run from a friend’s garage, instantly began returning better results than commercial search engines. The future of the web had arrived. It was called Google.

This is an extract from 100 Ideas that Changed the Web, available on Amazon UK and Amazon US.