It’s 25 years since Tim Berners-Lee proposed the Web and the universal, inclusive space he generously gave away is under threat. To mark the anniversary, the Southbank Centre in London is running the Web We Want Festival, asking people to play their part in shaping the web they want.
I was asked to do a talk on the history of the Web, defining some of the key terms and technologies and plotting the ongoing battle between an Open and Closed Web. Tim Berners-Lee was on first – I was on second…
So how do you follow Tim Berners-Lee at a festival about the Web? You don’t, you play a clip from 1950 of a short story called A Logic Named Joe on Dimension X, a science fiction radio show.
A Logic Named Joe, written by Murray Leinster in 1946, pretty accurately describes the web. The clip ends with a young boy trying to use a Logic to find out how to make dart poison. The information is restricted – even 43 years before the Web was created, the issues of censorship, regulation and control were already being considered.
Wikipedia, a champion of open standards gets the top slot but two kids making dart poison out of sun screen and hand sanitizer come in at number two. The third result is from a 90-year old who used to lead exhibitions to the Amazon, the fourth is an educational resource on native American people and the fifth is an academic article from Princeton University Press. The web truly is for everyone – and that’s what’s so special about it.
The Open Web – The thing underlying the success of the Web is inclusiveness. Anyone can make a webpage. Any content can be put on a webpage. Any webpage can link to any other. A webpage can be viewed at any screen size and any connection speed, from any device connected to the internet. Anybody can access the Web from anywhere.
Tim Berners-Lee built the web on existing technologies. As Sir Tim himself said ‘I just had to take the hypertext idea and connect it to the Transmission Control Protocol and domain name system ideas and ta-da! The World Wide Web.’ Easy.
OPEN WEB 1 – CLOSED WEB 0
Hypertext – it’s difficult for us to understand but until the middle of the last century, conditioned by 500 years of print, the type of non-linear narratives made possible by hypertext were unimaginable to most. One of the guy’s who could imagine it was Vannevar Bush, Director of the Manhattan Project that gave us the atomic bomb. Disillusioned with the pursuit of power of progress, he put his mind to discovering how the world’s knowledge could be harnessed for the common good. The result was his seminal essay, As We May Think, published in The Atlantic magazine in 1945. In it, he described the Memex, a device that could store every book and record created, each indexed and linked.
OPEN WEB 2 – CLOSED WEB 0
Transmission Control Protocol – The thing that makes the Internet work is packet switching, dividing messages into pieces, sending them via multiple paths and reassembling them on arrival. In the early 1970’s, several packet-switching networks existed, starting with ARPANET, a by-product of the Space Race. Robert Khan and Vinton Cerf had the idea of joining them up. To do this, they would need a common set of rules, these rules are Transmission Control Protocol. One of the drivers of TCP is a guy called John Postel. The behaviour he encouraged seems wise beyond sending content across computer networks ‘Be conservative in what you do, be liberal in what you accept from others’.
OPEN WEB 3 – CLOSED WEB 0
The Web and the Internet are often mixed up. The Internet is a global system of interconnected computer networks. It is the infrastructure that carries email, Instant Messages, Voice over IP, network games, file transfer and of course the Web. The Web is the most accessible component of the Internet, an interlinked system of hypertext documents. When I think of the relationship between the Internet and the Web, I think of the relationship between the tin can and the can opener, the can opener arriving forty or so years after the tin can.
Internet Service Provider – As the Internet became publicly available, Internet Service Providers appeared. CompuServe began offering a dial-up information service in 1979. It got its first serious competitor in 1985, a service for Commodore 64 owners appeared called Q-Link. In 1989, Q-Link extended its service to IBM-compatible PCs and changed its name to America Online. With its user-friendly interface and aggressive marketing it quickly became the dominant ISP. CompuServe and AOL both offered decent services but once the web launched they only allowed access to certain parts of the web – they operated walled gardens.
OPEN WEB 3 – CLOSED WEB 1
Domain Name System – Introduced in 1985 by Paul Mockapetris, DNS translates numerical Internet Protocol addresses into easily memorized domain names. In the first two years, fewer than 100 companies registered their domains but the birth of the Web created a domain name gold rush. The most expensive domains ever bought being insure.com ($16 million) and sex.com ($14 million) – which gives you some indication of where our priorities lie.
OPEN WEB 3 – CLOSED WEB 2
Uniform Resource Locators (URLS) – Initially, the Web was far from worldwide. It was only accessible by people who had a NeXT computer. But by the end of 1992 the Web was global. Its popularity was due to its tolerance of incoming links. It was, and still is, easy to link to another page. The use of one-way links drew criticism from Ted Nelson who said ‘Ever-breaking links and links going outward were precisely what we were trying to prevent’ but an easy to use, universal and tolerant system has got to be a good thing.
OPEN WEB 4 – CLOSED WEB 2
Markup Language – another key component of the Web is HTML – Hypertext Markup Language. Markup Language is the computerised equivalent of typesetting, it evolved from the marking up of paper manuscripts by editors to show how they’d like the page laid out.Originally based on only 18 basic tags, the simplicity of HTML was a major factor in the success of the Web. Anyone could make a web page. Including me.
OPEN WEB 5 – CLOSED WEB 2
Web Browser – When Tim Berners-Lee created the first webpage, we often forget that he also created the first web browser.The crucial point about his browser was that it was a browser-editor. Not only did it allow visitors to view a page, but also it allowed them to edit it too. The Web was envisaged as a multi-author environment from the start.
OPEN WEB 6 – CLOSED WEB 2
The first browser only ran on the NeXTstep operating system. Nicola Pellow, an intern at CERN, create a cross-platform, read-only browser, which helped spread the web. On the downside, browsers, have been passive ever since.
OPEN WEB 6 – CLOSED WEB 3
In 1993, Marc Andreessen and Eric Bina created the Mosaic browser, a first cross-platform, point-and-click browser, described by the New York Times as ‘a map to the buried treasures of the Information Age’. Developed at the National Center for Supercomputing Applications (NCSA) at the University of Illinois, it quickly became the Web’s most popular browser. Marc Andreessen saw an opportunity and left NCSA to form Netscape Communications.
OPEN WEB 6 – CLOSED WEB 4
Web Server – the other half of the client-server equation is the web server. Developed at the University of Illinois alongside the Mosaic browser, was the NCSA Web Server, led by an undergraduate called Rob McCool. The NCSA Web Server was not perfect – its password functionality needed re-writing. So Brian Behlendorf, webmaster at Hotwired.com, Wired Magazine’s website, wrote a patch that improved the password functionality.
Previously, he would have sent the patch to Rob McCool but along with most of the original Mosaic team, he had been lured to Netscape. Behlendorf could see the Web going the same way as the desktop, monopolized by a single company. He contacted other webmasters that were writing fixes and suggested they overhaul the NCSA code themselves. Because they represented a last stand against the commercialization of the web, they called themselves the Apache Group. It was also a good pun, their rewritten code being ‘a patchy server’. By the end of 1995, Apache was the most popular on the internet – a position it retains today – delivering around two-thirds of all websites.
OPEN WEB 7 – CLOSED WEB 4
Browser Wars – Netscape went public in November 1995, the shares more than doubling in value on the first day of trading. It was a sign of things to come. Microsoft decided the Web might catch on after all, licensed the Mosaic code and shipped Internet Explorer as part of Windows 95. In an effort to differentiate themselves, Netscape and Microsoft’s browsers started to diverge from their common roots, each creating bespoke elements of HTML. Release by release, they were becoming less compatible. Developers had to learn two sets of protocols and often create two sites, or at the very least add “best viewed in” caveats to sites.
OPEN WEB 7 – CLOSED WEB 5
Web Standards – In response to this divergence, Glenn Davis, George Olsen and Jeffrey Zeldman formed the Web Standards Project (WaSP) to campaign for common standards across browsers. As their mission statement declared, ‘Support of existing W3C standards has been sacrificed in the name of innovation, needlessly fragmenting the Web and helping no one. Our goal is to support these core standards and encourage browser makers to do the same, thereby ensuring simple, affordable access to Web technologies for all.’ They were largely successful, by 2001, the leading browsers were on their way to compliance. A vital battle had been won in the fight for the Web as an open, universal space.
OPEN WEB 8 – CLOSED WEB 5
Demise of the Walled Garden – As the Web grew, the walled garden of restricted services offered by ISPs like CompuServe and AOL became less appealing.A number of new ISPs emerged, offering unrestricted access to the Web. In 1994, Prodigy became the first of the early-generation dial-up services to offer full access. AOL and CompuServe stubbornly stuck to their guns and slowly lost market share, eventually relenting in 2006.
OPEN WEB 9 – CLOSED WEB 5
Search – It wasn’t until 1995 that a search engine emerged that crawled, indexed and ranked websites – it was called AltaVista and by 1997 it was the most popular page on the Web. Excite, Infoseek, Lycos and Magellan copied their model, returning results based on keywords. However, by 1998 there were millions of websites and people had learned how to manipulate search results using underhand techniques (black hat) and search results deteriorated.
OPEN WEB 9 – CLOSED WEB 6
Yahoo! bucked the trend and used people to create their search directory. That was never going to work. Sergie Brin and Larry Page had a better idea, the PageRank algorithm, ranking results based on the number of incoming links rather than keywords. The future of the Web had arrived. It was called Google.
Deep Web – But there are some areas of the web that can’t be searched, which we call the deep web, previously known at the invisible web. Mike Bergman, who coined the phrase, said that searching the web today can be compared to dragging a net across the surface of the ocean “a great deal may be caught in the net, but there is a wealth of information that is deep and therefore missed”.
It’s estimated that search engines can only crawl 0.2% of the entire web.Why is it missed? Search engines follow hyperlinks – no links means no search engine results. Ironically, search engines don’t search search boxes – search engine crawlers cannot type. And most of the Web’s information is buried in databases behind these search boxes. They also don’t search anything behind a password or a CAPTCHA and they can’t search non-HTML content such as Flash. Just because it can’t be searched, doesn’t mean the entire Deep Web is a haven for criminal activity most of it is not.
Dark Web – Some of it is though – and this is where you’ll find out how to make dart poison. A subset of the Deep Web is the Dark Web, or Dark Net, part of the Deep Web that has been intentionally hidden and is inaccessible through standard web browsers. A good chunk of the Dark Web is found on the TOR network accessed with the TOR browser. TOR stands for The Onion Router – it directs Internet traffic through a worldwide, volunteer network of more than five thousand layers. The Dark Web is where most online illicit activities take place, piracy, pornography, the sales of weapons and drugs – you name it, you can find it on the dark web. Anonymous doesn’t always mean bad. The TOR network is vital to groups opposing oppressive governments and Edward Snowden used TOR to send information about PRISM to The Guardian and the Washington Post. But the Dark Web is definitely closed.
OPEN WEB 9 – CLOSED WEB 7
Social Web – Bit of a misleading term, the web and before that the Internet have always been social. Launching in 2004, a year before Facebook and YouTube, Flickr quickly became the most popular image hosting site. Allowing you to share your pictures, follow and connect with other users, it was much more than a photo-sharing platform, it was a social network. It also helped popularise folksonomies – taxonomy by folk. A tagging system that distributes the task across lots of people is not only highly efficient, it automatically uses the vocabulary of the most important people on the Web – the users. It’s also an open platform. Sadly, this was not a characteristic shared by the social networks that followed.
OPEN WEB 9 – CLOSED WEB 8
OPEN WEB 10 – CLOSED WEB 8
The Mobile Web – The launch of the iPhone in 2007 took the web mobile, at least in Europe and the US. Key to its success was the best mobile Web-browsing experience the world had ever seen. This was largely thanks to the touchscreen interface and its pinch-and-zoom functionality. A webpage designed for a PC could now be viewed reasonably well on a phone. This comes at a price, iTunes and the App Store require proprietary software to access them. You cannot access them from a browser, you cannot link to them, and search engines cannot index them. They go against everything that makes the web the most widely adopted technology on the planet.
OPEN WEB 10 – CLOSED WEB 9
Net Neutrality – The other front that has opened up against a universal, open web is the issue of fast lanes on the Web. Content providers being able to pay ISP’s to prioritise their content. Net neutrality is the idea that all data that travels over the Internet is equal. This is not a new idea, the telephone, roads, railways and before them the canals all adopt the principle. However, in the US, earlier this year, Netflix agreed to pay Comcast to improve the speed of its service and has since struck deals with AT&T and Time Warner Cable. Even Netflix CEO Reed Hastings has called this a shakedown tactic and said the practice should be banned. Thankfully, it is banned by European law but in a global world, we’re still affected, following their agreement with Comcast, Netflix put up its prices in Europe too.
OPEN WEB 10 – CLOSED WEB 10
So this is the Web We Have, the scores are on a knife-edge (conveniently for my talk) and the only people that can tip the balance and keep the Web open, free and equal is us. But it’s also about the web fulfilling its potential.
Sematic Web – The potential is a system that not only turns data into information but a system that then draws meaning from this information. This may sound far-fetched but it’s already happening in small pockets across the Web. Take Facebook. People and applications interact to create an ever-smarter pool of knowledge. It’s learning and interpreting our behaviour. Facebook is thinking. The same is true of Amazon, Ebay, Flickr, Twitter and many other websites. But like the physical world, this knowledge is locked away in silos. By sharing this intelligence across an open network, we could take a giant step closer to realising the Semantic Web. A Web that thinks.
Final Thoughts – The web is a place that breaks down national and cultural borders. A place that blurs the boundaries between creating and exchanging ideas. A place that has toppled regimes and created new economic models. A place that has radically changed the way we work, play, shop, socialise and otherwise participate in society. But above all, a place that is for everyone. And if it doesn’t stay that way I’m going to hold be breath till I’m dead.
And here’s the talk itself, 28 minutes in toe-curling detail…