Where Wizards Stay Up Late - Neil O'Brien's Substack

Neil O'Brien's Substack

SubscribeSign in

Where Wizards Stay Up Late<br>And the state of the internet in 1972

Neil O'Brien<br>Oct 26, 2024

11

Share

The team building the first Interface Message Processor<br>This summer I read a great but ancient (1996) book about the various intellectual leaps that led to the creation of the internet.<br>Where Wizards Stay Up Late is an interesting book in lots of ways. Apart from anything else, it debunks the widespread myth that the internet was designed as a way to cope with a nuclear attack. Instead, the motivation mainly came from the desire of a major science funder in the US (ARPA) to get more benefit for its buck, and allow more people to use the (rare and expensive) computers they were funding.<br>I meant to write about the book ages ago - but have been prompted to finally write by this ace (and very retro) film looking at the state of the internet in… 1972. It features lots of the key people involved:

Jigsaw pieces

Where Wizards Stay Up Late is a great summary of all the ideas that had to come together, from many different people, in order to make it possible for you to read this post.<br>Bits of the jigsaw included:<br>A vision of what computers would eventually be able to do. (J. C. R. Licklider)

The basic idea of a generic system to access computers remotely. (Bob Taylor)

Splitting messages into smaller packets (“Packet Switching”) flowing around a decentralised network. (Larry Roberts, Paul Baran and Donald Davies)

The idea and creation of “server” computers (originally called “Interface Message Processors”) that would run a network and sit between each computer and the network. (Wes Clark, Frank Heart)

An initially paper-based idea-sharing platform (“Request for Comments”) to co-ordinate everyone. (Steve Crocker)

The idea of connecting networks like ARPANET to other networks to form an “internet” or “internets” with a “Transmission Control Protocol” and “Internet Protocol” - TCP/IP (Bob Kahn and Vint Cerf)

“E-mail” across the network (Ray Tomlinson)

The connection of whole local “Ethernets” of low powered machines to the network (multiple people)

The whole story is a great example of the non-linear nature of innovation. A bunch of people with different parts of the puzzle bounced around, tried things, tried other stuff, moved sideways, and cludged stuff to get it working.<br>Innovators had to work around large established firms (AT&T, Honeywell) that were not interested, and wanted to keep doing what they were doing: “Same but more ism.”<br>A bunch of organisations that were supposedly in charge of setting standards (like the International Organisation for Standardisation) also had to be worked around and were overtaken by protocols set by practitioners.<br>Lots of the uses of the internet (like email, message boards, the connection of people without a computer to computing resources) were not foreseen by those trying to build the network.<br>And a lot of the most interesting things happened not in universities, but in in projects commissioned by government from private firms - notably Bolt Beranek and Newman, which had been set up for a totally different purpose (acoustic engineering).<br>Why care?

Why care about any of this? The invention of the Internet / Arpanet is another example of the incredible strength of the US innovation-industrial system, and also the tendency of one type of innovation to lead to another.<br>The UK had a walk-on role in the invention of the internet: Donald Davies invented the idea of packet switching in parallel with people in the US, and influenced the creation of the Arpanet. Later on Tim Berners Lee fused the ideas of the internet with hypertext and domain names to invent the “Web”.<br>But the book also prompts the question - why is the US innovation system so much stronger than that of the UK or Europe?<br>Various people have tried to get UK innovation funding away from the current mire of (a) endless applications for tiny grants and (b) everything in Britain going through universities.<br>I have been involved in a couple of these. The Connell Review emphasised the importance of purchasing and the non-linear nature of innovation. Things like the Faraday Battery Challenge (FBC) and Battery Industrialisation Centre were intended to promote more work on the boundary with industry.<br>More recently, through the work of James Phillips, Dominic Cummings and others, the creation of ARIA is intended to be a big step towards more transformative research.<br>But we have a long way to go in changing things to make the UK the most attractive place to innovate. Changing research funding is a part of it. Even just within current budgets we could do better. Most innovation comes from younger people, but our systems of peer review and research council funding tend to put power and money in the hands of older people.<br>But research funding is only part. There’s a massive spatial element to US innovation. (Tom Wolfe’s history...