Does ChatGPT Pass the Turing Test? | Jessica Riskin | The New York Review of Books
Skip to Content
Advertisement
Visit us on Facebook! Opens External Webpage
Visit us on Twitter! Opens External Webpage
Visit us on Threads! Opens External Webpage
Visit us on Bluesky! Opens External Webpage
Mail to
Print page
Illustration by Lucas Adams
“Please write me a sonnet on the subject of the Forth Bridge.” This was apparently the first question that occurred to the English mathematician Alan Turing when, in a captivatingly strange 1950 paper entitled “Computing Machinery and Intelligence,” he imagined conversing with an intelligent machine and founded the field of artificial intelligence. The Forth Bridge, built in 1890, is a cantilever railway bridge spanning the Firth of Forth near Edinburgh, Scotland. Why a sonnet about the bridge? The juxtapositions are unexpected: a lovelorn poetic form, a 2,500-ton iron structure, and a computing device. If the last could produce authentic sense by applying the first to the second, Turing must have thought, that would indicate intelligence.<br>When I typed the same question into ChatGPT, it generated a bad poem in sonnet-like quatrains. How did Turing’s imaginary machine answer? “Count me out on this one,” it demurred. “I never could write poetry.” I guess it’s not surprising that I find Turing’s imaginary machine’s answer infinitely more persuasive than ChatGPT’s, since of course the first was written by an intelligent human: Turing himself. But it does seem surprising that the design process he established in his foundational paper has led to an “artificial intelligence” utterly unlike the intelligent machines he imagined in the same paper.<br>ChatGPT is a generative AI, meaning that it uses statistical models to extrapolate patterns from data and then applies these patterns to generate new text, images, or other products such as music or computer code. Generative AIs rely on machine learning techniques whose foundations Turing laid in his landmark paper, where he hypothesized a process for arriving at an intelligent machine. He imagined first building an “unorganised machine,” a bunch of interconnected neuron-like components, that would become organized through a training process, creating the blueprint for an approach to artificial intelligence that would later be called “connectionism” and would lead to neural networks like those constituting the new generative AI large language models. But although ChatGPT descends from Turing’s protocol, it is nothing like the machine interlocutors he conjured in his dialogues, and therein lies an interesting conundrum.<br>Turing used imagined conversations with intelligent machines to introduce his idea for a test of machine intelligence, known ever since as the Turing Test. This is the test some are saying the new generative AIs have cracked—often to conclude that, since the generative AIs clearly don’t have human-like intelligence, the Turing Test must not be a reliable measure after all. Others suggest the AIs are like Frankenstein’s monster, on the verge of taking on a sinister life and mind of their own. Even as they advertise the utility of their new products, computer scientists also warn that they are potentially very dangerous: they could make it impossible to distinguish information from disinformation, thereby fatally undermining democracies or any form of rational decision-making; might cause catastrophic harms to any complex system—economic, air traffic control, energy, nuclear weapons—by malfunctioning or pursuing their goals in unforeseen ways; and might escape human control even without becoming the robot villains of science fiction and Elon Musk’s sensationalist admonitions that we’re “summoning the demon.”1<br>In March Musk was among the initial signatories of an open letter calling for a pause in development of the technologies that was signed by thousands of the very people who have been bringing them to us: computer scientists, engineers, and tech CEOs. They asked, among other things, “Should we risk loss of control of our civilization?” In May the Center for AI Safety released a stark one-sentence warning: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”<br>But Turing’s imaginary intelligent machines weren’t the least bit menacing, nor even especially powerful, no more so than the next intelligent being. The Turing Test in its original form was a game played with the machine, the Imitation Game. A human examiner would converse with a machine and another human, both hidden behind screens and both trying to persuade the examiner that they were the true human. If the examiner couldn’t tell which was which, Turing proposed, then we...