"Expect the end of the world. Laugh." - by L. M. Sacasas

The Convivial Society

SubscribeSign in

"Expect the end of the world. Laugh."<br>The Convivial Society: Vol. 7, No. 9

L. M. Sacasas<br>Aug 18, 2026

113

20

Share

Welcome to the Convivial Society, a newsletter about technology and culture, both rather broadly construed. In this installment, I simply offer some observations about the recent discussions of the risks posed by AI in light of the OpenAI/Hugging Face incident a few weeks back. I suggest that while our attention is directed at the speculative prospect of spectacular risks, we might do better to be on guard against another kind of existential risks.<br>Thanks for reading,<br>Michael

“[W]hatever makes for greatness—nobility, dignity, steadfastness, and a kind of laughing courage—remains essentially the same throughout the centuries.”<br>— Hannah Arendt

Earlier this month, being on X/Twitter, or at least the version of the timeline I experience, felt like an eerie echo of the early days of 2020. Before most mainstream media outlets were giving developments in Wuhan much, if any attention at all, but it was nonetheless possible to encounter the rumor of rumors of an impending crisis circulating online. In this case, however, the impending crisis is not a health emergency but some variation of an AI-induced catastrophe. There’s a lot of “no one is prepared for what’s coming” and “there are only 500 people in the world who matter now” going around, the latter referring to those with the distinction of working at the frontier AI labs.1<br>These fears have been fanned by news that an unreleased OpenAI model-in-training escaped containment in a sandbox environment and hacked another company, Hugging Face. The OpenAI model achieved this in part by exploiting a vulnerability to establish an internal message board that agents were able to use in order to coordinate. OpenAI discovered this and shut it down, but did so in such a way that the model was able to reconstitute the message board and continue to pursue its goals, which eventually led it to hack Hugging Face in search of the information it needed to accomplish the task it had been set.2 It should come as no surprise that all of this has fueled even more anxiety in certain circles about the worst case AI scenarios beginning to materialize. That recurring refrain, “No one is prepared for what’s coming,” has, for the moment, taken on a more panicked pitch.

Agents communicating. Slide from Open AI Black Hat conference presentation.<br>I should clarify that I’m not convinced these fears of an AI-induced apocalypse, at least the most extreme variations, are necessarily well-founded, but you can take that with a grain of salt because I won't pretend to be qualified to adjudicate the relevant technical matters. That said, I do think it’s worth considering the following from Eryk Salvaggio’s recent discussion of the OpenAI/Hugging Face incident:<br>“When we say ‘AI models went rogue,’ we skip the entire story: the part where OpenAI manually removed the model’s cybersecurity blocks. We skip that OpenAI chose to test it on a machine with a live network connection. If you see AI as a system from nowhere, you can make the claim that the model ‘went rogue,’ and that it ‘broke containment,’ both of which place agency and decision-making onto the model itself rather than the people who set the stage for that behavior.<br>When you expand the boundary of the system to include the people building and deploying it, the case becomes much less science fiction and more like incompetence.”

“Nobody is losing control of an emerging intelligence,” Salvaggio goes on to argue. “They are redirecting their own agency to the thing they are building.”<br>Perhaps it’s merely a function of overlapping terminology (“labs,” “containment,” “virus” in relation to computing, etc.), but these considerations raise a further disconcerting parallel between discussions of AI and the pandemic: careless and possible negligent security measures. And, to take it one step further, upon reflection it’s striking how closely the argument that “super-intelligent machines are potentially catastrophic for humanity but also inevitable so we must be the ones to develop them responsibly” echoes the arguments made in defense of gain-of-function research. Make of that what you will.<br>Now, I think it is possible to agree with Salvaggio, to refuse narratives of inevitability meant to obfuscate responsibility, and, nonetheless, to acknowledge that there is ample room for ruinous mischief and severe economic and societal turbulence.3 I’d be lying if I told you that I was utterly unperturbed by the potential confluence of certain trends—in AI, robotics, autonomous weapons systems, and drone warfare, for example.<br>And, indeed, returning to the assemblage of human actors implicated in the systems that produce what we call AI, it is also possible to acknowledge a longstanding dynamic identified in connection with the emergence of...