Matthew Butterick | Extinction-level capitalism

Extinction-level capitalism a citizen’s thoughts<br>on AI risk<br>AI is inher­ently polit­ical tech­nology. If AI works as intended, it will grad­u­ally corrode our liberal democ­racy, risking an irre­versible shift into another polit­ical and economic config­u­ra­tion. Among AI risks, this one deserves more consid­er­a­tion, because it requires no addi­tional condi­tions like malign actors or AI malfunc­tion. AI only needs to amplify existing trends, espe­cially around concen­tra­tion of capital. This damage will occur even assuming that in the near term, AI will broadly improve mate­rial well-being.<br>About me<br>I’m a self-employed author, designer, programmer, and lawyer. In 2022, I learned that my own works were in the training datasets of gener­a­tive-AI compa­nies. In response, I invented the first set of lawsuits chal­lenging the legality of these prac­tices. I’m currently co-counsel for plain­tiffs in a number of AI cases. Though I discuss certain legal issues below, I am not your lawyer, and nothing here is held out as legal advice. These are my personal views as a citizen and economic actor; I speak only for myself. This piece is typeset in Equity, Advo­cate, and Trip­li­cate, fonts I designed. They can be licensed for your own polemics and pamphlets.<br>Emergent effects<br>Two billion years ago, the rock layers comprising what is now called the Colorado Plateau began to form: first igneous and meta­mor­phic rocks, followed by many layers of sedi­men­tary rocks. About fifty million years ago, through tectonic action, this plateau gained thou­sands of feet of eleva­tion. About five million years ago, a river began to flow. The river carried silt and debris, scraping out the begin­nings of a canyon. The river deep­ened the canyon, exposing its walls to weather and erosional forces that widened the canyon further. Today the waterway is the Colorado River. The geolog­ical forma­tion is the Grand Canyon.

The forma­tion of the Grand Canyon required zero human agency. Zero tech­nology. Zero coor­di­na­tion among the river, the land, and gravity. In that sense the Grand Canyon is an emer­gent effect: a complex, unfore­see­able output arising from simpler inputs.<br>But we would never wonder whether the river is sentient. Or whether the river cares about the dirt that it carries out of the canyon. The water is just doing what water does: flowing down­hill. The dirt just happens to be in the way.<br>Inherently political technology<br>Langdon Winner is a polit­ical theo­rist. Winner wrote the excel­lent and influ­en­tial essay “Do Arti­facts Have Poli­tics?” (1980). Winner sought to debunk the tradi­tional framing that “tech­nolo­gies are … neutral tools that can be used well or poorly, for good, evil, or some­thing in between.” Instead, Winner proposes two ways that a tech­nology can affect its polit­ical envi­ron­ment:<br>The tech­nology is designed to have certain polit­ical effects. For example, the Great Fire­wall of China, a bundle of tech­no­log­ical measures that limit Chinese citi­zens’ access to foreign infor­ma­tion sources. Antipo­dally, the Tor Project intends to maxi­mize user anonymity and thwart govern­ment intru­sion.

The tech­nology is inher­ently polit­ical. This is Winner’s key analytic fulcrum. Winner describes two versions of inher­ently polit­ical tech­nology. The first is where the tech­nology “actu­ally requires … a partic­ular set of social condi­tions as [its] oper­ating envi­ron­ment.” For instance, nuclear weapons: the only respon­sible way to possess such dangerous tech­nology is to place it within “a central­ized, rigidly hier­ar­chical chain of command … the [atom] bomb must be author­i­tarian; there is no other way.” The second version is where the tech­nology is “strongly compat­ible” with a certain polit­ical arrange­ment (even if not strictly required) and thus tends to bring that arrange­ment to fruition.

As an example, Winner considers the mechan­ical tomato harvester. Devel­oped at UC Davis in the 1950s, the machine was tremen­dously produc­tive. But it was also expen­sive. Only well-capi­tal­ized tomato growers could afford it. Those without couldn’t compete. According to Winner, the number of Cali­fornia tomato growers dropped from ~4000 in the early 1960s to ~600 in 1973, costing ~32,000 jobs and the compounding nega­tive effects on those commu­ni­ties. Winner summa­rizes:<br>What we see here … is an ongoing social process in which scien­tific knowl­edge, tech­no­log­ical inven­tion, and corpo­rate profit rein­force each other in deeply entrenched patterns that bear the unmis­tak­able stamp of polit­ical and economic power … oppo­nents of inno­va­tions like the tomato harvester are made to seem “antitech­nology” or “antiprogress”. For the harvester is not merely the symbol of a social order that rewards some while punishing others; it is in a true sense an embod­i­ment of that order.

Not merely the symbol—the embod­i­ment. A facially neutral...