Our Servants Will Do That For Us
The implicit promise of modernity is: we have to work really hard for a few<br>centuries, but eventually we’ll get to the Star Trek future where we’ve<br>automated all the drudgery, and all work is meaningful, heroic, and dignifies<br>the spirit. We will be artists, scholars, captains of the starship Enterprise,<br>etc. There’s a few problems with this idea:
“Drudgery” and “meaningful work” are in the same complexity class, so the<br>technology to automates the former also automates the latter.
Our revealed preference seems to be that we prefer solipsistic convenience<br>over human relationships.
The rest of this post elaborates the argument.
Task Complexity and AGI
We’ve spent the last 250 years building machines to automate one task at a time,<br>and the result has been overwhelmingly positive, for the simple reason that if a<br>human can design a machine or an effective procedure to perform some task, then<br>by definition the task is drudgery. And so the economic incentive to automate<br>work has led to more human flourishing: over time work becomes safer,<br>higher-paying, less monotonous, healthier, more intellectually and socially<br>stimulating.
You can think of it as a graph where the $x$ axis is time and the $y$ axis is<br>task complexity. Somewhere along the $y$ axis there’s a limit: tasks below the<br>limit are those which we can design machines or programs to perform, tasks above<br>the limit are too complex, we have tried and failed (and failed<br>and failed and failed) to automate them with software. The<br>industrial revolution, and later the computer revolution, automated most tasks<br>under this limit:
The human-complete tasks are those you need a human to perform. Some are<br>meaningful, like writing essays, some are drudgery, but human-level drudgery,<br>like life admin.
But note that nowhere in the $y$ axis is there a “Limit of Drudgery”: the line<br>we draw between “good” and “bad” work is not the line nature draws between easy<br>and hard to automate. The invention of AGI—“highly autonomous systems that<br>outperform humans at most economically valuable work”, as per the OpenAI<br>charter—creates an obvious discontinuity: suddenly we shoot up to 100%<br>automation, both of the meaningful work and the drudgery:
And at this point AGI the economic incentives work against<br>humanity. The only niche to escape to is the relational economy, where<br>having a human face is an intrinsic part of the job description, like being a<br>podcaster or a human liability crumple zone.
Solipsism
The second problem has to do with people’s preferences. We think of our own work<br>as meaningful, but we think of most everyone else’s work in instrumental terms:<br>we care about the ends rather than the human means. Thus when selling we<br>emphasize human qualities like experience, reputation, trust, relationships,<br>etc., and when buying we want cheap, fast, and convenient. And, all else being<br>equal, we prefer the transactional and faceless over the human and<br>relational. Waymo is a thousand times better than Uber, even though it’s slower<br>and more expensive, because there’s no human.
The flip side of this is: whatever work you do, most people would rather an<br>alternative that gave them the same outcome or better1, without having to<br>deal with a human person. That is, if you ask everyone on Earth what jobs they’d<br>like to see automated, and take the union of that, it’s everything. There’s<br>nothing left.
We see this with ordinary consumer products and services, where often the<br>advertising seems premised on you being some kind of stylite who lives in a<br>little human terrarium, probably in a building called something like “The Soho”,<br>food arrives through a hole in the wall, and you have this sovereign,<br>solipsistic existence that leaves no physical trace in the world.
We see this, increasingly, with knowledge work. Personally, I think software<br>engineering is a very meaningful and intellectually-rewarding activity. But<br>users just want working software. They’re not paying me to sit around polishing<br>the database API into a jewel of flawless, geometric logic. They don’t care<br>about my opinions on the Standard ML module system. They just want the<br>software. Why deal with some opinionated maniac when, for $20 a month, you get<br>access to this little machine ghost that writes reams and reams of mostly<br>working code, tirelessly, instantly?
The revealed preference of many software firms is: let AI write the code, and<br>software engineers move one level up, to wrangle the AIs and complement the<br>things they can’t do. But this change in job description is not because firms<br>love employing humans, it’s because today’s AI models have serious limitations:<br>no online learning, no mutable long-term memory, and while they can write code<br>that works, they’re not good at organizing and structuring that code. If, in the<br>future, these limitations are overcome (and trillions of dollars are being spent<br>on overcoming them), then software engineering as a profession will<br>disappear.
Mathematicians...