natural general intelligence to solve this… using only emails’ is a good defamiliarizing trick for thinking about AI workflows and the difficulty of outsourcing of white-collar work or your life."> -->

“Useful Outsourcing Is Hard”, by Gwern · Gwern.net

Skip to main content

Warning: JavaScript Disabled!

For support of key website features (link annotation popups/popovers & transclusions, collapsible sections, backlinks, tablesorting, image zooming, sidenotes etc.), you must enable JavaScript.

Asking ‘what if I tried to use an amnesiac natural general intelligence to solve this… using only emails’ is a good defamiliarizing trick for thinking about AI workflows and the difficulty of outsourcing of white-collar work or your life.

by: Gwern<br>2024-09-16–2026-06-02<br>finished<br>certainty: log

Tools can be highly capable but still not useful for automation or white-collar workers due to overhead, friction, lack of context, and ignorance of what to use them for. This is why it is difficult to “outsource” things to grad students, secretaries, people overseas, etc.

This is true of AI as well. So, if someone struggles to find a use for LLMs which actually saves them time, and blames the LLMs for being too stupid or incapable, they may be wrong; it may be that this is something which is difficult to outsource.

A simple way to distinguish between these two problems is to simply ask, “could you outsource it to a human being who worked for free?” (You can even try to pretend to be that human, to test out a specific scenario.) If the answer is “no”, then it cannot be a matter of artificial versus natural intelligence.

Often, we will find the answer is “no” and this is why even famous important people with little time were not “outsourcing” most things even in the past.

An important implication of this fact is that it means that the level of “outsourcing” to LLMs is not a good measure of capability, because it is related more to factors like how much individuals are willing to rework their life to make outsourcing possible, to make “context” explicit (preferably textual), chance outcomes of LLMs being able to handle specific limiting steps, etc.

For example, even though many of my essays are, I think, fairly “obvious”, given my corpus as a whole, it is impossible for me to hand my notes and corpus to any human or LLM, specify a 1-sentence essay idea, and get back a publishable essay; they cannot see or memorize my entire corpus, they are unable to imitate my style, and so on. And this is despite being more capable than me in many ways, like coding or math. (Nevertheless, they are more capable than me in many ways.)

So LLMs may become highly capable and not be useful for outsourcing for any individual… right until the point where they suddenly are capable of simply replacing that individual (possibly to their great shock). Which is good for whoever employs the LLM, but not for that individual.

If you’re having trouble coming up with tasks for ‘artificial intelligence too cheap to meter’, it could be because you are having trouble coming up with tasks for intelligence period. Just because something is highly useful doesn’t mean you can immediately make use of it in your current local optimum; you may need to seriously reorganize your life and workflows before any kind of intelligence could be useful.

There is a good post on the LW2 front page right now as I write this, about exactly this problem: “The Great Data Integration Schlep” Most of the examples in it do not actually depend on the details of ‘AI’ vs employee vs contractor vs API vs…—the organization is organized to defeat the improvement. It doesn’t matter whether it’s a data scientist or an AI reading the data if there is some employee whose career depends on that data not being read and who is sabotaging it, or some department defending its fief.

I usually call this concept “automation as colonization wave”: many major technologies of undoubted enormous value, such as steam or the Internet or teleconferencing/remote-working, take a long time to have massive effects because you have everyone stuck in local optima and potentially outright sabotaging any integration of the Big New Thing, and potentially have to create entirely new organizations and painfully liquidate the old ones through decades of bleeding.

There are few valuable “AI-shaped holes” because we’ve organized everything to minimize the damage from lacking AI to fill those holes, as it were: if there were some sort of organization which had naturally large LLM-shaped holes where filling them would massively increase the organization’s output… It would’ve gone extinct long ago and been replaced by ones with human-shaped holes instead, because humans were all you could get.

This is why LLM uses are pretty ridiculous right now as a % of GDP—oh wow, it can do a slightly better job of “grammar-checking my emails”? I can have it write some code for me? Not exactly a new regime of hyperbolic global...