So, Where Does Next-Token Prediction Leave Us? - POP RDI; RET;

Table of Contents

Solved/Cooked

AI maximalists in some corners of the internet hate it when people refer to LLMs as just “next-token predictors” or “stochastic parrots”. It is instinctively taken as a pejorative. They use the words “solved” or “cooked” to signal the end of industries or classes of work that take real human creativity, expertise or effort. “Animation is solved”, “Hollywood is cooked”, “coding is solved”, “postgrad students are cooked” and so forth. It is far from a neutral description of progress, there is a certain glee to it. They celebrate the obsolescence. There is a belligerence in discussions and it is starkly reminiscent of people taking political sides online. I cannot think of any other piece of technology that comes close to this level of tribalism. Hmm, maybe cryptocurrency? Arch Linux users? Not even close.

These are extreme examples.1 I had to put this on top, because from where I stand, I feel like the same people cheering now are the same people being economically priced out of this.

Why?

Machines that can think have been an important trope in our collective and literary fiction for so long. It was always a question of when and not if and the graph of when and what is going parabolic. I feel there is something primal underneath it: the hubris of creation, playing God, intelligence squeezed out of sand.

So, what makes a fanatical proponent? Why do they seemingly have contempt towards human ingenuity and labour? Do they have an overly optimistic view of living off of universal basic income, spending all their time in leisure while the machines subsonically2 hum away at work that could end up being considered beneath humanity? We cannot and obviously should not generalise these things. But lately I’ve been thinking if it is just a class issue?

This cohort of people likely have a cushion that softens the concussive blows they are doling out right now. They perhaps have the luxury of a somewhat functioning government and a social safety net that they are witness to in all walks of life. Over half the world does not. Science and technology, I feel, has always had a certain apathy towards the plight the people at the bottom rungs. And it is by design, I fear. To break in, or bear the fruits of, you at least had to have been in a position to get an education.

The cushion and the safety net is largely transitory. It cannot be sustained forever, unless we could do something like tax the corporations for cutting out labour or something crazy like that. We cannot even fairly tax garden variety billionaires right now.

In the long run, there is no winning team here. There is no Basilisk to appease and no side to be on, the real world implications are coming for you too.

The Meta-Contract

The founding tenet of AI, “saving us” from all sorts of difficult things: climate, disease, poverty, conflict is falling, fast. The frontier labs have shifted to signaling at a more mundane, perverse motive: that of simply cutting labour.

Even the most exploited classes of people, always had - at varying degrees - a bargaining chip. A chip that corporations spend billions trying to snatch away. The chip of labour, of being needed. CEOs and proponents of AI flippantly announce and proclaim this chip will not be legal tender anymore. Anthropic’s CEO Dario Amodei has quite the reputation in this department. OpenAI’s CEO Sam Altman says, college degrees are worthless now because AI can teach better, in a more personalized way. Downstream to these people, VCs say in the next 5 years this and this class of work will be “solved” - only with the finesse of coming from money.

What happens when you strip a human of their economic utility? The promise of capitalism has always been - you will have a spin at the roulette table. If you work hard enough, you can make it. We chugged along and ushered in this ultra capitalistic world where if you work at an Amazon warehouse, a bottle is your bathroom, a dollar for your dignity. What happens to billions of people whose only shot at upward economic mobility is a kid that got a degree and goes on to find a job?

AI democratises the very things that the CEOs want obsolete. Sure, it may teach you stuff and build you dashboards, but you will not be able to sell them. It concentrates, it is the literal concentration of (if the dream is realized in full) the means of production into the hands of a wealthy few. A capitalist’s dream. It is behind a $200 subscription or behind a beefy GPU and the goodwill of labs to release open weight models. AI raises the ceiling, of human output, at the same time, it also raises the barrier to entry.

I see a lot of knowledge workers online adopting the mantra “with AI, not by AI” and this largely reads as cope. The...