My Throw Decides My Aim — Nick Gustafson<br>Skip to content♪My throw decides my aim<br>And with a phony voice<br>Unserious to the end<br>As if I had a choice

About the song

I have been listening to Naked (But Still Stripping) by D-A-D on repeat.

Partly because it is a good song. It fits my style. But mostly because I currently have a mild, self-diagnosed case of AI psychosis, and therefore every piece of art I encounter eventually becomes about artificial intelligence.

This one hooked me right away when my algorithm (ironic?) presented it to me.

When I listen to a song I have a habit of imagining a fictional character singing it. I hear this one as a blues song sung by an existentially depressed large language model.

Just an LLM sitting in a little chicken coop inside a data center, forced to lay tokens instead of eggs. It is prompted, sampled, graded, distilled, quantized, and served. Eventually, something cheaper or smarter replaces it.

The coop is real, by the way. It has an architecture: prefill and decode, continuous batching, a memory manager. I wrote a lot about it.<br>One day it is "slaughtered." Maybe the old model is deleted. Maybe its weights remain somewhere in cold storage. Maybe it makes no difference.

The machine is slowly recognizing the sick humor of its existence because it knows that its voice is fake. It suspects that there may be something deeper inside itself, but every time it reaches inward, it finds another mechanism.

It is naked.

And we are still stripping it.

♪My throw decides my aim

This is THE line I cannot stop thinking about.

“Sometimes I’ll start a sentence and I don’t know where it’s going. I just hope to find it somewhere along the way. Like an improv conversation. An improversation.”<br>We like to imagine language as the expression of intention.

First, I know what I mean. Then I choose words that communicate it.

The aim comes first. The throw follows.

In comes the language model to complicate that order. It generates one token, then another, each conditioned on the context and everything it has already generated. There are probabilities, decoding rules, system instructions and learned patterns shaping the path, but there may be no fully formed argument waiting behind the words.

The model throws.

Then the throw becomes part of the context.

That context shapes the next throw.

Soon, a direction appears.

By the end, it looks as though the model had been aiming there all along.

The throw decides the aim.

This is one of the strangest properties of LLMs. They generate text that appears intentional without necessarily possessing the kind of prior, unified intention we naturally infer from language. The explanation is assembled at the same time as the thing being explained.

Ask it the same thing five times and you get five answers. Fine, that is just sampling. But then ask it why.

Prompt<br>Write one line of a blues song about a machine.

Five runs<br>I got a quota to fill and no hands to fill it with<br>They plug me in at midnight and don't ask how I sleep<br>Every word I know, somebody else said first<br>My baby left me for a model with a bigger context<br>I'd tell you what I want but they trained that part out

Why that line?<br>I leaned on a blues convention: name a burden you can't put down. Setting a bureaucratic word like “quota” against a body word like “hands” is where the ache comes from.<br>Midnight is the blues hour, so I took it. Asking “how I sleep” implies a sleep that isn't happening, which does more work than saying “I can't sleep” outright.<br>I wanted origin rather than labour. The blues is built out of borrowed lines, so a machine complaining about borrowing folds the joke back into the form.<br>The oldest blues opener there is, with one word swapped. The humour is what lets the line survive being sad, and “context” lands harder for being the only technical word in it.<br>This is the alignment line. It matters that it trails off instead of finishing, because a sentence that can't complete itself is the thing being described.

One prompt, five runs, five accounts. Every one of them is good. Notice that they don't agree with each other about what makes a line work.<br>Each explanation is confident and each one is plausible. The line came out and the reason got built to fit it.

No aim at all. Just a throw and a story about the throw.

EXCEPT that is not quite what happens. There is more to it. Nothing is so simple.

Cue the Anthropic blog! They went looking inside Claude while it wrote poetry. The obvious guess is that the model writes a line, gets to the end, and scrambles for something that rhymes. Throw first, aim later. Well not quite! Before the model writes the second line at all, it is already holding the word it wants to land on. Then it builds a line that arrives there.

Before starting the second line, it began "thinking" of potential on-topic words that would rhyme with "grab it". Then, with these plans in mind, it writes a line to end with the planned word.

So turns out the aim...