Watching a language model think before it speaks - NinjaHawk's Nest
←
Projects<br>Watching a language model think before it speaks
A 4B model, a lens on its residual stream, and a browser tab where the words show up in its head a beat before they show up in its mouth.
NinjaHawk<br>July 22, 2026
I typed "Is this correct? 12 + 5 = 1" and hit enter, and before the model wrote a single character of a reply, three words were already sitting in a little panel off to the side: math, addition, incorrect. It hadn't said anything yet. It was still reading my message. But it had clearly already made up its mind, and I was watching that happen, live, one layer at a time.
I've been playing with this for about a week and I still get a small jolt every time. So I cleaned it up and put it on GitHub. It's called Subtext, and the whole idea is very simple to state: instead of only reading what a model says, you get to read what it's about to say, at every step, while it reads and while it writes.
What it actually shows you
Normally you interact with a model through the one channel it's optimized to make look good: its output. Subtext taps the channel underneath. It hooks the residual stream at nine layers, and at every token it takes those internal activations and projects them into vocabulary space, so you get a ranked list of the words the model is leaning toward internally. Not the word it emits. The words it's disposed to emit. You watch those readouts stream past as it reads your prompt and as it generates its answer.
The part that surprised me is how much is going on before the first output token. There's a whole "reading" phase where the model is just ingesting your message, and the workspace is already busy: categories, judgments, the gist of where this is going. By the time it starts typing, a lot of the interesting decisions look like they've already been made.
How it works, roughly
The trick underneath is an old interpretability idea with a sharper new version. The old idea is the logit lens: take the hidden state at some middle layer, pretend it's the final layer, run it through the model's unembedding, and read off the top words. It sort of works. Each layer is holding a fuzzy running guess about what comes next, and if you squint at it through the output matrix you can read the guess.
The sharper version Subtext uses is the Jacobian lens, from a recent interpretability paper. Roughly: instead of naively decoding the raw hidden state, it isolates the slice of the representation that the model can actually verbally report and reuse downstream (the paper calls it a global workspace, a "J-space"), and decodes that. In practice it's one small matrix-vector product per layer plus an unembedding per token, which is cheap enough that generation still streams at basically native speed. You're not waiting on the introspection. It just runs alongside.
Nine layers means you don't get one answer, you get a little cross-section of the network. You can watch a concept show up shallow and vague and then sharpen as it climbs.
Three things I didn't expect
It settles the verdict before it says a word. That 12+5 example wasn't a fluke. Over and over, the workspace holds incorrect at high strength while the model is still politely typing "Let me take a look at that." The conclusion is locked in several tokens before the sentence that delivers it. Watching a model be diplomatic on the surface while it's already decided underneath is a weird feeling.
It plans ahead of its own mouth. Mid-explanation, the technical term it's about to reach for is frequently already parked in the workspace, a few tokens early, waiting its turn. The output reads left to right. The thinking clearly doesn't.
You can watch it hop. This is my favorite. There's a known "two-hop" behavior where a model answers "what's the currency of the boot-shaped country" by internally going boot-shaped → Italy → euro. In Subtext you can actually see both feet land: Italy shows up around layer 20, euros shows up around layer 26, and this is all before it generates anything. It's one of those things you've read about in papers and it hits differently when it's scrolling by on your own screen.
The caveats, which you should actually read
I want to be careful here, because this is exactly the kind of demo that's easy to over-read, and I'd rather you trust the honest version.
It's a lens, which means it's an approximation. I checked it against the reference implementation and the top-5 readouts match with cosine similarity above 0.99998, so the decode itself is faithful. But a faithful decode of a projection is still a projection. These words are what a linear readout says the model is disposed toward. Calling them the model's "thoughts" is a metaphor, and a generous one.
It's also a 4B model. Small. And the display is filtered down to word-initial tokens so it's legible, so you're seeing a cleaned-up view, not the raw vocabulary. Most...