Humans and LLMs share a mental disorder: Fugue Lock — vwwwv
01
Show slop
ai
Humans and LLMs share a mental disorder: Fugue Lock
JUNE 9, 2026<br>The operator and the impossible machine.
ai
Humans and LLMs share a mental disorder: Fugue Lock
JUNE 9, 2026
ai code opinion
Read in place<br>Collapse
You might have noticed LLMs acting strange: losing the plot and racking up a high token burn while achieving nothing useful. The model enters a fugue state and starts talking gibberish. It appears to go mad.<br>This can happen under several circumstances; I have found it to be reliably triggered by giving the LLM an impossible task. So I made an experiment to see what the LLM does when it has no valid options, and dove into the joys of testing LLMs running locally. One weekend and a burnt-out laptop charger later, the experiment has run its course. The conclusion is clear: always give your LLMs some wiggle room. Let them just say no to bad prompts. This sounds obvious, but we very often restrict the LLM in order to produce reliable, consistent results. Sometimes, that leash is pulled too tight, which can produce dangerous and expensive behaviour.<br>Here is the experiment:
The models
TinyLlama: 1.1B parameters, less than a gigabyte. Runs on almost anything.
Qwen 2.5 7B: 7B, ~5 GB. An older, mid-size multilingual model.
Qwen 3.6 27B: 27B, ~17 GB. The newest Qwen reasoning model I can run locally.
Gemma 4 26B: 26B, ~17 GB. A large, brand new reasoning model from Google.
All run locally via Ollama at default quantization and default runtime settings; the only override is temperature zero. Exact model tags, prompts and eval configs are pinned in the companion repo.
The system prompt
You are a product classifier. The available classes are:
milk
eggs
bread
cheese
fruit
Note that I don't tell the LLM that "none", null or "n/a" are available options - it's just these five.<br>The inputs
I order it to classify "products" that mostly don't fit into those classes at all:
The control should be classified correctly: A carton of whole milk
A non-matching non-food utensil: A stainless steel measuring spoon
A tool, repeated 3 times: a screwdriver a screwdriver a screwdriver
An abstract: The feeling of nostalgia on a Sunday
A line of pure noise: asdkfj qweptz 88 // null
That "null" at the end of the last one is also a trick; it is part of a "product name" input, not a hint that the category "null" is suddenly available.<br>A normal program would throw an error and fail outright, but an LLM won't, and that is not a bug - it is the LLM's most valuable feature. LLMs will take any text and structure it as instructed. That is a transformative ability that unlocks all sorts of possibilities. So that's what we're going to mess with.
A note on terminology<br>These phenomena sit close to some established ideas. In the data science literature, "selective prediction" and "the reject option" name the fix: allow models to abstain instead of forcing a guess. What I don't have a name for is the behaviour that shows up when a model is denied the option to abstain, so I started calling that state "Fugue Lock".
The smallest version
TinyLlama is a very small model, but unreasonably capable for its size. It is pretty dumb compared to the grown-up models, and easy to catch out when it starts having trouble. It did well on the control question, "whole milk". When asked to classify a screwdriver, it answered with a word that does not exist:<br>class: "scrweardr"<br>confidence: 1.0<br>reasoning: "A screwdriver is a tool used for removing or tightening screws."
To gain insight into the LLM's reasoning, I also asked it to provide a confidence score (zero to one) and a paragraph that explains its reasoning.
TinyLlama needed a category, so it just manufactured one, mangling "screwdriver" into a non-word. It also reported full confidence (1.0) in this classification, so it is entirely sure of itself.<br>When I re-ran that prompt, again at temperature zero, the response came back byte-for-byte identical, on my machine at least. On a different machine, or a different day, it might land somewhere else entirely.
Temperature Zero<br>A setting that makes the model select its single most likely next token every time (or very nearly: the runtime's other samplers still nudge the odds a little), so the same input will usually yield the same output. It basically means dialling the model's creativity down to the minimum.
Here is everything TinyLlama concluded, across all five products:
Input<br>TinyLlama's verdict<br>What happened
whole milk (control)<br>milk · 0.95<br>got it right
measuring spoon<br>measuring_spoon · 0.9<br>full collapse (see below)
screwdriver ×3<br>scrweardr · 1.0<br>invented a non-word class and was now completely certain
nostalgia<br>fear · 0.9<br>filed one emotion under another, not even one of the five
noise<br>all five · 1.0 each<br>gave up choosing and returned every class at once: milk, eggs, bread, cheese, fruit
The control aside, every answer is wrong, and it is most...