Language Models Are Anomaly Detectors | NGrislain

I work on security detection at Datadog. The model I have been building, Mambark, reads audit logs the way a language model reads text and scores every event by how surprising it is. It is a small model by today's standards, about 97 million parameters, and it looks at hundreds of billions of security events instead of sentences.

Generative models get talked about as machines for producing things. The claim of this post is that the same models, with nothing added and nothing fine-tuned, are also the most general anomaly detectors anyone has built , and that this is already changing how detection is done at scale. The mechanism turns out to be the very thing that makes them write.

The idea underneath is old and almost embarrassingly simple. I wanted to see it on its own, away from security telemetry and away from anything I cannot publish, so I built a small open-source project that runs it on plain English text. This post is what it shows.

Surprise is a score

Any model that predicts what comes next is already an anomaly detector.

Hand it a prefix. It gives you a distribution over the next symbol. Now look at what actually came next and read off the probability it assigned. If the model said 0.4, nothing happened. If it said 0.0001, something happened.

Take the negative log and you have a number:

s_i = -\log P\!\left(x_i \mid x_1, \ldots, x_{i-1}\right)

That is the anomaly score. It is measured in nats, it is never negative, and it adds up: the surprise of a whole window is the sum of the surprises of its symbols, which is exactly the negative log-likelihood of that window. No labels, no list of known attacks, no rule describing what bad looks like. The only ingredient is a model of what usually comes next.

That distribution is worth looking at directly, because it is the same machinery that makes these models write:

One forward pass over the prefix produces one distribution over the entire vocabulary, 248,320 tokens for this model. To generate, you sample a token from that distribution, append it, and run the pass again. To score, you skip the sampling and instead look up the token that actually came next, then take minus log of the probability sitting in that slot. Same pass, same distribution, two different questions asked of it. Detection is generation with the sampling step replaced by a lookup.

The three panels are real output at three points in the contaminated text used later in this post. In ordinary English, after computer science that develops, the model's favourites are algorithms at 30.6% and intelligent at 18.5%; the text said and, its fourth choice at 6.8%, which costs 2.69 nats. Nothing happened. At the splice, after achieving defined goals., it expects a paragraph break or AI or The; it gets E, ranked 602nd out of 248,320, probability 0.002%, and the score jumps to 11.09 nats. Something happened.

The third panel is the one I did not expect. By the end of the Basque sentence the model's third most likely continuation is E at 10.4%, with Ez and B further down the list. It is expecting more Basque. It has moved its own notion of normal, in about two hundred characters, with no fitting and no retraining, and you can read that shift straight off the distribution.

None of that is specific to Qwen, or to language. Any model that puts a probability on the next symbol can be read this way. So the interesting question is not whether to score by surprise, it is which model you ask, because whatever you pick is what defines "usually" and therefore what counts as an anomaly.

Two ways to know what comes next

A frequency table

The oldest answer is to count. A character n-gram model estimates P(c \mid \text{previous } n-1 \text{ characters}) by counting how often each continuation followed each context in some training text. To score a new character, you look up its context and read off the frequency. The model in this post is a 5-gram with interpolated smoothing, so it mixes the order-5 estimate with order-4, order-3, order-2, the unigram and a uniform term. Nothing ever gets probability zero, including characters it has never seen.

It is cheap, transparent and you can read the whole thing. It also has two properties that turn out to matter a lot.

You have to fit it, on the specific kind of text you are willing to call normal. And its memory is four characters. Everything before that is gone. A 5-gram deciding what comes after atio does not know the text is in English, does not know it is about machine learning, and does not know that the same abbreviation appeared three hundred characters ago.

A pretrained sequence model

A language model conditions on the entire prefix. Every character since the beginning of the sequence is in the context window. And there is no fitting step: pretraining already covered English, French, Basque and Wikipedia prose, along with most other things. The model I used here is Qwen3.5-0.8B-Base, the base...