How does Pangram work? - Pangram

SubscribeSign in

How does Pangram work?<br>The way we tell the difference

Pangram<br>Jul 17, 2026

36

Share

We think it is important to be able to reliably identify AI-generated writing, so we built a detector that does it. Because the problem is so important, it’s only prudent to be transparent about how that works.<br>When any author writes a sentence, they’re making thousands of conscious and unconscious decisions about how it should be composed. Fundamentally, AI detection is a problem of author identification – categorizing which decisions are prototypical of what kind of author. We can do this because, while LLMs like ChatGPT aren’t human, they are still a single author making decisions. The kinds of decisions they’re liable to make are also constrained: assistant models need to produce helpful, clear writing to effectively answer questions. These preferences are ingrained in the model during training, and once they’re there, it’s very hard to prevent them from surfacing.<br>What is Pangram?

Pangram is a type of neural network called a classifier model. In the broadest sense, Pangram reads a segment of writing and – using an enormous set of learned patterns – estimates whether it is AI-generated.<br>Pangram learns to distinguish human and LLM writing by building a kind of map where authors with similar writing styles are placed close together, while those whose styles differ are positioned farther apart. On this map, human-written and AI-generated text form distinct clusters, and our research shows that Pangram can even locate major commercial language models, such as ChatGPT and Claude, in different regions.

When Pangram encounters a piece of writing it has never seen, it determines where it should go on the map by comparing it to the text it already knows. No single aspect of the writing determines where it lands, so a human writer is not at risk of being misclassified simply because ChatGPT has adopted one of their favorite devices, such as the em dash. Visible LLM-isms are only one signal among hundreds of thousands that Pangram considers.

Subscribe

If the resulting coordinates fall in a human-written region, Pangram predicts that the text is human; if they fall in a more ambiguous area, or if different passages from the same text point in different directions, Pangram begins to suspect it may contain AI writing. This is why Pangram has a minimum word count: more words mean more precise coordinates. The same approach also helps Pangram generalize to new AI model releases, which tend to inherit much of their predecessors’ style and voice, and so occupy very similar regions of the map.<br>Pangram is better than other AI detectors

Previous attempts at detecting AI writing have been abject and public failures. This is because they attempted to reverse-engineer the way large language models produce text. After every word of a sentence, these detectors asked: if I were an LLM, what word would I expect to come next?<br>For instance, given a sentence fragment such as “I’m going to take my dog for a …,” an AI model might consider “walk” or “run” to be very likely next words, whereas non sequiturs like “harbinger” or “verglas” would not be likely candidates. Early detectors attempted to reconstruct an LLM’s ranking of likely words and compare it with the word that actually appears, so the more often the text followed the predictable choice, the more likely the detector was to classify it as AI-generated. In essence, this reflects how perplexed an LLM would be to encounter a word next in a sentence, so this mode of AI detection was called perplexity analysis.

Perplexity analysis has a lot of problems. It’s not a good measure of the thing it’s trying to distinguish, because human sentences are frequently just as canalized as LLM prose. So predictable or structured human writing is often falsely flagged by perplexity detectors, while at the same time, those detectors are trivially bypassed by obvious strategies like swapping words for less common synonyms.<br>Famously, perplexity models also fail on many texts that predate the advent of the computer. Perplexity-based detection likes to tell you that the Bible, the Declaration of Independence, or Mary Shelley’s Frankenstein are 100% AI-generated. It does this because LLMs read the Declaration of Independence thousands of times during the course of their training, so every next word contained therein is very unsurprising from the LLM’s perspective. The detector misinterprets that likelihood as evidence the writing is AI-generated.<br>Pangram avoids this because it is not trying to reverse-engineer how an LLM generates text. Rather than looking at each word and asking “what would an LLM write here?” Pangram looks at the whole text and asks “who does this sound like?” This way, ChatGPT’s level of familiarity could never cause Pangram to misclassify historical documents as AI-generated – but not every case is so easy.<br>How we trained Pangram

It...