LLMs don’t just mimic human text - Pangram

SubscribeSign in

LLMs don’t just mimic human text<br>As long as language models are designed to be distinct authors, their output will remain detectable.

Pangram<br>Aug 20, 2026

15

Share

This essay is written by Bradley Emi, co-founder and CTO of Pangram. You can also find it on the Pangram blog.<br>Some people think that AI detection isn’t possible because LLMs are trained to imitate human text, but this is not the case. In fact, LLMs write in a distinctly different way than humans do, and this is what makes AI detection possible.<br>In his Substack essay, Freddie DeBoer writes:<br>“Where do [AI writing] patterns come from? They come from human written texts that are in the training corpora that are used to build LLMs. Every LLM textual pattern - every LLM textual pattern - is ultimately the product of human text production. The very nature of an LLM is to model human text based on human text.”

He concludes that it is therefore not possible to build an AI detector with an extremely low false positive rate, because there’s always a chance that some human can think of and write down the same text string that an LLM would output.<br>We commonly hear different versions of this:<br>“LLMs are sampling a distribution of human writing; therefore you cannot detect LLM generated text because the LLM training distribution is human.”<br>“LLMs are trained to write like humans, so if they are doing their job, AI writing will beat any detector.”

These arguments appear to be sound at first glance, but they fail to capture that modern LLMs are complex systems. Modern LLMs are trained to do much more than just match the human text distribution. In this blog post, we’ll go over why and where modern LLMs, such as ChatGPT, Claude, and Gemini, diverge from the human text distribution and what gives them their unique style and voice that makes their writing possible to detect.<br>Summary: It is tempting to believe that because LLMs are trained to model human text, their output will become indistinguishable from human writing. But in reality, LLMs don’t just approximate human text: they are post-trained to be helpful assistants that follow instructions, which constrains their writing to a narrow set of choices that are detectable algorithmically.<br>In The Beginning: Next Token Prediction

The story of language models began with completion models. These early language models were straightforward: they learned how words commonly followed from each other from a large set of text data. Then, a user would give the completion model a few initial words, and from that the model would predict the most likely next word.

Image credit: AIML.com<br>Under the surface, a completion model takes in an initial string, or prefix, such as “Deep learning is very,” and then it outputs a list of probabilities over what is likely to come next. These probabilities are aggregated over all of the completions in a very large training corpus, mostly of Internet data. The result was a probability model of all human writing, and one that could guess what words were most likely to appear in order given what the natural frequencies of words are in the training dataset.<br>This gave rise to the first language models: GPT-1, GPT-2, and GPT-3. These models are now referred to as base models: they represent the raw distributions over human language, but their outputs are raw and unpolished: they don’t follow instructions, and cannot conduct a conversation. Today, we call this initial stage of language model development pre-training, and it’s only the first stage of making an AI assistant or chatbot — there is a lot more that happens after that.<br>From GPT to ChatGPT

When did we get from a raw distribution over words to being able to talk to a language model like a person? To answer that question, we have to go back to March 2022, when the first major breakthrough in turning language models into assistants happened: InstructGPT, released by OpenAI.<br>InstructGPT was the first time that a language model was explicitly trained not just to imitate human language, but to follow instructions given to it by a user, which we now call prompts.

Here is a figure from the original InstructGPT paper. The original paper demonstrated that you could apply a technique called post-training to customize a base model, in this case GPT-3, to be a helpful, instruction-following assistant — warping the style of the model to take on a specific persona.<br>The paper describes three steps:<br>Human writers provide examples of what they consider to be acceptable AI responses to prompt input

AI researchers develop a reward model that learns which of these provided responses humans tend to prefer

Using the reward model, AI researchers train the LLM to maximize the reward over the course of a large set of generated responses

This is similar to Pavlovian conditioning from psychology! The researchers reward the model for some outputs and not for others, and by repetition, the model...