Stereotypical LLMs - by Chenuli Jayasinghe

Ordinary Intelligence Co.

SubscribeSign in

Stereotypical LLMs<br>It is, sadly, not all about training data<br>Chenuli Jayasinghe<br>Jul 08, 2026

Share

If you feed an engineering resume to Claude or ask ChatGPT to write a story about a doctor and a nurse, you should probably expect what happens next. The model, with almost comical predictability, will assume that the engineer is a man, the doctor is a “he” and the nurse is a “she”.<br>When you— or people in general— get mad about stereotypical LLMs, the companies behind them tend to point to the obvious: Garbage in, garbage out. Large Language Models are trained by scraping the entire internet, and the internet, historically, is heavily skewed. The AI is simply acting as a mirror, reflecting our own biases right back at us.<br>While it is undeniably true that these stereotypes originate from the raw training data, it is equally false to say that the data is the only issue. Blaming the internet assumes that if the companies just cleaned datasets hard enough and hired enough human reviewers to filter out every “bad” thing, we would eventually build a perfectly unbiased AI.<br>Which, again, is fundamentally not true. Stereotypical LLMs are a brutal, perhaps unavoidable mathematical necessity of how these models are built, and the data problem is simply fuel to the fire.<br>Thanks for reading Ordinary Intelligence Co.

Subscribe

Stereotypes vs. Nuances<br>We tend to think of LLMs as these libraries where every fact and concept gets its own neat little place, clustered around. In reality, they are more like incredibly lossy ZIP files; you are taking the entirety of human history, culture, and knowledge to a fixed, small mathematical space.<br>To fit all that data into a limited number of parameters, the model has to take shortcuts, for it does not have enough memory to store everything perfectly. This is where things go sideways. To understand what goes wrong here, though, it would help to define what stereotypes are.<br>In human psychology, a stereotype is a cognitive shortcut, more or less; we take a trait and attribute it to a group so we can “understand” how things work (needless to say that it may often be wrong, but that’s another story). In an LLM, too, a stereotype is a literal, if mathematical, shortcut. This is due to the fact that grouping tokens to “engineer”, “aggressive” and “male” into the exact same mathematical corner of the model saves an immense amount of computational space.<br>Nuance, on the other hand, is expensive. A female engineer who is softly spoken and happens to love 19th-century poetry? That is a rare, complex combination which requires the model to create unique trails across its millions of parameters to remember, which is costly, inefficiently so.<br>When you train a model, however, the goal is to lower its error rate and pick the cheapest and most efficient path to get the answer right. And technically speaking, stereotypes are always the cheapest path.<br>More stereotypical LLMs<br>It gets worse. The math actually forces the model to be more stereotypical than the humans who trained it, which is indeed interesting.<br>Let’s say the raw training data does lean toward a stereotype, but it’s not absolute. Maybe in a specific dataset about tech leadership, the pronoun "he" appears 55% of the time, and "she" appears 45% of the time. When the model tries to generate a sentence, it calculates a probability distribution for the next word, but before it spits that word out onto your screen, it passes those numbers through a mathematical filter. This filter does not maintain that 55/45 split. Instead, it might act as a winner-takes-all amplifier, which encourages the highest probability and literally crushes the lower ones.<br>Due to this, the models are designed to flatten out human variance— this results in stereotypes as well as other subtle things, such as how an AI cannot create stories with highly nuanced details. Models are built in a way that if they take a slight, messy societal bias, it would turn into a hard mathematical rule, creating stereotypical LLMs we see every day.<br>Fixing stereotypes in LLMs<br>As established in an earlier essay, you cannot just “delete” these stereotypes and biases from a model. And the safety guardrails, as we know them, are just a cage; the model still “thinks” in the exact same biased math. It’s just not saying it out loud.<br>Then, to actually solve stereotypes in LLMs, we have to rethink the core architecture from scratch. We would need a completely new way of building models— one that does not rely on lossy compression, and one that doesn't use winner-takes-all filters to guess the next word. We would need a computational system capable of holding onto overlapping, contradictory human truths without instantly flattening them into an average, that allows LLMs to highly adhere to the fact that the world is nuanced.<br>As magical (and possible) as it may sound, doing this is incredibly expensive. Until it...