AI doesn't know how to forgive and cannot forget | Tejas Parthasarathi Sudarshan<br>BackThe phrase is forgive and forget, and we say it like the two things are cousins. They aren't. Forgetting is something that happens to you. Forgiveness is something you do. One is a property of the substrate and the other is a skill learned on top of it, and a machine has neither. It cannot forget, because nothing in it was built to decay. And it doesn't know how to forgive, because forgiveness is an operation no architecture we ship actually has.
I want to make that literal, at the level of the plumbing, because this is one of those claims that sounds like a mood and is actually just engineering.
Four memories, none of them fade
When people say "AI never forgets," they're usually gesturing at one system and imagining it's the whole thing. It helps to separate them. A modern AI system holds your past in four different places, and each one forgets in its own broken way, or not at all.
The weights are the oldest memory. Whatever a model saw in training is smeared across billions of parameters, and once it's in, we do not know how to get it out. This isn't a policy gap, it's an open research problem. Machine unlearning, removing the influence of specific training data without retraining from scratch, is unsolved at any real scale. You can delete the row from your database. You cannot delete the gradient it left behind. This is the quiet reason "the right to be forgotten" keeps colliding with machine learning: the law assumes forgetting is a delete, and the substrate has no delete.
The context window is the working memory, and it forgets like a light switch, not a dimmer. Inside the window, attention treats every token as equally reachable; your first sentence is as addressable as your last, with no softening for age. Then the session ends and it is all gone, completely, at once. Two states, eidetic or void. Humans spend our entire lives in the space between those two, and that space is where nearly all of the social machinery lives.
The retrieval store is where this gets its teeth. Embeddings don't blur. Cosine similarity has no time axis. A memory from three years ago is returned at exactly the fidelity of one from yesterday, with no felt sense of long ago attached. In vector space the whole past is co-present, sitting one nearest-neighbor lookup away. The past isn't a foreign country. It's the adjacent row.
And the logs , the snapshots and replicas and backups, mean that even deliberate forgetting is a distributed-systems project: tombstones, retention windows, backups that outlive the delete you thought you ran.
Fig 1 · Fidelity over time
event<br>later<br>high<br>low
AI retrieval
catastrophic forgetting
human memory
Human memory decays and settles. Detail falls away fast, gist survives at a floor. Machine retrieval never moves off the top. And when neural nets do forget, they don't decay, they fall off a cliff.
The forgetting machines do have is the wrong kind
Here's the part that makes the picture stranger than "AI never forgets." Neural networks forget constantly, they're just terrible at it.
Train a network on a new task and it tends to overwrite the old one, wholesale. This is catastrophic forgetting, named by McCloskey and Cohen back in 1989, and it is still the reason you can't casually teach a deployed model something new without risking what it already knew. That's the terracotta cliff in the chart above: perfect retention right up until a sudden, total loss.
And even within a single context, transformers forget by accident. The "Lost in the Middle" work (Liu et al., 2023) showed that models recall information placed at the beginning and end of a long context far better than the same information buried in the middle. Nobody designed that. It's an emergent dead zone, an unprincipled forgetting that lands wherever the architecture happens to be weak.
Fig 2 · Lost in the middle
start of context<br>end<br>recall
the dead zone
An accidental forgetting. The same fact is recalled well at the edges of a long context and poorly in the middle, not by design, but as a side effect of how attention is spent.
So the real situation isn't a machine that remembers everything. It's a machine that either remembers perfectly or loses everything, with almost nothing in between. And that in between is exactly what human memory is.
Forgetting is a feature, not a bug
The thing we call "forgetting" isn't decay for its own sake. Hermann Ebbinghaus measured the curve in the 1880s, memorizing nonsense syllables and watching them slip: fast at first, then leveling off. But the modern reading is the interesting one. Richards and Frankland made the case directly in their 2017 paper The Persistence and Transience of Memory: forgetting is an active, adaptive process that exists to help you generalize. A brain that kept every detail at full resolution would overfit to its own past. Letting the specifics fade is how the gist gets promoted to...