What's so hard about continuous learning?Why can’t models continue to get smarter after they’re deployed? If you hire a human employee, they will grow more familiar with your systems over time, and (if they stick around long enough) eventually become a genuine domain expert. AI models are not like this. They are always exactly as capable as the first moment you use them.

This is because model weights are frozen once the model is released. The model can only “learn” as much as can be stuffed into its context window: in effect, it can take new information into its short-term working memory, but not its long-term memory. “Continuous learning” — the ability for a model to update its own weights over time — is thus often described as the bottleneck for AGI1.

Continuous learning is an easy technical problem

However, the mechanics of continuous learning are not hard . The technical problem of “how do you change the weights of a model at runtime” is straightforward. It’s the exact same process as post-training: you simply keep running new user input through the training pipeline you already have. In a sense, every LLM since GPT-3 is already capable of continuous learning (via RL, RLHF, or whatever). It’s just that the continuous learning process is stopped when the model is released to the public.

Internally, the continuous learning process might continue. I think it’s fair to guess that OpenAI’s GPT-5 is constantly training in the background, at least partly on outputs from ChatGPT and Codex2. New checkpoints are constantly being cut from this process, some of which eventually become GPT-5.2 or GPT-5.3. In one sense, that’s continuous learning!

So why can’t I use a version of Codex that gets better at my own codebase over time?

Continuous learning is a hard technical problem

The hard part about continuous learning is changing the model in ways that make it better, not worse . I think many people believe that model training improves linearly with data and compute: if you keep providing more of both, the model will keep getting smarter. This is false. If you simply hook up the model to learn continuously from its inputs, you are likely to end up with a model that gets worse over time. At least right now, model learning is a delicate process that requires careful human supervision.

Model training also has a big element of luck to it. If you train the “same” model a hundred times with a hundred different similarly-sized datasets (or even the same dataset and different seeds), you’ll get a hundred different models with different capabilities3. Sometimes I wonder if a big part of what AI labs are doing is continually pulling the lever on the slot machine by training many different model runs. Surprisingly strong models, like Claude Sonnet 4, might represent a genuinely better model architecture or training set. But part of it might be that Anthropic just hit on a lucky seed.

Learning lessons from fine-tuning

The great hope for continuous learning is that it produces an AI software engineer who will eventually know all about your codebase, without having to go and research it from-scratch every time. But isn’t there an easier way to produce this? Couldn’t we simply fine-tune a LLM on the codebase we wanted it to learn?

As it turns out, no. It is surprisingly non-trivial to do this. Way back in 2023, everyone thought that fine-tuning was the next obvious step for LLM-assisted programming. But it’s largely fizzled out, because it doesn’t really work4. Just fine-tuning a LLM on your repository does not give it knowledge on how the repository works.

It’s unclear to me exactly why this should be. Maybe each individual piece of training data is just too small to make much difference, like a handful of grains of sand trying to change the shape of an entire dune. Or maybe LoRA fine-tuning doesn’t go deep enough to really incorporate implicit understanding of a codebase (which can be very complex indeed). Or maybe you’d need to incorporate the codebase much earlier in the training process, before the model’s internal architecture is already established.

In any case, fine-tuning a coding model on a specific codebase may be useful eventually. But it’s not particularly useful now, which is bad news for people who hope that continuous learning can easily instil a real understanding of their codebases into a LLM. If you can’t get that out of a deliberate fine-tune, why would you expect to get it out of a slapdash, automatic one? There may well be a series of ordinary “learning” problems to solve before “continuous learning” is possible.

Continuous learning is unsafe

Another reason why continuous learning is not currently an AI product is that it’s dangerous. Prompt injection is already a real concern for LLM systems that ingest external content. How much worse would weights injection be?

We don’t yet fully understand all the ways a LLM can be deliberately poisoned by a piece of training data, though some Anthropic...