The Missing Step Between Agents and Continual Learning
xg.li
SubscribeSign in
The Missing Step Between Agents and Continual Learning
xg.li<br>Jul 23, 2026
Share
An experience only becomes something you can learn from once reality has signed it.
1. The arrow everyone skips
Liang Wenfeng’s investor talk got quoted to death for one line: the roadmap. Chain-of-thought → agents → continual learning → AI improving AI → embodied intelligence. Most people fixated on the two ends — one end is “once models learn continually they’ll start improving themselves, the singularity is coming,” the other is “eventually it moves into robots, into the physical world.” I care about the arrow in the middle that almost nobody unpacked: between agents and continual learning, what actually happens?<br>He said the next generation of models has to learn over time the way an employee does — to stop showing up with amnesia every morning. I completely agree with that. But “let the agent learn continually” usually gets heard as: give it a memory store. Save the chat logs, save the action history, let it look things up next time.<br>That’s three layers, not one thing. Stuffing a lot into this one conversation is long context. Retrieving relevant material from a database next time is external memory, RAG. Distilling regularities out of long experience, growing new skills, and not forgetting the old ones as you learn the new — that’s continual learning. Liang means the third layer. He wires the agent straight into it, and that middle rung, I think, got skipped — and that middle rung happens to be exactly the problem I’ve spent the last year elbow-deep in.<br>2. Experience doesn’t automatically become knowledge
The hard part of the third layer isn’t remembering. It’s deciding which memories are trustworthy enough to learn from.<br>We build a system that runs Meta ad campaigns. An ad goes out, it fails, Meta returns an error — but “it failed” doesn’t carry the causal story you’d actually learn from. This is credit assignment: the world hands you a signal, and before it earns a spot in long-term memory you have to know what it means and which step to blame.<br>And there’s a split hiding here that the rest of this piece keeps coming back to: feedback comes fast or slow, and the two are worlds apart in how hard they are to attribute. A synchronous Meta rejection throws a precise error code right back — a missing field, an invalid pixel, a targeting option incompatible with a field, a permission or account-status problem. You locate it at a glance.<br>The other kind is nothing like that. The ad publishes fine, ROI sags three days later, the account gets restricted two weeks on. The result is real, but you can barely point to which step caused it — creative fatigue, targeting too narrow, the bid strategy, the account itself? Success lies to you the same way: running fine this time doesn’t mean the method was sound; maybe this account just didn’t get flagged in this window.<br>And the scarce thing was never success. In a production system like this, the harder thing to accumulate is the failure boundary. Positive-signal data is everywhere; negative-signal data is rare, which is exactly why I remember the landmines more easily — and that, in some sense, is my value. What a backend engineer banks over six years isn’t “how much correct code I wrote,” it’s “I know where things blow up, why they blow up, and how to clean up after.” That part is the hardest to distill automatically from one operation after another.<br>I realized later that “data goes stale” isn’t a model-only problem either. A backend engineer six years in is also handling today’s problem with yesterday’s memory. The reason I can still work isn’t that my history never expires — it’s that every time I act, today’s reality comes back and corrects it.<br>So there’s a question I’ve been sitting with: if the model gets smart enough, do I stop having value? How smart, exactly, before I’m worth nothing? Work it through and the answer isn’t “humans have some mystical intuition AI can never learn.” People and models both run on priors built from the past. The real difference is who owns a system that lets today’s reality keep correcting the past.<br>3. To learn for six years, first survive six years
So push it one step further: if I let an agent run six years like a backend engineer — a PM feeding it requirements the whole time, it ships its own code, it handles its own on-call — does it end up better than me? I thought about it for a long time, and the answer isn’t “impossible.” It’s: sure, but it has to survive those six years first, and it has to know which lessons from those six years actually hold.<br>Drop a bare agent straight into production and the first irreversible action can wreck an account, burn money, corrupt data — it dies long before it banks six years. A probabilistic generator taking irreversible, expensive actions will hit a landmine eventually, as long as its error rate isn’t zero — that’s not a...