Toward Self-Improving Agents - Salesforce

Skip to Content

Skip to Footer

0%

Two teams ship an agent on the same Monday. Same model, same use case, same starting accuracy. One team activates a governed self-improvement loop around the model and its harness. The other ships the agent and moves on.

Three months later, one agent is dramatically better and noticeably cheaper to run. The other has been subjected to laborious manual patches that don’t evolve quickly enough to keep up with changing usage patterns, foundation model capabilities, and market expectations.

The foundation model both teams started with has been updated (or will be soon). But both teams got that upgrade for free; nothing about the model explains the gap between those two agents. The only thing that changed was how quickly the first agent learned from its contact with users, and how it improved at addressing the needs of those users.

Today, enterprise agents are often deployed like static applications. Improving them usually means employing human subject matter experts who spot a recurring problem, hypothesize possible fixes, and ship a patch while rolling it back if it didn’t work.

The goal of recursive self-improvement (RSI) , however, is to use the AI agent to automate that cycle at scale: detect what’s failing, diagnose root causes, test multiple improvements using simulations, and learn from the combinations that increase performance on both technical and business KPIs.

At Salesforce, more than 11 million Agentforce calls run each day, and no two sessions are the same. The system can’t learn on its own, no matter how many calls you have, and no team can manually tune every recurring failure at any kind of reasonable cost. This is where RSI can help convert that universe of tasks, contexts, and outcomes into trustworthy upgrades to agent performance.

But if a better frontier model arrives for free every few months, why invest in a loop at all? Because the model is the one part of your agent you don’t own, and it’s depreciating fast. Stanford’s AI Index 2025 found that the inference cost of leading GPT-class models fell more than 280 times in about 18 months (from $20 to $0.07 per million tokens) while the gap between the best open and closed models kept closing.

The durable value, then, lives in what compounds on top of cheap compute: searching over agent designs, learning from accumulated experience. And for an enterprise, that “experience” is your own production traffic.

The upshot is this: A model is something you rent. A loop belongs to you, and it’s something that compounds.

The self-improvement flywheel

To make an agent better, you first must define what “better” means. You set success metrics like accuracy, speed, cost, and business KPIs, as well as guardrails around things like policy violations and regressions. Those definitions are business and product decisions, so owners need to be able to inspect or revise them.

Recent work on agents as iterative optimizers has found that greedy hill climbing often matches these more elaborate methods while requiring fewer separate evaluations, so it’s often fine to start simple.

With definitions of what “better” looks like ingested by the agent, an automated search process begins: Measure the current performance, identify issues and root causes, evolve parts of the configuration, evaluate the candidate(s) with simulations, and keep only the changes that improve outcomes without violating constraints. Then repeat the whole process. Computer scientists sometimes call this hill climbing : iteratively searching for better configurations.

The default method for hill climbing is called greedy search, where you accept the first improvement found. It’s the simplest approach in a broad optimization toolkit that also includes evolutionary methods, multi-armed bandits, and reinforcement learning. Recent work on agents as iterative optimizers has found that greedy hill climbing often matches these more elaborate methods while requiring fewer separate evaluations, so it’s often fine to start simple.

Whichever optimization method you use, the resulting changes are usually larger than a prompt but smaller than an entire product: the system around the model, not the model itself. An optimization engine watches what the agent does; simulates discrete, testable changes; and learns from the combinations that work to repeat the cycle.

Here’s what that flywheel can look like in practice when you optimize the system around the model instead of the model weights themselves:

Open Image ModalThe frozen-weight self-improvement loop is a flywheel in which the agent powered by a fixed frontier model continuously improves through interconnected cycles of autonomous observation, diagnosis, simulation, and human input.

Image Modal

The frozen-weight self-improvement loop is a flywheel in which the agent powered by a fixed frontier model continuously improves through interconnected cycles of...