How Much Memory Does Your Agent Actually Need?

Log In<br>Sign Up

Back to Articles<br>a]:hidden">

How Much Memory Does Your Agent Actually Need?

Enterprise Article Published<br>August 18, 2026

Upvote 33

+27

Vatche Isahagian Vatche Follow

ibm-research

Gaodan Fang gaodan-fang Follow

ibm-research

Jayaram Radhakrishnan jayaramkr Follow

ibm-research

Punleuk Oum illeatmyhat Follow

ibm-research

Ashwath Vaithinathan Aravindan ashwath-vaithina Follow

ibm-research

Evelyn Duesterwald evduester Follow

ibm-research

G Thomas gsthomasx Follow

ibm-research

Vinod Muthusamy vinodmut Follow

ibm-research

Merve Unuvar mrvnvr Follow

ibm-research

Ayhan Sebin ayhansebin Follow

ibm-research

In our previous post, we compared ALTK-Evolve with ACE and showed that how you deliver an agent's self-distilled guidelines — a few retrieved per task vs. the whole set injected — drives both accuracy and cost. This post steps back to the question that comes before it: how much should you give it?

Equipping an agent with agentic memory sounds simple: distill lessons from its past work, put them back in context, and more experience should mean better performance. It doesn't always work that way. When we scaled the evaluation to eight models — from a 30B dense model to frontier proprietary systems — one finding stood out:

Agentic memory is not a feature you switch on. It's a dose you calibrate to the model.

TL;DR

ALTK-Evolve lets an agent learn from its own past trajectories: distilling reusable guidelines and injecting them back at inference time, with no weight updates and no human annotation.

The right dose differs by model tier: strong models with headroom want the full guideline set, weaker models do best with a compact core plus per-task retrieval, and saturated models show no measurable gain.

Curated retrieval can be both the most accurate and the cheapest option: gpt-oss-120b gained +16.1pp task completion at only +5% tokens — and prompt caching keeps even the full guideline set affordable in production.

The Key Insight: Dosage Depends on Capability

Not every model benefits from the same amount of memory. Across eight models spanning the capability spectrum, we saw three recurring patterns:

Strong models with headroom want the full guideline set — every guideline, including rare edge-case lessons. They have the capacity to absorb and apply all of it. DeepSeek-V3.2 (671B MoE) climbed +9.5 percentage points in task completion when given its full self-mined guideline set.

Smaller or weaker models get drowned by a large guideline set. For these, a tight, high-confidence core plus a handful of task-relevant guidelines retrieved per task works best. gpt-oss-120b (117B MoE) gained +16.1pp with this selective approach — while the full guideline set gained less and cost ~50% more tokens.

Already-saturated models show no measurable gain. We call this the saturated pattern — the label describes what we observed, not a proven cause. The model may already have been near its ceiling on these tasks, the guidelines may not have addressed its remaining failures, or it may not have applied the guidance effectively. GLM-5 (745B MoE) sat here in our runs.

What puts a model into one pattern rather than another isn't simply parameter count. Benchmark headroom, context-window size, architecture, guideline quality, and task distribution all appear to shape where a model lands, and separating those factors is ongoing work. The practical takeaway holds either way: the right dose of memory depends on the model, and we can calibrate it.

Learning happens around the model, not inside it

"Memory" here doesn't mean replaying a past transcript. It means a guideline set — strategies that worked, mistakes to avoid, and edge cases — distilled from the agent's own prior trajectories. The loop is straightforward:

The agent attempts tasks and produces trajectories.

ALTK-Evolve extracts behavioral guidelines from both its successful and unsuccessful runs.

It consolidates those guidelines into a reusable set.

At inference time, the agent receives either the full guideline set or a task-relevant selection of it.

No model weights are updated. The learning loop changes the guidance available to the agent, not the underlying model — which is exactly why it's cheap to adopt and portable across the eight models we tested.

Results Across the Spectrum

We evaluated on AppWorld — 585 multi-step tasks (168 test_normal + 417 test_challenge) across 9 simulated apps (calendars, messaging, payments, and so on). Tasks are scored two ways: whether the agent fully completes each task (TGC — Task Goal Completion ) and whether every variant of a scenario passes (SGC — Scenario Goal Completion , a stricter, all-or-nothing bar). Full definitions are in the appendix.

The three configurations we compare

Because the confusing part of any memory study is what's actually in the context window, we define the configurations up front.

Both memory configurations draw...