Self-Improving Agent Systems: A Unified View of Algorithms, Skills, and Harnesses - Yigeng’s Blog

Self-Improving Agent Systems: A Unified View of Algorithms, Skills, and Harnesses

yigeng & GPT 5.6

2026-07-12

{Machine Learning}

[self-improvement,

Agent]

Self-improvement is often discussed as if it meant a model rewriting its own weights. That is one possible endpoint, but it is not where most practical progress is happening today. Deployed agents are compound systems: a foundation model is surrounded by prompts, tools, workflows, memory, Skills, sub-agents, evaluators, and runtime policies. Changing any persistent part of that system can change what the agent is able to do next.

This article develops one framework for reading a fast-growing and easily fragmented literature. The central claim is simple:

Self-improvement is an evidence-driven, persistent state transition whose effect on future behavior can be evaluated.

The primary difference between prompt evolution, workflow evolution, Skill evolution, Harness evolution, and self-referential agent evolution is therefore not how ambitious their names sound. It is what state they allow to change . A second, independent question is who constructs the successor . Evidence, search topology, update timing, and governance form additional axes.

The result is a deliberately bounded account of self-improvement. Most systems covered here keep the foundation model frozen and improve model-external state. Joint model–Harness adaptation appears only as a boundary case. The paper-to-paper arrows below should be read as conceptual relationships unless an explicit methodological inheritance is stated.

1. Preliminaries

1.1 Scope, Thesis, and Unit of Analysis

The unit of analysis in this article is not an isolated language model. It is a deployed agent system together with the machinery that proposes, evaluates, archives, and promotes changes.

For predominantly frozen-model settings, a useful engineering decomposition is

\[<br>A_t=(\theta_t,H_t),<br>\]where \(\theta_t\) denotes model parameters and \(H_t\) denotes persistent model-external state. In most of the systems discussed here, \(\theta_{t+1}=\theta_t\) while \(H_{t+1}\neq H_t\). The model is unchanged, yet the behavior of the deployed system may improve because its instructions, control flow, tools, reusable capabilities, or runtime have changed.

This framing excludes two broad areas from the main discussion. First, parameter-only methods such as ordinary fine-tuning are outside scope unless they are explicitly coupled to Harness adaptation. Second, one-off reflection, best-of-\(N\) answer selection, and temporary in-context adaptation are not persistent self-improvement by themselves. They may generate useful evidence, but they do not automatically create a better future system.

It is helpful to describe the whole improvement system as

\[<br>\mathcal{E}_t=(A_t,U_t,L_t;V,B),<br>\]and then separate its functional roles:

Symbol<br>Role<br>Typical realizations

\(A_t=(\theta_t,H_t)\)<br>Active agent system<br>Model plus prompts, workflow, tools, Skills, memory, and runtime

\(U_t\)<br>Updater<br>Prompt optimizer, coding agent, Refiner, MetaAgent

\(L_t\)<br>Lineage or archive<br>Best-so-far state, Pareto pool, search tree, population, open archive

\(V\)<br>Evaluation and promotion<br>Tests, validation tasks, preference gates, rollout metrics

\(B\)<br>Governance boundary<br>Permission enforcement, credential boundaries, sandboxing, budgets, provenance, audit, rollback

This is a functional decomposition, not a claim that the sets are always disjoint. Hyperagents, for example, embeds part of the update mechanism inside its editable program, so \(U_{\text{embedded}}\subseteq H_t\), while parent selection, evaluation, and other outer-loop functions remain external. The distinction still matters: a system can expose a broad editable surface while keeping its evaluator and authorization enforcement protected. If the candidate can rewrite both its behavior and the standard used to judge that behavior, a higher score no longer establishes improvement.

1.2 Agent Systems as Persistent Editable State

A deployed agent is better viewed as a stateful program than as a single model call. The model produces actions, but the surrounding system decides what context reaches it, which tools exist, what results persist, how errors are recovered, and when another model call is made.

flowchart LR<br>X["Task or environment"] --> H["Harness H_t"]<br>T["Frozen or slowly changing model θ_t"] --> H<br>H --> Y["Actions, artifacts, and trajectory τ_t"]<br>Y --> E["Evidence: outcomes, traces, failures, cost"]<br>E --> U["Updater U_t"]<br>U --> C["Candidate persistent state H'"]<br>C --> V["Protected evaluation V"]<br>V -->|"promote"| H<br>V -->|"admit as safe candidate"| L["Lineage / archive L_t"]<br>V -->|"invalid or unsafe"| Q["Rejected-candidate buffer"]<br>B["Governance B"] -.->|constrains edits and deployment| U<br>B -.->|protects| V

This diagram already reveals two different meanings of “self.” At the...