Language model harnesses are compositional generalizers | Alex L. Zhang

%F0%9F%AB%A1">

Language model harnesses are compositional generalizers

Harnesses can lead to compositional generalization: we observe a property in training RLMs, in which similarly structured tasks are viewed as isomorphic and all individual LM calls in the harness become in-distribution.

Modern post-training has become a brute-force paradigm of curating ever more environments and ever longer training horizons. In large part, this is because frontier Transformers are still poor at compositional generalization, the ability to solve unseen problems by composing familiar ones. Unless our models compose the individual lessons they learn, scaling will have slower returns than it should, as every new domain will demand its own investment in the form of training data.

Training data is not the only lever, of course. For the past few years, we’ve attacked harder tasks by scaffolding the Transformer, first with chain-of-thought reasoning, then with tool use, and so on. Yet across all of this, generalization itself has been left to the underlying neural network and to its 2017 token-level inductive biases. We argue that better generalization is largely the job of what today is called a harness. A harness is the program that sits between the external world and the neural network: it decides how to encode the current state of the environment, which can be arbitrarily long and complex, into one or more inputs to the LLM and how to determine the next action. The primary job of the harness should be to carry a higher-level inductive bias that can reduce unfamiliar and complex problems to compositions of simpler ones for the underlying neural network.

Concretely, we think a good harness is one that shapes each call to the underlying Transformer so that every observation is locally in-distribution , i.e. each Transformer call handles a prompt that is in-distribution with respect to its training data. In fact, a good harness can frequently reduce problems that seem to require breakthroughs in post-training into almost mundane capabilities of the existing generation of language models. We first showed a version of this for long context processing nearly a year ago and, in this post, we show that this principle extends to the efficiency of learning itself. That is, what a model learns through a well-designed harness generalizes across task lengths and across domains far better than training the neural network on its own does.

We test this by using reinforcement learning (RL) to train a Recursive Language Model (RLM), a harness in which the model offloads its context and defers execution to programmatic decomposition and recursive sub-calls. The results are summarized in Figure 1. Training on only short tasks generalizes to held-out tasks 8–32x longer, with roughly 10x the eval lift with the same train lift over training the underlying Transformer directly. Moreover, training on one domain transfers to other domains at a far better rate than that of a vanilla Transformer.

Figure 1. We train Qwen3-30B-A3B-Instruct-2507 as an RLM and as a base Transformer (+ YaRN) on a suite of tasks, and plot train and eval lift. On the left, we independently train 6 different short task environments, and evaluate on the same task environments, but a split that is 8-32x longer in context length. On the right, we train on 3 different task environments, and evaluate on 3 different domains that use a similar decomposition strategy. We find that the eval lift for the RLM matches or exceeds the train lift, while the base Transformer struggles to generalize. In some length-generalization experiments, we find that the RLM starts with a non-generalizable solution that only works on short tasks but eventually discovers a more generalizable decomposition strategy, yielding a higher eval lift than train lift.

We observe this generalization effect because the RLM harness induces an equivalence relation between tasks with latent similarities, meaning the RLM’s main context sees nearly the same token-level trajectory between these tasks. This is illustrated in Figure 2. These results suggest that, similar to tuning model architectures and training recipes, well-designed harnesses can reduce the cost of curating more data and generating longer rollouts while increasing the coverage of tasks solvable through post-training.

Figure 2. An illustrated example of a Recursive Language Model (RLM) harness on two different tasks, BrowseComp-Plus and OOLONG, which can (in theory) see the exact same trajectory on the root LM's context window by deferring task-specific queries to its sub-calls and moving information through REPL variables. If the RLM trained on one task, it would generalize to the other because the root LM now can see them as isomorphic. This isomorphism occurs when two tasks share latent structure that the RLM can exploit programmatically using sub-calls, while offloading...