Recursive Language Models and Neurosymbolic Context Management

Encyclopedia Autonomica

SubscribeSign in

Recursive Language Models and Neurosymbolic Context Management<br>New inference strategies using recursive memory architectures in long-horizon reasoning

Jan Daniel Semrau (MFin, CAIO)<br>May 25, 2026

Share

“Recursion: see Recursion.” — Dictionary definition joke

“To iterate is human, to recurse divine.” — L. Peter Deutsch

The more agents iterate over your workspace, the more long-context reasoning and memory management remain open challenges. As you are well aware, traditional transformer-based LLMs are limited by fixed token windows: once the prompt exceeds the context length, earlier information is simply forgotten.<br>As I wrote in my research paper “Production Rule Systems In Autonomous Agents”, I propose that recursive designs might be an interesting new approach to evaluating intelligence, and Recursive Language Models (RLM) specifically could be a new paradigm with a fundamentally different inference strategy.<br>What is an RLM?

Subscribe

An RLM treats the prompt not as raw text fed into a transformer, but as part of an external environment (e.g., a variable in a REPL) that the agent can programmatically explore.

source<br>More on my work on REPL, GSPO, and GRPO.<br>Concretely, given a long prompt P, the RLM initializes a programming environment with P loaded, and provides the agent with high-level context (like the length of P). The agent then writes code to inspect, decompose, and query P, for example, splitting the input into segments or searching for relevant parts, and recursively calls itself on those subtasks.

Effectively, RLMs work by offloading prompt management to a symbolic layer (code and environment) while using the transformer only on manageable snippets. And maybe it makes sense to reframe “prompt” here as an instruction rather than a question. Especially if you are working within the context of workspace agents, the agent would request additional information and/or trigger a service to execute a task. Naturally, it can be expected that there will be a constant back and forth between the main conversational task and tertiary agents.<br>This leads to frequent decomposition and self-invocation. This approach, if solved by RLMs, lets RLMs handle arbitrarily long inputs (far beyond any model’s token limit) and has been shown to dramatically improve quality on long-context tasks. In experiments with GPT-5 (Zhang et al.), RLMs maintained accuracy on problems 100× longer than its context window, whereas a vanilla transformer’s performance quickly degraded.<br>Transformer-based agents and RLMs thus contrast sharply in principle:<br>Transformers process the entire prompt in one shot (subject to memory and cost constraints), whereas RLMs treat large inputs as a symbolic environment and iteratively “consume” them. Crucially, an RLM still uses the same neural core (e.g. GPT) but augments it with a memory management layer.

The RLM framework is akin to out-of-core computing in databases: small fast memory plus clever external storage. By building a memory-first architecture, RLMs avoid simply concatenating more tokens.

Instead, they mimic programs that fetch, filter, and summarize context as needed. This inversion of the memory role is the key theoretical foundation: RLMs prioritize structured memory access over expanding the raw context window.

As I have already touched upon in Grounded Autonomy: Neuro-symbolic Representations in the Reasoning Loop (Aug 2025)<br>Neurosymbolic Reasoning, Structures, and Context

I believe that neurosymbolic architectures that tightly integrate symbolic knowledge with LLMs to manage context and reasoning operate much more effectively on long-form tasks. A prerequisite for agent implementations in production. In these neurosymbolic agents, high-level structures (e.g., logic rules, symbolic memory graphs, planning modules) work alongside neural networks. For example, neuro-symbolic agents may use an agent as a language interface while delegating precise computation or consistency checks to symbolic components.<br>In fact, Narayanan (2026) describes neuro-symbolic agents as systems that “combine neural models with symbolic structures (rules, graphs, typed schemas)” to gain better controllability and verifiability. These designs preserve the flexible language reasoning of neural nets, while offloading correctness, long-term memory, or discrete planning to symbolic modules.<br>Neurosymbolic context management often takes one or more of these forms:<br>Symbolic memory stores : Here, a graph or database records knowledge derived from the environment. For instance, semantic memory consists of structured knowledge bases or graphs that the agent can query. Such memory can be indexed by concepts or embeddings, enabling retrieval when needed. My Superbill app “Sentinel”, for example, stores facts and rules about a stock in an RDF knowledge graph, then queries it via symbolic logic to inform its...