From Memory to Agency · arkalabs<br>← TravauxDesign Essay<br>From Memory to Agency.<br>How an AI agent accumulates experience without letting its past call the shots<br>Jeremy Grimonpont Author<br>August 2026 Date<br>~7 min read Reading

An agent's memory is not limited to what it retains. The moment a past experience can shape a future decision, it gains power. This essay lays out the vision behind Cortex.deck: turning traces of work into revisable learnings, then governing what they are allowed to influence.

ContentsYesterday, someone told me exactly what I had said three weeks earlier.

I remembered the conversation. I had shared a hunch: something would eventually have an impact. He remembered something stronger: "You said it was going to break everything."

That version sounded exaggerated. He checked his notes. The phrase wasn't there. He just remembered it that way.

The interesting part wasn't who was right. It was watching two people reconstruct entirely different memories from the same conversation. And how much those reconstructions had shaped his decisions over three weeks.

This is the problem I face every day with AI agents.

The model is not the agent

When we talk about artificial intelligence, we confuse the model and the agent. They are not the same thing.

A model like GPT or Claude is a reasoning engine. It takes a context, produces a response, and stops. Its parameters don't change because a conversation went well.

The agent is the system built around that engine: tools, permissions, files, rules, memory. The same model can power very different agents. Give it a different context and its behavior changes, even though the engine stays identical.

I've used several models in a single Cortex.deck session. They didn't reason the same way, but I never felt like I was switching collaborators. The engine changed. The system persisted.

That led me to a distinction that shapes everything else: experience can belong to the system, not the model.

Why a memory file isn't enough

The first idea looks obvious: save the conversation history. Many agents have a memory file or a full transcript they reread at startup.

Imagine taking notes after every meeting for a year. Two months later, starting a new project, would you reread all your notebooks? Probably not.

Yet that's exactly what we ask agents to do. We inject the full file into the context. Important information gets buried amid outdated decisions and irrelevant details. The token cost is only part of the problem. The noise distorts the reasoning.

I initially thought moving to structured JSON would fix everything. It was a real improvement. But the underlying problem came back fast: content stayed frozen in files, and the agent writing into them could destroy the very material that explained how a conclusion had been reached.

I needed a database. I chose SurrealDB because it runs directly inside Cortex and combines several capabilities in one engine: structured storage, keyword and semantic search, and a native graph model.

The detective's case board

A graph is easy to picture. Think of a detective's case board. A witness claims they saw a red car leaving the scene at 10 PM. Detectives identify the owner. A first hypothesis takes shape.

The next day, surveillance footage shows the license plate was misread. Another piece of evidence places the owner miles away.

In a notebook, you'd cross out the old conclusion and write the new one. But part of the investigation would vanish with it. Why was this person suspected? What made the lead look credible? What eventually broke it?

On a case board, nothing needs to be erased. The testimony stays connected to the first hypothesis by a red string labeled "supports." The footage is connected to the same hypothesis by another string labeled "contradicts." Each connection has a date and a status.

That's what Cortex does. When new evidence arrives, the system doesn't silently rewrite the past. It adds to the chain.

When a conversation becomes an experience

A conversation starts when you open a chat and ends when you close it. Work rarely follows that boundary.

I found a better mental model in TV shows. Each conversation turn resembles a scene. An early scene introduces the problem. Another sets up the plot. A decision is made, an action follows, and a later scene reveals the result. Watched in isolation, the middle scene loses its meaning if the opening scene has disappeared.

The assembly of these scenes forms what I call an Episode. An Episode connects a starting situation, a goal, the decisions made, the actions taken, and the moment understanding shifted. It doesn't manufacture a story cleaner than the sources. Every element stays linked to the exchanges that produced it.

Multiple Episodes compose a season and build a broader understanding. The next season might reveal that an earlier interpretation was incomplete. The story grows without requiring previous versions to be erased.

The cockpit warning...