Dreams, Reflections, and Inceptions - Paper Compute BlogSkip to contentBTW, this is what computers are made of

DocsBlogChangelogLight modeDark modeSearchLogin

ESC

Start typing to search

← All postsThoughtsAugust 12, 2026<br>Dreams, Reflections, and Inceptions<br>bdougieCo-Founder<br>ShareY

I taught an agent to speedrun Pokémon Red. The original story follows the project from its first button press through 1,000 turns. Viridian Forest is where it kept falling apart.

The agent could move. It could battle. It could make it into the forest. Then it would reach the same patch of trees and lose all confidence. It confused two kinds of boundaries: tree stumps were real blockers, but the edge of the screen was not. One run discovered part of the path. The next run started over like that discovery had never happened.

That was the useful part of the demo.

I was not trying to build the world’s best Pokémon player. I was trying to understand what an agent needs in order to get better from experience. Every attempt produced a high-fidelity trace: the prompts, model responses, tool calls, button presses, observations, failures, and recovery attempts. I had the whole record.

The record was not the same thing as memory.

That is where dreams started for me.

A trace knows what happened

The raw material is the trace. tapes records the session at high fidelity, including the parts most agent interfaces hide. For the Pokémon runner, that meant I could inspect what the model saw, what it tried, where it moved, and what happened next.

The model was not navigating a live game. After each turn I captured a screenshot and fed that image back into the next turn. The agent saw a sequence of still frames, not continuous movement. That created a specific perception problem: it often treated the edge of the screenshot as the edge of the playable world. To the model, the viewport boundary looked like a wall.

The Pokédex catches the runner alternating between up and down while the anomaly counter climbs. The screenshot shows the obstacle; the trace shows the repeated mistake.<br>This matters because a summary written after the fact usually preserves the punchline and throws away the path. “The agent got stuck in Viridian Forest” is easy to understand, but it is not enough to improve the next attempt. Which obstacle confused it? What did it try first? Did the recovery work? Was the model reasoning badly, or was the state representation unclear?

A trace can answer those questions. But after ten runs, I did not have one answer. I had ten long transcripts.

High fidelity solves the capture problem. It does not solve the meaning problem.

Dreams are labels

The word “dream” makes this sound more mysterious than it is. The useful implementation starts with a boring operation: label the trace.

One section of a Pokémon run might become an observation like this:

"type": "perception_anomaly",<br>"location": "viridian_forest",<br>"observation": "the model repeatedly treats the screen edge as a wall",<br>"cause": "navigation is inferred from one screenshot per turn",<br>"recovery": "cross the viewport boundary and inspect the next frame"<br>The label is not a new event. It is an interpretation attached to events that already happened. The trace remains the source of truth. The label makes a meaningful moment searchable, comparable, and reusable.

This is the grounding that matters. A dream should not be an agent inventing a lesson because it sounds plausible. It should be an agent reading recorded experience, identifying important moments, and preserving the connection back to the sessions that support the lesson.

A dream is not magic memory. It is structured interpretation over traces.

That distinction is the difference between a story about what your agent did and an artifact you can inspect.

Reflect before you dream

Before I let the system dream across sessions, each session gets a reflection.

A reflection is the note you write at the end of the day before going to sleep. It reduces one session into the observations that may matter later: the decision that worked, the loop that wasted time, the obstacle that appeared more than once, the recovery that finally succeeded.

The reflection does not need to preserve every token because the trace already does that. Its job is to point back into the trace with enough structure that another agent can find the important parts without rereading the entire run.

Speed run through Viridian Forest<br>No statusClaude Opus 5019ff1a2Aug 10, 9:23 AM<br>youSpeed run through Viridian Forest.<br>agentLoaded the forest speedrun skill, checked the ROM and world-map state, and launched the runner from a pristine map.

youWhat happened to the run?<br>agentIt appeared stuck after 8,000 turns. The trace shows repeated up/down decisions, but the captured events confirm it crossed the viewport boundary and exited the forest on map 37.

Reflection<br>Ran a Pokémon agent speedrun through Viridian Forest. The run hit 8,000 turns and looked wedged,...