Your agents don't belong in your codebase - Empirical
/blog/agent-runtime<br>Your agents don't belong in your codebase
Arjun Attam<br>July 16, 2026
On this page<br>Like most AI agent products, we’ve gone through our fair share of product<br>rewrites as the foundation models have evolved. Our last rewrite (fourth? by some<br>measure) was anchored on splitting the agent runtime from the agent definition .
The runtime is how an agent runs: the loop that calls the model and executes<br>tools, with capabilities like receiving messages mid-turn (“steering”)<br>or checkpointing work. The definition is what an agent is: its system<br>prompt, its model, and which tools, skills, and subagents it has. The runtime is<br>in code, but the definition is just data.
This split has served us well: agent iterations are much faster, and<br>the runtime feels more solid than before. This is not strictly a new idea: any<br>“managed agents” service, like the ones from Anthropic or Vercel, ship a runtime along with<br>an interface to define the agent. We chose to (mostly) hand-roll ours: important for<br>us to have minimal external parts between us and the model.
Background
Our previous stack ran the agent loop in a Cloudflare Durable Object,<br>with external infra for heavy tools that needed headless browsers. It organically<br>grew to expand runtime capabilities (e.g., edit messages to create new branches<br>for message history). The agent definition lived next to it, and everything was<br>versioned with git. That worked well until we realized we wanted two new agents:
A code review agent that adversarially reviews the main agent’s output.
A video analyzer subagent that reads video frames without inflating the<br>main agent’s context window.
Both needed their own context window, system prompt, and tool set, plus integrations<br>with the main agent. We managed to fit them in, but then realized we needed to support a lot<br>more: more custom agents that had to run independently or as subagents, and versions of<br>those agents we wanted to evaluate separately.
Should we outsource the runtime?
“How the agent runs” and “what the agent does” are distinct problems, and<br>the refactor forced us to question why we were solving both in-house.
Building the runtime in-house was always a questionable decision. We had looked at<br>LangGraph back when it was the default answer, and a few things<br>put us off: too many building blocks that a basic agent loop does not need, docs that<br>lagged the actual code, and a Python-first approach (we’ve always been TypeScript). Mastra<br>and others prioritized TypeScript, but they all seemed to overcomplicate a basic<br>loop.
The “thinking in<br>LangGraph”<br>approach makes agent development sound like a “graph problem”, but it is not. It is<br>about the model’s context, when it reaches for a tool, and where its judgment breaks<br>down. Fighting an external abstraction in the core product path feels like taking<br>valuable time away from actually building the product.
Discovering Pi
But then I found Pi. Others have already<br>written the love letters, so I’ll keep<br>mine short: a minimal core, an extension system that can carry real weight, and<br>source code that’s a pleasure to read.
We realized that we can run Pi in RPC mode, and the<br>extensions API was enough to cover all of our runtime needs. Today<br>our agent runtime is basically the Pi core, with 16 custom extensions around it.
subagents custom-message-poller git-checkpoint set-alarm memory upload-media session-entry-sync agent-lifecycle telemetry system-prompt-capture provider-cache-diagnostics vertex-anthropic impacted-tests run-test-report create-matcher run-matcher<br>subagents Turns each configured helper agent into a callable tool, running it with its own context window and returning its output to the parent.
“Hive” is where our agents live
Pi + extensions solved our agent runtime, and our agent definitions needed a place<br>to live. That became Hive, an internal tool that solves “versioned prompt management”<br>for our agents, tools, and skills.
Every off-the-shelf LLM evaluation/tracing tool bundles a “prompt management” feature<br>that probably does 50% of what you want. When we started building<br>Empirical, we explored a few of them and then chose to keep everything in the<br>repo. Git gives versioning and code reviews for free, right?
In 2026, you can build your own tool that does 100% of what you want. Building<br>reviews and versioning with a sick diffs UI library is a few<br>sessions with your favorite coding agent.
Hive is now our central registry for agents, their versions, and the tools and<br>skills they can use. We’ve also built in support for shared “prompt blocks” that<br>render into more than one agent’s prompt.
Here’s how Hive expresses an agent definition in TypeScript.
interface HiveAgentVersion {<br>version: number;<br>system_prompt: string;<br>model: string;<br>tools: string[];<br>all_tools: boolean;<br>skill_refs: string[];<br>all_skills: boolean;<br>subagent_refs: string[];<br>Agents become data that implements the type. As an illustration, the Pirate...