Just Let Agents Write Code · Redo Engineering

AI agents are everywhere, and everyone is trying to figure out how to make them work best.<br>Ask any engineer building AI products and they’ll tell you their favorite combination of model,<br>harness, router, eval system, and workflow system. Ask them again a week later, and they’ll have a new answer.

But one pattern that keeps popping up over and over and over again is the concept of “Code Mode”.<br>Originally named by Cloudflare last year, code mode is where agents write code to access tools as functions instead of explicit “tool calls” in the LLM output.

At Redo, we believe this “code mode” is a powerful unlock for the agents we’re building, and have created a homegrown sandbox that’s both fast and secure, and works seamlessly with the rest of our systems.

But first, a quick primer.

What is code mode???

In case you aren’t familiar with the concept, here’s how it works.

Normally, to let an AI agent work in your system, you’d expose a series of tools giving it access to various CRUD operations.<br>For example, here we are exposing tools for searching through products and updating their prices.

import { generateText, tool } from 'ai';<br>import { z } from 'zod';

const result = await generateText({<br>tools: {<br>searchProducts: tool({<br>inputSchema: z.object({<br>query: z.string(),<br>}),<br>execute: async ({ query }) => {<br>// ... find the data and return it<br>},<br>}),<br>updateProductPrice: tool({<br>inputSchema: z.object({<br>id: z.string(),<br>newPrice: z.number(),<br>}),<br>execute: async ({ id, newPrice }) => {<br>// ... save the updated price for the given product id<br>},<br>}),<br>},<br>prompt: "Please increase the prices of all red shoes by 10%."<br>});<br>For the agent to update product prices, it first has to call the searchProducts tool (which would return all the relevant info about any products that match the query), and then, for any products that need updating, it calls the updateProductPrice tool with the product IDs and the new prices.

For an agent with a couple of tools and a simple task, this can work well! It’s still wild to me how good the LLM is at regurgitating exact IDs for future tool calls; the models have gotten so good that they rarely make mistakes here.

But there are some other problems with this approach:

To update the product price, we had to make three round trips to the LLM, more depending on how many times it wanted to call the updateProductPrice tool.

A large amount of unnecessary information was added to the context, i.e. all of the product data.

LLM Harness increase red shoe prices by 10% searchProducts('red shoes') 200 products · every row updateProductPrice(id, +10%) ok … repeat per matching product ×N<br>Code mode solves these problems by having the agent write code that interacts with the tools deterministically, giving it more control over what data it gets and what it does with it. Tool calls can be chained together, with conditional statements, loops, and functions, providing all the flexible flow control we love about code.

This technique requires exposing only two actual tools to the LLM:

search-tools accepts a query string and enables the LLM to discover which functions are available in the code. The tool response is a list of the matching functions, their parameters, and a description of how to use each one, similar to how a “normal” tool would be sent.

execute-code accepts a chunk of code to be executed in the sandbox and returns the results. The response in this case is the text written to stdout / stderr during code execution.

This not only enables a very large number of tools to be exposed to the LLM, but also lets the agent take many chained actions much much faster and cheaper, due to not filling the context window.

LLM Harness Sandbox increase red shoe prices by 10% executeCode("...") run the script found = searchProducts(...) for p in found: updateProductPrice( p.id, p.price * 1.1 ) 12 prices updated ok<br>This seems simple, until you realize you have to execute the untrusted code the agent wrote, connect the tool implementations, validate tool arguments, capture stdout and stderr, enforce permissions, allow human-in-the-loop, and about a dozen more things… all without the security team hunting you down.

Remote Code Execution as a Service

At first, we tried several code sandbox tools that basically ran a new container for each request. Linux containers are great for sandboxes, and cgroups let you control pretty much everything: CPU limits, memory limits, network access, etc. Plus, you can run any language you want by just installing it in the container.

In order for the tool calls from the agent’s code to run our implementations, we created a small library that the LLM-generated code could import, giving it tools that, when called, make HTTP requests to our internal services. This worked fairly well, but adding new tools was painful because it required modifying the container image each time, not to mention a secondary auth path, which added complexity around...