The Agentic Loop: Three loops in a trench coat

The Thought Drop

SubscribeSign in

The Agentic Loop: Three loops in a trench coat<br>The building blocks for autonomous agents aren't as simple as they seem.

Robert Ross<br>Jul 14, 2026

Share

Agent loops are often oversimplified. They’re presented as a single loop, when really it’s three loops in a trench coat that make up an “agentic” experience for a customer. I’m here to write (yes, I wrote this, insane right?) yet-another-blog about agent loops. The example code blocks are also pseudo-code and for illustrating these ideas. Also I’ve omitted streaming, which complicates the post but the shape of these stays the same.

I even made this image!<br>The Inference Loop

The most reductive way to explain Large Language Models is that they take text, and predict the next charact... err... tokens. Unlike your ex, they really do complete your sentences. This is achieved with an inference loop.<br>Your inference loop has three responsibilities:<br>Make chat completion API calls (infer the next words)

Pass a tool usage request to your tool loop (more later)

Manage chat history persistence (of tool results, or more user messages)

When you’re building an agent, the first “outer” loop you’re creating is the inference loop. This is the loop that sends a system prompt, user and assistant messages, and available tools to an LLM’s “chat completion” endpoint. Anthropic has one. OpenAI has one. OpenRouter makes all of them easy to use.<br>Once an LLM returns its messages, it’s your responsibility to append them to a “chat history” so that a conversation can be continued. This can be an array, a database table, Redis keys, whatever. Your prerogative.<br>chatMessages = [<br>role: "system",<br>content: "You are a couples therapist. You help people either make it work, or make them realize it will never work. The user you're speaking with is named Tom."<br>},<br>role: "user",<br>content: "I kinda miss Laney, what should I write to her?"

continueInferenceLoop = true<br>while continueInferenceLoop<br>inferred = aiClient.completeChat(messages: chatMessages)

chatMessages.push({<br>role: "assistant",<br>content: inferred.message.content,<br>toolCalls: inferred.toolCalls,<br>})

if inferred.toolCalls.length == 0<br>continueInferenceLoop = false<br>else<br># Handle tools (see more later)<br>end<br>end

finalResult = chatMessages.last.content<br>puts "Assistant responded with: #{finalResult}"<br>The API design of (most) Large Language Model providers is a stateless design. This means the model provider has no idea what your previous conversation was with it. You need to provide the entire conversation every time. This is why you see the warning of “Large conversations use tokens faster” — yes I typed an em-dash — because you are sending the tome of messages you’ve built up about whether you should reach out to your ex.<br>Don’t do it, Tom.<br>The Tool Loop

LLMs are brains in a jar. They provide no functional value on their own. The tools you give an LLM are what make it an agent.<br>When you tell a model “here are the tools you have” in your outer inference loop, the model may try to “use” them in its inference (response). This is the same thing as a brain sending an electrical signal telling your index finger to hover over the enter key of the email you desperately want to send Laney. Tom, we need to set boundaries my man.<br>The separate tool definitions you include in your API request are usually serialized into the system prompt field of the token stream the model processes. And it may infer the usage of multiple tools in one turn. (Hence: Tool Loop).<br>chatMessages = [<br>role: "system",<br>content: "You are a couples therapist. You help people either make it work, or make them realize it will never work. The user you're speaking with is named Tom."<br>},<br>role: "user",<br>content: "I kinda miss Laney, what should I write to her?"

tools = [<br>type: "function",<br>function: {<br>name: "send_ex_girlfriend_an_email",<br>parameters: {<br>type: "object",<br>properties: { email_message: {type: "string"} },<br>required: ["email_message"]

continueInferenceLoop = true<br>while continueInferenceLoop<br>inferred = aiClient.completeChat(<br>messages: chatMessages,<br>tools: tools

chatMessages.push({<br>role: "assistant",<br>content: inferred.message.content,<br>toolCalls: inferred.toolCalls,<br>})

if inferred.toolCalls.length == 0<br>continueInferenceLoop = false<br>else<br># Behold... the Tool Loop!<br>while tool = inferred.toolCalls.shift<br># Do tool things in here like sending an email<br># You might have a switch/case statement to control<br># which tool based on the name.<br># Tool calls also have an ID<br>toolResult = "...see next section..."

# Tool result shapes vary by an provider's API<br>chatMessages.push({<br>role: "user",<br>content: [{<br>type: "tool_result",<br>toolCallId: tool.toolCallId,<br>content: toolResult<br>}]<br>})<br>end<br>end<br>endYour tool loop must look up the tool the model inferred it should use, and call your own function with the parameters the model provided as well. But keep in mind a few things:<br>Because tool calls are also inferred text - it...