How to Debug AI Agents with Traces and Evals | No TimeSitemapOpen in appSign up<br>Sign in

Medium Logo

Get app<br>Write

Search

Sign up<br>Sign in

Mastodon

No Time

Share through stories.

Member-only story

How to Debug AI Agents with Traces and Evals

Your AI agent failed, but the chat transcript doesn’t explain why.

Sukhpinder Singh

8 min read·<br>Just now

Listen

Share

Press enter or click to view image in full size

This image was created using an AI image generation program.So someone edits the prompt, reruns one example, and calls it fixed.<br>That is how agent quality turns into guesswork.<br>A better workflow is slower at first and faster later: capture traces, label what actually went wrong, convert those labels into evals, and only then change the prompt, tools, routing, guardrails, or harness. OpenAI’s Agents SDK tracing docs say traces can capture LLM generations, tool calls, handoffs, guardrails, and custom events during an agent run.<br>This article is about that loop.<br>Not observability as decoration.<br>Not dashboards for screenshots.<br>A real trace-to-eval loop.<br>Do not rewrite the prompt until you can replay the failure.

The common mistake: treating the prompt as the whole system<br>When an agent fails, the prompt is the easiest thing to blame.<br>It is visible.<br>It is editable.<br>It feels like the control panel.

Published in No Time<br>10.6K followers<br>·Last published just now

Share through stories.

Written by Sukhpinder Singh

3.1K followers<br>·40 following

C# .Net developer 👨‍💻 who's 100% convinced my bugs are funnier than yours. 🐛💥 #BugLife Pubs: https://medium.com/c-sharp-programming

Help

Status

About

Careers

Press

Blog

Privacy

Rules

Terms

Text to speech