Instrument an LLM agent with OpenTelemetry · TripleCloud Blog

How to instrument an LLM agent with OpenTelemetry<br>OpenTelemetry can trace an LLM agent the same way it traces the rest of your stack. A hands-on guide to the GenAI semantic conventions: zero-code auto-instrumentation, manual agent and tool spans, token usage and cost, opt-in prompt capture, grouping multi-turn conversations across traces, and where to send the data. None of it ties you to a vendor, so the data stays yours.<br>July 30, 2026<br>18 mins to read

Share

If you run an LLM in production, OpenTelemetry can trace it the same way it traces the rest of your stack. This guide shows how to instrument an LLM agent using the GenAI semantic conventions, the shared vocabulary that every major vendor now maps to: the gen_ai.* attributes plus the chat, execute_tool and invoke_agent spans. It covers zero-code auto-instrumentation, manual spans for agent and tool structure, token usage and cost, grouping multi-turn conversations, and where to send the result. None of it ties you to a vendor, so the data stays yours.

Every step below was run end to end against a local model and a local IceGate, and the runnable recipes are public in our integrations cookbook. IceGate is our open-source observability engine: it receives OTLP natively and stores traces as Apache Parquet and Iceberg files on object storage you own. It is where the traces land in this guide. Any other OTLP backend works the same way, so you can follow along with Jaeger or whatever you already run.

Why trace an LLM agent with OpenTelemetryWhy trace an LLM agent with OpenTelemetry

To ordinary tracing, an LLM call is a black box. You see a slow HTTP request. You do not see which model was called, how many tokens it burned, or whether it looped on a tool. The OpenTelemetry GenAI semantic conventions fix that by standardizing how LLM operations are recorded: the model, the token counts, the finish reason, and, if you opt in, the full content of prompts, completions and tool calls.

The reason to bet on them is who is behind them. The working group's contributors include people from Amazon, Elastic, Google, IBM, Microsoft, Langtrace, OpenLIT, Scorecard and Traceloop, and vendors like Datadog already consume the conventions natively. When that many implementers agree on a schema before it is even finalized, adopting it early is a small risk.

The practical payoff is that you instrument once. Emit standard gen_ai.* telemetry and you can point it at any OTLP backend, then switch backends later without touching the instrumentation.

Where the spec stands does affect how you pin things. The GenAI conventions are still in Development status and most attributes are experimental, so pin your instrumentation versions and expect occasional breaking changes as the spec settles. They also live in their own repository now, open-telemetry/semantic-conventions-genai; the copies still sitting in the main semantic-conventions repo are marked as moved. That split doesn't change any of the attribute names below, but it is the repo to watch.

The GenAI semantic conventions in one tableThe GenAI semantic conventions in one table

Almost everything you'll see on a GenAI span comes from this short vocabulary:

What<br>Attribute / span<br>Example

Operation (also the span name prefix)<br>gen_ai.operation.name<br>chat, execute_tool, invoke_agent, embeddings

Provider<br>gen_ai.system (renamed gen_ai.provider.name in newer versions)<br>openai, anthropic, aws.bedrock

Requested model<br>gen_ai.request.model<br>gpt-4o-mini

Tokens in / out<br>gen_ai.usage.input_tokens, gen_ai.usage.output_tokens<br>412 / 96

Why it stopped<br>gen_ai.response.finish_reasons<br>["stop"], ["tool_calls"]

Agent<br>gen_ai.agent.name, gen_ai.agent.id<br>travel-concierge

Tool<br>gen_ai.tool.name, gen_ai.tool.type, gen_ai.tool.call.id<br>get_weather / function

Content (opt-in)<br>gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions<br>full prompt / response

Span names follow {operation} {model-or-name}, which gives you chat gpt-4o-mini, execute_tool get_weather, invoke_agent travel-concierge. There are also two client metrics: gen_ai.client.operation.duration (a latency histogram, in seconds) and gen_ai.client.token.usage (a token histogram, split by gen_ai.token.type).

There is no combined total-tokens attribute. The conventions define input and output only, so if you want a total, sum the two.

Step 1: Auto-instrument the OpenAI SDK in Python (no code changes)Step 1: Auto-instrument the OpenAI SDK in Python (no code changes)

This step is Python, and so is the application code in every step after it. The package names, the launcher and the environment-variable handling below are all specific to the Python instrumentation. If you're on Node or TypeScript, three of those details change, and the last section of this step covers them.

The official OpenTelemetry instrumentation for the OpenAI SDK is opentelemetry-instrumentation-openai-v2, part of opentelemetry-python-contrib...