Observability Design for the AI Era — Application / Infrastructure / CI / LLM, Each in Its Own Shape (Part 1) — ryantsuji.dev<br>Contents<br>What Does "Observable to AI" Even Mean?<br>Application — OTel + Loki + Tempo, the Standard Stack<br>Infrastructure — Cloud Run / BigQuery / Pub/Sub Metrics, All Into Mimir<br>CI — Ship Logs to Loki via Post-Hoc Pull, Not Webhook Push<br>LLM — Gemini and Claude Code, Two Different Shapes<br>Gemini — Prometheus, Cost Visible in Real Time via Client-Side Estimation<br>Claude Code — Send to BigQuery, Built for SQL Aggregation<br>To Be Continued<br>What Does "Observable to AI" Even Mean?<br>Application — OTel + Loki + Tempo, the Standard Stack<br>Infrastructure — Cloud Run / BigQuery / Pub/Sub Metrics, All Into Mimir<br>CI — Ship Logs to Loki via Post-Hoc Pull, Not Webhook Push<br>LLM — Gemini and Claude Code, Two Different Shapes<br>Gemini — Prometheus, Cost Visible in Real Time via Client-Side Estimation<br>Claude Code — Send to BigQuery, Built for SQL Aggregation<br>To Be Continued
Hi, I'm Ryan, CTO at airCloset.
In the previous series, code-graph deep dive (Part 2), I wrote about making a 46-repo codebase semantically searchable for AI. The final issue I left open in that piece was the absence of dynamic analysis :
What lives on the graph is the fact that "this edge exists statically." How often that edge actually gets used in production isn't recorded.
A graph that gives you static facts is one thing. Telling AI what's actually happening in production right now is a separate problem. So the same shaping discipline I applied to the static graph needs to apply to the observability stack too.
This post is the first half of that story. I split it into two: Part 1 (this post) covers how I shape four different monitoring surfaces (application / infrastructure / CI / LLM). Part 2 covers PII handling, the integration surface, and Self-Healing — coming a week later.
What Does "Observable to AI" Even Mean?
The biggest lesson from the code-graph series was: the data has to be shaped before AI can consume it . Throwing 46 repositories of source at a model blows past the context window and invites hallucination. So we shaped it — static analysis into a graph, boundary nodes given meaning, SAME_ENTITY joins between graphs — and only then handed it over.
The observability stack has the exact same problem. Throw raw production logs at AI and you get:
Sheer log volume that drowns the context window
No way for the model to tell errors from noise
Metrics, logs, and traces that don't link to each other
Questions like "what are we spending right now" that raw logs don't answer at all
In other words, logs have to be reshaped before AI can use them. Same problem, different domain.
The catch is that the right shape depends on what you want AI to answer . At cortex (the internal AI platform), I split the monitoring surface into four axes and let each one settle into its own form:
Note : "cortex" here refers to airCloset's internal AI platform codename. Unrelated to Snowflake Cortex, Palo Alto Networks Cortex, etc.
Monitoring target<br>What you want AI to answer<br>Shape
Application<br>"What's happening in production right now?" (exploration)<br>log + trace
Infrastructure<br>"Do we have enough resources? Anything down?" (time series)<br>metric
CI<br>"What broke? Since when?" (alert + history)<br>log + alert
LLM<br>"How much are we spending? Who's using how much?" (real-time + structured aggregation)<br>metric + structured records
"Just push everything through OTel and dump it all in Loki" is an option. But the moment you do, you're asking one backend to answer wildly different kinds of questions — real-time "what's spending right now" alongside "monthly cost broken down by team via SQL" — and one of them is going to suffer. Splitting by purpose is the choice I made.
Let me walk through each of the four axes. Application and infrastructure are the foundation, so I'll keep those brief. CI and LLM are where the AI-era design judgments actually surface, so I'll dig into those.
Application — OTel + Loki + Tempo, the Standard Stack
The foundation is unremarkable. Every cortex application is instrumented with OpenTelemetry, with traces going to Tempo, logs to Loki, and metrics to Mimir — the standard Grafana Cloud setup.
There's no special trick here. What matters is the discipline: every app emits logs and traces in the same shape . That uniformity is what lets AI later run something like {service_name=""} |~ "error" through MCP and investigate across services.
I covered the actual instrumentation in AI Harness Series Part 4 (Self-Healing), so I'll leave the details there. The point worth repeating is: a standard OTel stack, properly laid down, is the precondition for everything AI-driven that comes later .
Infrastructure — Cloud Run / BigQuery / Pub/Sub Metrics, All Into Mimir
cortex runs on GCP and stitches together Cloud Run, Cloud Run Jobs, BigQuery, Pub/Sub, Cloud Tasks, and the usual suspects. Each GCP resource's metrics (CPU, memory, execution...