The Control Layer: Why the Next Era of AI Is About Infrastructure, Not Just Models

Sign in<br>Subscribe

The Model’s the Easy Part - How to Get, and Keep, Value<br>Here’s how I see the evolution of AI in enterprises over the last few years:<br>Autumn of 2022, the world thinks it’s going into a recession. IT budgets are frozen for 2023.<br>November 30, 2022: ChatGPT launches, and the non-technical parts of the C-Suite have a tangible interaction point with AI - a simple chat interface available online.<br>CEOs, CFOs, CROs go home for the holidays and are wowed by early-stage GenAI - making Taylor Swift rap like Eminem, summarizing emails, chatting about travel plans. Impressed, they unlock IT budget only for GenAI pet projects in 2023.<br>2023: pet projects, experimentation. The only new budget was in GenAI, so that’s where all IT teams focused.<br>2024: the great culling of 90% of GenAI pet projects not being promising, and the 10% that were starting to go through GRC for deployment.<br>2025: applications go into production, with varying levels of guardrailing, cost tracking, and ROI measurement. (Also, agentic coding becomes real in late 2025 - so product deployment velocity increases.)<br>2026: internal and external usage of GenAI in production explodes. Annual budgets are blown away in months or less. Concerns around system and data ownership increase with sovereign AI discussions and increased government involvement with frontier labs.<br>Put simply, we’ve moved from "should we experiment with AI?" to "why isn't this in production yet?" to “what’s the ROI, and my lord how much did that cost?!, and where did my data go?!.”<br>Very different discussions!<br>Experimenting is cheap: spin up an API key, see what happens, move on. Production is different. You need reliability, auditability, cost control at scale. You may need hard constraints over geography, on-premises compute, the ability to own your own models. That's why we built Otari. Not because models aren't good enough - in fact, the opposite, so many models are good enough for so many tasks. But, the infrastructure to manage them at an organizational level doesn't exist yet. We're building it.<br>What Changed in the Last Two Years<br>A contentious take: the most important shift hasn't been model capability. Models have improved dramatically, sure. Open source and open weight models especially. But the real change is adoption velocity. AI in production has gone from a handful of well-resourced tech companies to thousands of teams across every sector. With that came problems nobody fully anticipated.<br>First: fragmentation. Most teams aren't using one model - they’re using dozens. At a high-level, that might be a GPT release for text summarization, Claude for coding, an on-prem open-weight model for something sensitive. But even if they’re an “OpenAI shop”, they’ll still have teams using GPT-5.5, -5.4, -5.4-mini, -5.4-nano, and legacy models that worked well at deployment and haven’t been touched since. Each with its own API, pricing, latency profile, rate limits. What looked like flexibility quickly became operational chaos.<br>Second: cost opacity. AI inference scales non-linearly. A feature that costs $200/month in testing can cost $20,000/month in production if usage shifts. This is only getting more important as the “VC subsidies” on tokens lift with the upcoming frontier lab IPOs, and the true cost of a token becomes less opaque. Most teams don't find out what they’re on the hook for until they get the invoice. There's no native tooling across providers to surface this before it's too late.<br>Third: governance gaps. As AI moves into regulated industries - finance, healthcare, legal, education - "which model said what, when, to whom, and why" becomes a compliance requirement. And the sovereign AI discussions happening worldwide add complexity to these requirements. Current infrastructure has no answer for this.<br>The Challenge of Managing Multiple Providers<br>Here's what multi-provider complexity actually looks like in practice. A product team is routing to three or four different model providers, with multiple models per provider - and they’re routing to ad hoc local solutions. They've built custom failover logic for outages. They've got spreadsheets tracking costs. Engineers are manually tuning which model handles which request type based on gut feel and incomplete data.<br>This isn't a sustainable architecture.<br>The problem isn't that teams are doing something wrong. The tooling just hasn't caught up. When cloud computing matured, organizations stopped managing servers manually and adopted platforms that abstracted the complexity away. We're at the same inflection point with AI. Models are the compute. The control layer above them is what's missing.<br>Cost Visibility Is a First-Class Problem<br>Cost is underappreciated as a strategic issue, although that’s changing in 2026 as organizations start to realize how much “tokenmaxxing” is burning capital for questionable return. That said, most...