Introducing Otari: The Open-Source LLM Control Plane
Sign in<br>Subscribe
If you are building LLM-powered applications today, you are probably managing multiple LLM providers, a pile of API keys, and your own logic for routing, budgets, and failovers. Accessing language models is no longer the challenge. Operating LLM infrastructure effectively is.<br>Today, we are excited to launch Otari, the open-source LLM control plane built by Mozilla.ai.<br>A Unified Approach to LLM Infrastructure<br>Otari provides a single control plane for your LLM infrastructure, helping developers and engineering teams manage routing, budgets, governance, deployment and reliability across multiple LLM providers.<br>From one platform, developers can:<br>Route requests across multiple LLM providers through a single endpoint.<br>Control LLM spend with built-in budgets, usage visibility and automatic budget enforcement.<br>Manage governance through centralised API keys, workspaces and access controls.<br>Deploy where you need to using either hosted or self-hosted environments.<br>Improve reliability with intelligent routing and automatic failover.<br>Build agent-ready applications with Agent Harnesses that provide built-in tools and orchestration.<br>Otari gives developers one place to control LLM infrastructure, so they can spend less time managing complexity and more time building AI applications.<br>Why Mozilla.ai Built Otari<br>At Mozilla.ai, we believe developers should be able to build AI systems that are open, flexible and under their control.<br>As we worked with developers and organisations adopting LLMs, we saw the same infrastructure challenges appear again and again. Teams were building custom routing logic, managing provider keys, tracking usage, enforcing budgets and implementing governance independently across every application.<br>These capabilities should not have to be rebuilt every time a new LLM-powered application is created.<br>We believe they belong in a dedicated control plane that sits between applications and LLM providers, giving developers one place to manage and operate their infrastructure.<br>That is why we built Otari.<br>Start Building with Otari<br>Otari is available today.<br>Explore the platform, read the documentation, visit the open source project on GitHub and start building with Otari.<br>Discover Otari at otari.ai .<br>Control your LLM infrastructure before complexity controls you.
Read more
Announcing transcribe.cpp
Meet transcribe.cpp, a new open-source C/C++ speech-to-text inference library with portable, GPU-accelerated support for multiple STT models. Developed through Mozilla.ai's Builders in Residence program, it makes adding fast, local transcription to applications easier than ever.
Using Octonous as a Product Manager
A look at how we use Octonous inside mozilla.ai to reduce the everyday overhead of product work, from turning Slack feedback into GitHub issues to staying on top of product changes and finding context across the tools where work already happens.
Image Classification Comes to encoderfile
Encoderfile now handles images. Starting with image classification, you can run vision models as a single executable — no Python runtime, no serving infrastructure, just a file path in and a label out.
What is an LLM control plane?
Runaway agents? Provider outages? Discover why your AI stack needs an LLM control plane, not just a gateway, to handle production routing, budgets, and privacy.
Powered by Ghost
Mozilla.ai's Blog
Subscribe to get the latest news and ideas from our team
Subscribe