Don't Put All Your Tokens in One Basket | TMD AI is present everywhere these days, whether you like it or not, especially in our industry.The degree of reliance goes to extremes, to the point where a service disruption means no work gets done at all.I am not here to debate the pros and cons of AI in development, but to explore alternative strategies to reduce risks and tame the costs of money-hungry AI machines.What we are trying to avoid<br>Outages continue to be a regular occurrence. Anthropic's status page paints a good picture of the situation, with incidents practically every day. GitHub Copilot struggles to keep up with demand, just to name a few.Rate limits have tightened as providers shift from heavily subsidized to trying to (unsuccessfully) monetise.In early 2025, OpenAI lowered default tokens-per-minute caps for new accounts, and Anthropic introduced stricter tier limits that require escalated payment plans to bypassRegional access remains unpredictable. Anthropic updated its list of supported countries in early 2025, dropping service in several markets without advance notice, while OpenAI's terms continue to shift in response to evolving data sovereignty rules. A global product can find its core feature legally inaccessible in a key jurisdiction overnight not because the code changed, but because a compliance checkbox was unticked.Not to mention the increasing involvement of governmental entities actively in "regulation" with less-than-opaque reasoning, like the debacle around Anthropic's Fable 5 (their latest model family).In a multi-jurisdiction world, trusting a single provider's availability map is an operational gamble.Those are the headlines, broadcasted everywhere. There are subtle and more prevalent issues that can be just as damaging for teams that heavily rely on AI.Silent Downgrades<br>The API stays identical, but the model behind it doesn't. Your prompts stop working as expected, and nobody sends a changelog. We see this particular example with the Fable 5 now back in the picture, which will fall back to Opus 4.8 for various tasks.This could be due to a myriad of reasons:A/B testingCost optimizationSafety or policy updates.The reason is not really important for us, LLM's are not known for their determinism when you add all those extra variables on top it doesn't help.Creeping billing<br>Unlike the early days, the open buffet of tokens is closed. We are seeing the landscape shift from attracting users to desperately showing the world that the AI money-burning-machine can be a sustainable business model.Providers employ many tactics to make sure their IPO is within reach: they deprecate old model IDs, or quietly remove the most cost-efficient variants from their catalogues. Your prompts still work, but the underlying model has been swapped to a more expensive one without a notice period.In 2025, OpenAI began enforcing usage tiers that unlock cheaper rates only after a monthly spend threshold, meaning small teams and startups pay a premium simply for being small. The economies of scale don't get passed down; they get priced out.Token pricing has also been updated to a more complex and difficult-to-predict model: input, output, and even per-route within the same model family have different prices.Strategies<br>The following strategies are not exclusive; they work best when combined into a layered approach that matches your team's risk tolerance and budget.Standardise on the OpenAI API format<br>OpenAI API design has become the de facto standard for LLM interaction. Most providers and local runtimes (Ollama, llama.cpp servers, vLLM, etc.) support this interface, assuming your stack can target it.Write your code against the interface, not against a specific vendor. Swapping providers becomes a simple configuration change, not a code rewrite. This decoupling protects you from vendor lock-in and enables flexible experimentation.Choose the right tool for the job<br>Not everything requires a frontier bazooka. Most people would be surprised how far smaller models have come. Running locally (or self-hosted) won't beat frontier models on raw capability, but they don't disappear, don't raise prices, and don't relay information home. For a lot of tasks—classification, summarisation, structured extraction—they're already good enough.Consider a tiered approach for models:Baseline models, where you can balance speed and performance for most of everyday tasks: Claude Sonnet 4.6, GPT-4o, DeepSeek v3.2<br>Smaller, quicker models for completion or simple tasks: Qwen2.5-Coder 1.5B, DeepSeek-Coder 3B<br>Frontier models, for tasks that require deep, nuanced understanding, like planning a difficult technical solution: Claude Opus 4.8, DeepSeek v4-pro, GPT-5.5<br>It is prudent to also maintain a chain of fallbacks with different providers mechanism ensuring that your core functionality remains robust and independent.Abstract the provider behind your own layer<br>Don't scatter SDK calls across your codebase. Route everything through one...