How Microsoft Vaporized a Trillion Dollars, Pt. 3

SubscribeSign in

How Microsoft Vaporized a Trillion Dollars, Pt. 3<br>Inside the complacency and decisions that eroded trust in Azure—from a former Azure Core engineer.

Axel Rietschin<br>Apr 01, 2026

75

Share

(Continued from Part 2)<br>Circling back to the origins of Azure, Cutler’s intent was to produce a system with the same level of quality, unshakable reliability, and attention to detail he was famous for in his work on VMS and NT.<br>In a 2009 interview with ZDNET, he declared that the intent [for the Azure Fabric Controller] was that “it manages the placement, provisioning, updating, patching, capacity, load balancing, and scale out of nodes in the cloud all without any operational intervention. ” (emphasis added)<br>From my years with one of the original contributors to the Fabric, I learned that touching the nodes by hand was also strictly off-limits: the original design intent was that Azure would operate without human intervention.<br>When discussing the discretion around Azure promises at the time, Cutler said, “The answer to this is simply that the RD group is very conservative and we are not anywhere close to being done.”<br>He further added that “[they] are taking each step slowly and attempting to have features 100% operational and solidly debugged before talking about them.”<br>That was on February 24, 2009. A mere 48 weeks later, Azure shipped for general consumption.<br>Fast forward to Summer 2025, and the Secretary of Defense, Pete Hegseth, publicly mentioned “a breach of trust” with Microsoft, following an article from ProPublica describing “digital escort sessions” conducted on Azure computers.<br>The article details how escort sessions involve specialized $18/hour employees who copy/paste and execute commands on government cloud nodes under direction from Microsoft support personnel, often based in foreign countries, including China.<br>However, direct node access and manual interventions are common daily practices that extend well beyond government clouds.<br>Cutler’s vision of a “no human touch” cloud service unfortunately never materialized, as the article mentions “hundreds of interactions” each month for the government clouds alone.<br>The article reveals that the program was devised at the highest levels of the company, with support from CVP-level contributors who declared that “the digital escort strategy allowed the company to ‘go to market faster,’ positioning it to win major federal cloud contracts.”<br>Azure shipped as an unfinished product under intense market pressure, and major corners were cut. Notably, routine manual intervention on the nodes was part of the strategy.<br>Marketing and competitive pressure often work in mysterious ways; however, the article does not explain why manual repairs were needed on the nodes.<br>The answer is now simple: the software didn't work as well as hoped, in large part because the system was rushed under intense pressure.<br>Cue the post-launch talent exodus, its replacement by people of very different experience levels, and you end up with a system that over-promises and under-delivers, drowning in unsolvable problems.<br>This gap between Cutler’s “no human touch” ideal and the reality of hundreds of monthly manual interventions wasn’t abstract for me.<br>In the Overlake team and Compute Node Services, the same underlying fragility I observed since day one, namely chronic crashes, resource leaks, malformed VMs, and a bloated agent ecosystem that no one could fully explain, created exactly the kind of instability that demanded constant human firefighting, including on sensitive government clouds.<br>What I encountered in 2023–2024 was not occasional edge cases, but a steady stream of symptoms from a system that had never been allowed to stabilize, despite the foundations, namely the hypervisor and Windows OS, being robust.<br>The manual escort sessions were, in many ways, the visible symptom of deeper architectural and process debt.<br>I began raising these issues internally, including through formal warnings that eventually reached the highest levels of the company.<br>On one particular occasion, a feature that had been baking for eleven months, intended to exchange secret encryption keys between some actor in the guest VMs and the host OS, generated two Sev-2 incidents within hours of being rolled out to general production.<br>It turned out that one of the agents was calling into another through an unknown endpoint, generating errors that were logged on both sides.<br>An infinite retry loop caused both agents to be busy logging errors, saturating the circular logs and reducing their horizon from the usual 2-3 days to about two hours.<br>This incident illustrates the lack of deep code ownership, overly complex inter-agent interactions, technical leadership gaps, and testing practices that allow major defects to reach production.<br>I distinctly remember asking the dev manager for permission to halt the worldwide rollout, and it took the teams the entire...