GitHub blames 8-hour outage on autoscaling fail and VS Code retry storm

stevehipwell1 pts0 comments

GitHub blames 8-hour outage on autoscaling fail and VS Code retry storm

Jump to main content

Search

REG AD

SAAS

GitHub blames 8-hour outage on autoscaling fail and VS Code retry storm

Load balancers buckled after a monitoring blind spot allowed traffic to spiral

Richard Speed

Richard<br>Speed

MICROSOFT ECOSYSTEM REPORTER

Published<br>wed 19 Aug 2026 // 12:40 UTC

GitHub has published its account of this week's nearly eight-hour outage, tracing the developer pain to saturated load balancers, a faulty autoscaling policy, and a "latent retry bug in Visual Studio Code."<br>According to GitHub, problems began at 1328 UTC on August 17 and weren't fully resolved until 2115 UTC – a 7-hour, 47-minute incident that produced elevated errors across Issues, Pull Requests, APIs, Actions, and Copilot.<br>The immediate cause was network saturation on load balancers in the company's Central US facility, triggered when an Istio sidecar reached its concurrency limit.

REG AD

Surely autoscaling would add capacity as those limits were reached? Alas, no. A misconfigured policy monitored the host service but not the sidecar's concurrency limit, allowing a cascading failure to develop. "The problem," according to GitHub, "was worsened by optimistic retry logic which overloaded internal load balancers."

REG AD

Engineers mitigated the problem by temporarily reducing gateway retries through a code change and configuring the load balancers to reject inbound Copilot Token Service requests with HTTP 403 responses.<br>Ah yes, Copilot. GitHub explained: "Delayed replies to a single internal endpoint triggered a latent retry bug in VS Code that amplified traffic by approximately 10x and caused delayed recovery for the Copilot Token Service."<br>Most services recovered by 1636 UTC and Actions by 1803 UTC, but the Copilot Token Service took until 2102 UTC.<br>"Complicating factors that impeded recovery included a number of scraping attacks on codeload endpoints," GitHub added.<br>The Microsoft tentacle says it will correct the autoscaling policies, review retry limits, audit Istio concurrency settings, and address the VS Code behavior "that amplified Copilot token traffic."<br>This latest incident could be the tipping point that sends some developers scurrying for an alternative. CloudBees CEO Moritz Plassnig noted in a LinkedIn post that "Cursor, OpenAI and several smaller startups are already building competitive solutions."

MORE CONTEXT

Holy git! Microsoft code-sharing site suffers downtime, despite move to Azure

GitHub has Issues as repo downloads hit 50% error rate

Latest GitHub outage squeezes Actions, Pages to death

GitHub Actions outage told devs 'your account is suspended'

"GitHub won't be the default solution going forward and we are looking at a much more bifurcated ecosystem (which is good and bad)."<br>The findings will raise eyebrows among engineers. A misconfiguration and retry storm degraded a critical piece of infrastructure on which many organizations depend, leaving developers unable to work normally for hours.

REG AD

GitHub's reliability problems stretch back well beyond this week, as the company itself acknowledges. Developers have choices, and the pain/gain equation is not looking too favorable for the source shack. As Plassnig noted, alternatives keep cropping up – sometimes at the most awkward moment.<br>As GitHub staggered, SpaceX-owned Cursor announced an early beta of Origin Code Hosting. ®

microsoft<br>github<br>copilot<br>saas<br>visual studio code

REG AD

DATABASES

Postgres pioneer credits Oracle with helping his database take over the world

Big Red's acquisition of MySQL sent users running toward the open source alternative, says Michael Stonebraker

SAAS

GitHub blames 8-hour outage on autoscaling fail and VS Code retry storm

Load balancers buckled after a monitoring blind spot allowed traffic to spiral

devops

Platform Engineering 2.0: your platform was built for a different era. AI just exposed it

PARTNER CONTENT: Platform engineering won the argument. Now it has to grow up fast and evolve for the AI era.

ON-PREM

Scotland's AI Growth Zone gets a wee £300M power-up

DataVita secures funding to expand datacenter capacity, while Dell brings some desks

COLUMNISTS

Be a hater all you want, AI's here to stay

The good news? One of the worst bits, tech giants controlling it all, might soon be over

SOFTWARE

SAP consultant job ads overshoot final offers by up to 12%, recruiter claims

Both sides apparently start high then price in the inevitable haggle

MOST POPULAR

AI and ml

Google buys crashed airline Spirit’s data at auction, because AI

AI and Ml

Excel's Copilot function is headed for the Recycle Bin

DEVOPS

GitHub has Issues as repo downloads hit 50% error rate

security

Expired credit cards revived by researchers to make unauthorized payments

PUBLIC SECTOR

Capita handed vital role in next pandemic despite civil service pension failures

software

Microsoft MVP creates site to remind you of all the brands Redmond...

github code retry copilot outage autoscaling

Related Articles