GitHub blames 8-hour outage on autoscaling fail

GitHub blames 8-hour outage on autoscaling fail and VS Code retry storm

Jump to main content

REG AD

SAAS

GitHub blames 8-hour outage on autoscaling fail and VS Code retry storm

Load balancers buckled after a monitoring blind spot allowed traffic to spiral

Richard Speed

Richard Speed

MICROSOFT ECOSYSTEM REPORTER

Published wed 19 Aug 2026 // 12:40 UTC

GitHub has published its account of this week's nearly eight-hour outage, tracing the developer pain to saturated load balancers, a faulty autoscaling policy, and a "latent retry bug in Visual Studio Code." According to GitHub, problems began at 1328 UTC on August 17 and weren't fully resolved until 2115 UTC – a 7-hour, 47-minute incident that produced elevated errors across Issues, Pull Requests, APIs, Actions, and Copilot. The immediate cause was network saturation on load balancers in the company's Central US facility, triggered when an Istio sidecar reached its concurrency limit.

REG AD

Surely autoscaling would add capacity as those limits were reached? Alas, no. A misconfigured policy monitored the host service but not the sidecar's concurrency limit, allowing a cascading failure to develop. "The problem," according to GitHub, "was worsened by optimistic retry logic which overloaded internal load balancers."

REG AD

Engineers mitigated the problem by temporarily reducing gateway retries through a code change and configuring the load balancers to reject inbound Copilot Token Service requests with HTTP 403 responses. Ah yes, Copilot. GitHub explained: "Delayed replies to a single internal endpoint triggered a latent retry bug in VS Code that amplified traffic by approximately 10x and caused delayed recovery for the Copilot Token Service." Most services recovered by 1636 UTC and Actions by 1803 UTC, but the Copilot Token Service took until 2102 UTC. "Complicating factors that impeded recovery included a number of scraping attacks on codeload endpoints," GitHub added. The Microsoft tentacle says it will correct the autoscaling policies, review retry limits, audit Istio concurrency settings, and address the VS Code behavior "that amplified Copilot token traffic." This latest incident could be the tipping point that sends some developers scurrying for an alternative. CloudBees CEO Moritz Plassnig noted in a LinkedIn post that "Cursor, OpenAI and several smaller startups are already building competitive solutions."

MORE CONTEXT

Holy git! Microsoft code-sharing site suffers downtime, despite move to Azure

GitHub has Issues as repo downloads hit 50% error rate

Latest GitHub outage squeezes Actions, Pages to death

GitHub Actions outage told devs 'your account is suspended'

"GitHub won't be the default solution going forward and we are looking at a much more bifurcated ecosystem (which is good and bad)." The findings will raise eyebrows among engineers. A misconfiguration and retry storm degraded a critical piece of infrastructure on which many organizations depend, leaving developers unable to work normally for hours.

REG AD

GitHub's reliability problems stretch back well beyond this week, as the company itself acknowledges. Developers have choices, and the pain/gain equation is not looking too favorable for the source shack. As Plassnig noted, alternatives keep cropping up – sometimes at the most awkward moment. As GitHub staggered, SpaceX-owned Cursor announced an early beta of Origin Code Hosting. ®

microsoft github copilot saas visual studio code

REG AD

PATCHES

Homeland security cybercops say patch TrueConf (Russia's Zoom) if you're using it

Ukrainian hacktivists exploiting the bugs, but TrueConf's reach stretches well beyond home turf

OFFBEAT

Musk fumbles the timetable for first Starship catch

Billionaire walks back earnings call optimism as orbital test approaches

devops

Platform Engineering 2.0: your platform was built for a different era. AI just exposed it

PARTNER CONTENT: Platform engineering won the argument. Now it has to grow up fast and evolve for the AI era.

cybercrime

SickKids children’s hospital bandages up careers website after intruder breaks in

Toronto org says it wasn’t the only one to be affected by the third-party software vulnerability

COLUMNISTS

Be a hater all you want, AI's here to stay

The good news? One of the worst bits, tech giants controlling it all, might soon be over

SECURITY

Hackers poison popular Rust crates to steal developers' credentials

Malicious updates turned routine builds into a delivery system for infostealer malware

GitHub blames 8-hour outage on autoscaling fail

Related Articles

Incident with Github.com

(no title)

Scientists reverse brain aging, with a nasal spray

NHS apologises and admits Palantir have access to identifiable patient data

Cursor Origin