Microsoft says a series of cascading failures sparked by network saturation at its Central US data centre were responsible for an almost eight-hour GitHub outage this week.

An incident report posted after the event said web/API error rates had reached 20% during its peak, with the issue persisting after capacity problems were resolved due to a “retry storm” at another data centre.

It said: “Archive and raw-content downloads reached approximately 50%. SAML/OIDC authentication, SCIM, and Team Sync were also affected, as well as Actions workflows in GHEC with Data Residency that depend on public workflow step definitions hosted on GitHub.com.”

Several scraping attacks on GitHub codeload endpoints during the incident also complicated recovery.

What happened?

Get the full story: Subscribe for free

Join peers managing over $100 billion in annual IT spend and subscribe to unlock full access to The Stack’s analysis and events.

Subscribe now

Already a member? Sign in