Microsoft: DDoS defense error amplified attack on Azure, leading to outage
A DDoS attack that started on Tuesday has made a number of Microsoft Azure and Microsoft 365 services temporarily inaccessible, the company has confirmed.
Microsoft’s mitigation statement on the Azure status history page
Microsoft Azure, 365 outage triggered by DDoS
“Between approximately at 11:45 UTC and 19:43 UTC on 30 July 2024, a subset of customers may have experienced issues connecting to a subset of Microsoft services globally. Impacted services included Azure App Services, Application Insights, Azure IoT Central, Azure Log Search Alerts, Azure Policy, as well as the Azure portal itself and a subset of Microsoft 365 and Microsoft Purview services,” Microsoft said.
“An unexpected usage spike resulted in Azure Front Door (AFD) and Azure Content Delivery Network (CDN) components performing below acceptable thresholds, leading to intermittent errors, timeout, and latency spikes.”
Users were also unable to access some Microsoft 365 services – Microsoft 365 admin center, Intune, Entra and Power Platform – but SharePoint Online, OneDrive for Business, Microsoft Teams and Exchange Online remained accessible and responsive.
Microsoft said that the outage was triggered by a Distributed Denial-of-Service (DDoS) attack, and that its effect was amplified by an error in the implementation of Azure DDoS defenses.
The company has promised to publish a preliminary post incident review in the coming days, to detail what happened and how they responded.
This is not the first time this happened
Microsoft’s services have been similarly hit in early June 2023. And, as security researcher Kevin Beaumont noted, Microsoft tried to keep the incident from getting a lot of public attention.
That attack was claimed by Anonymous Sudan hacktivists – or, as Microsoft tracks them, Storm-1359. The group used a collection of botnets and tools to “launch DDoS attacks from multiple cloud services and open proxy infrastructures,” the company said after the attack.
This latest attack, according to Beaumont, involves the Meris botnet, which mostly consists of compromised routers and switches.
UPDATE (August 1, 2024, 12:20 p.m. ET):
Microsoft has shared more details about the cause of the outage.
In short, multiple Azure Front Door and CDN sites were hit by a volumetric distributed TCP SYN flood DDoS attack, and the Azure Network DDoS protection service kicked into gear and mitigated it.
But when it came time to resume default traffic routing to the Azure Front Door service, the network routes could not be updated (due to a local power outage) within a specific site in Europe, and the traffic inside Europe continued to be forwarded to AFD through the DDoS protection services, instead of returning directly to AFD.
“This event in isolation would not have caused any impact. However, an unrelated latent network configuration issue caused traffic from outside Europe to be routed to the DDoS protection system within Europe. This led to localized congestion, which caused customers to experience high latency and connectivity failures across multiple regions,” the company found.