At 3:07 AM on August 15 2025 a critical alert indicated the entire site was inaccessible, leading to a 2‑hour, 500 k‑user outage caused by an expired domain that entered serverHold status, and this postmortem details the detection, root‑cause analysis, emergency recovery steps, and long‑term remediation measures.
On the night of November 18 2025, Cloudflare suffered a three‑hour core failure that crippled roughly half of the internet, and this article details the timeline, global impact, root cause in a ClickHouse permission change, and the remediation steps taken to restore service.
Cloudflare suffered a massive multi‑hour outage that knocked offline popular sites and AI services, traced to a sudden traffic spike, a mis‑configured Rust‑based bot‑management module, and a database permission change that doubled a feature file size, overwhelming its routing software.
On the night of November 18 2025, Cloudflare suffered a worldwide outage that crippled services like ChatGPT, X, Spotify, and major gaming platforms, and a detailed post‑mortem reveals that a ClickHouse permission change caused an oversized bot‑management configuration file to crash edge nodes.
On November 18 2025 Cloudflare suffered a widespread network failure that crippled its CDN, DNS, and reverse‑proxy services, causing global sites like ChatGPT and X to return 502‑504 errors before engineers applied temporary mitigations and gradually restored service.
On October 19‑20 a massive AWS failure in the US‑East‑1 region crippled a large portion of the internet, exposing how a faulty internal monitoring tool, DynamoDB’s lack of cross‑region replication, and unchecked retry storms can cascade into a widespread outage, and offering concrete operational lessons for cloud teams.
In late October 2025, a DNS failure in AWS’s DynamoDB service triggered a cascade of outages across EC2, load balancers, and Lambda, causing a 14‑hour global disruption that impacted over 3,500 applications, while a simultaneous Taobao overload highlighted the challenges of scaling during traffic spikes.
GitHub’s recent outage, caused by an unexpected configuration change, prevented unauthenticated users in China from accessing the site from April 12 to 13 UTC, prompting widespread discussion and urging developers to consider the risks of relying solely on foreign code‑hosting platforms.
On December 11, OpenAI suffered a severe outage across ChatGPT, its API, and Sora due to a misconfigured telemetry service that overloaded Kubernetes control planes worldwide, prompting a cascade of failures and a coordinated recovery effort.
The December 11, 2024 OpenAI outage, caused by a misconfigured monitoring service that overloaded the Kubernetes control plane, led to a four‑hour service disruption and was resolved through cluster scaling, API blocking, and resource expansion, highlighting critical infrastructure risks for large‑scale cloud‑native operations.
On August 19, NetEase Cloud Music suffered a severe infrastructure‑related outage that prevented user login, playlist loading, and song search, prompting a two‑hour recovery effort, a brief free‑membership compensation, and highlighting the critical role of proper change management, gray releases, disaster recovery, and cross‑functional coordination in large‑scale services.
On August 19 the NetEase Cloud Music service suffered a major outage that was traced to a complex migration of its Hangzhou data center to Guizhou, highlighting large‑scale operational risks, technical debt, and strict continuity constraints for high‑traffic internet platforms.
On August 14, 2024, GitHub experienced a massive site-wide outage caused by a database infrastructure configuration change that disrupted traffic routing, leading to loss of database connections and affecting core services such as Pull Requests, Pages, Copilot, and the API, with full restoration confirmed later that evening.
A detailed account of GitHub’s recent worldwide outage reveals that a rollback of database infrastructure changes caused widespread service failures across GitHub.com, Pages, Copilot, and the API, highlighting the challenges of stateful database reliability in large platforms.
A collection of firsthand accounts reveals how seemingly harmless actions—changing system time, mistyping a script name, accidental deletions, and reckless debugging—triggered large‑scale service disruptions, forced emergency rollbacks, and costly penalties, highlighting the high stakes of operational negligence.
In November 2023, Alibaba Cloud suffered a massive outage that disrupted client, web, and mobile access for nearly eight hours, affecting a wide range of Alibaba products, over 300,000 enterprise customers, and numerous global data centers, highlighting the fragility of large-scale cloud services.
On November 12, Alibaba Cloud suffered a global-scale failure that disrupted dozens of Alibaba‑affiliated apps—including Taobao, DingTalk, and Aliyun Drive—affecting services from e‑commerce to campus laundry, prompting widespread frustration and sparking discussions about cloud reliability and potential migration to other providers.
A prolonged Azure outage in Sydney caused by a sudden power drop that disabled cooling systems, compounded by insufficient on‑site staff, led to service disruptions for over 24 hours and highlighted critical operational lessons for cloud data‑center management.
On December 18, 2023, Alibaba Cloud's Hong Kong Region experienced a severe cooling‑system failure that caused a 14‑hour outage of ECS, OSS, EBS, RDS and other services, prompting extensive emergency procedures, service impact analysis, and a detailed post‑mortem with improvement actions.
On December 18, 2022, Alibaba Cloud's Hong Kong Region Zone C suffered a massive service interruption—the longest in its operational history—prompting a detailed incident response, extensive service impact across compute, storage, and networking, and a thorough analysis that led to concrete infrastructure and communication improvements.
Roblox’s detailed post‑mortem reveals that a three‑day outage was caused by a Consul streaming bug and a design flaw in BoltDB’s freelist, which together created CPU contention and latency spikes on its massive on‑premises infrastructure, leading the team to disable streaming, add a second data‑center, and redesign their architecture.
The article reviews ten significant 2021 internet outages—both domestic and international—analyzing their root causes, from server room power failures to configuration bugs, and highlights the operational lessons engineers can learn to improve system resilience.
Facebook experienced a six‑hour worldwide outage that knocked out its main site and services like Instagram, WhatsApp, Messenger and Oculus, and engineers later traced the incident to a misconfigured backbone router that broke BGP routing and DNS resolution, sparking conspiracy rumors about data leaks.
In this article, a programmer recounts the recent Bilibili outage, analyzes its timeline, proposes technical root‑cause hypotheses such as CDN failure and service‑chain avalanche, shares insights from the platform’s high‑availability architecture, and outlines preventive techniques for building more resilient backend systems.
The July 13, 2020 GitHub outage, triggered by load‑balancer misconfiguration, a database connection error during partitioning, and a network‑config mistake, sparked worldwide developer panic, highlighted reliability concerns, and revealed challenges in scaling cloud infrastructure amid the pandemic.
A four‑hour Cloudflare outage was triggered by an unauthorized cable removal during a planned maintenance, compounded by unclear instructions and unlabeled wiring, highlighting the need for better cable management, clear operational procedures, and robust single‑point‑of‑failure mitigation.
On March 26 2020 Google’s core services—including Search, Gmail, YouTube and G Suite—experienced a worldwide outage caused by a router failure in an Atlanta data center, a third‑party software bug that disrupted traffic across multiple regions, prompting detailed analysis from Google, DownDetector, ThousandEyes and other observers.
In late March, users in China reported errors accessing GitHub Pages and the main site, prompting investigations that suggest a possible man‑in‑the‑middle attack affecting GitHub’s services, with evidence such as a suspicious certificate issued to a QQ email, network hijacking on port 443, and similar disruptions across major Chinese ISPs, while the issue resolved by mid‑afternoon.
The Redis website experienced an unexpected outage caused by an OOM error due to insufficient memory on a low‑cost DigitalOcean droplet, and the maintainer quickly fixed it by upgrading the instance and limiting allkeys‑lru usage, revealing surprising details about the site’s infrastructure.
When a Saturday early-morning outage left the company’s website and mobile app inaccessible for many users, the team traced the issue to an unpaid domain causing DNS resolution failures, detailed the investigation steps, temporary fixes, and lessons learned about DNS processes and operational readiness.
