What Triggered Facebook’s 7‑Hour Global Outage? Inside the DNS and Backbone Failure
On October 4, a faulty maintenance command disabled Facebook’s DNS servers and severed its global backbone network, causing a seven‑hour outage that affected billions of users, highlighted architectural flaws, and underscored the need for redundant DNS and better operational safeguards.
On October 4 at around 11:39 AM EST, Facebook, Instagram, and WhatsApp suffered a massive outage that lasted nearly seven hours, marking the longest downtime since 2008.
A routine maintenance operation issued a bad command that rendered Facebook’s DNS servers unusable, cutting off the backbone network that connects its worldwide data centers.
The backbone is a global fiber‑optic network linking data centers; user requests are first routed to the nearest facility and then travel through this backbone to larger centers for processing.
During normal maintenance, engineers sometimes take portions of the backbone offline for repairs or upgrades. In this case, a command intended to assess global backbone capacity inadvertently disconnected all connections, and the system could only issue warnings, not prevent execution.
Although the DNS servers remained operational, they could not be reached, causing BGP announcements to be withdrawn and preventing the internet from resolving Facebook.com, effectively removing the map that tells computers where to find Facebook’s services.
Repair was delayed because engineers could not access internal tools via the normal network and DNS failures blocked standard troubleshooting; physical access to data centers was required, which is time‑consuming due to strict security policies.
Experts noted the lack of redundant DNS as a key factor that prolonged the outage, suggesting that dual‑DNS setups (e.g., using both Dyn and UltraDNS) could mitigate such incidents.
The outage had significant economic impact, with an estimated $160 million loss per hour, a 6% drop in Facebook’s stock, and a $6 billion reduction in Mark Zuckerberg’s personal wealth.
Facebook denied any data breach despite rumors of user data being sold on hacker forums.
After services began to recover, concerns remained about a potential new crash due to traffic spikes, and the incident highlighted architectural weaknesses and renewed antitrust scrutiny.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
MaGe Linux Operations
Founded in 2009, MaGe Education is a top Chinese high‑end IT training brand. Its graduates earn 12K+ RMB salaries, and the school has trained tens of thousands of students. It offers high‑pay courses in Linux cloud operations, Python full‑stack, automation, data analysis, AI, and Go high‑concurrency architecture. Thanks to quality courses and a solid reputation, it has talent partnerships with numerous internet firms.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
