The Challenges of Building Continuously Available Systems: Entropy, Murphy's Law, and the 'Divine Doctor Paradox'

This article explores the fundamental challenges of building continuously available systems in the context of 2023's internet service outages. The author examines two key principles that create inherent limitations: the Second Law of Thermodynamics (entropy increase) and Murphy's Law (anything that can go wrong will go wrong).

The article discusses how entropy manifests in software systems through rushed development cycles, accumulating technical debt, and constant introduction of new technologies. Murphy's Law is illustrated through various failure scenarios including hardware failures, software bugs, and human errors.

A central concept introduced is the "Divine Doctor Paradox" (神医悖论), derived from a story about the physician Bian Que. This paradox highlights the difficulty of proving the value of availability work - when systems work well, the work goes unnoticed; when problems occur, the team is blamed regardless of preventive efforts.

The author describes a cyclical pattern where teams face pressure to deliver features quickly, leading to system degradation, followed by crisis-driven availability projects, then gradual decline again. Breaking this cycle requires cultural change rather than just technical solutions, including sustained investment in availability work, avoiding purely cost-driven decisions, and regular system maintenance.

The article concludes that building continuously available systems is fundamentally a cultural challenge requiring organizational commitment rather than just technical expertise.