High Availability Architecture

Oct 20, 2016 · Big Data

Understanding HDFS EditLog Format and Quorum Journal Manager Recovery Process

This article explains the HDFS EditLog file structure, the design of the Quorum Journal Manager for high‑availability, the write‑path optimizations such as batch flushing and double‑buffering, and the detailed Multi‑Paxos based recovery algorithm including isolation, segment selection, prepare and accept phases, and handling journal node failures.