Erasure Coding Practice in HDFS at Didi: Principles, Implementation, and Lessons Learned

Didi migrated HDFS to Hadoop 3.2 and implemented erasure coding—using XOR and Reed‑Solomon RS(6,3) striping—to replace three‑replica storage for cold data, building back‑ported clients, automated conversion tools, and cross‑datacenter backup pipelines, while addressing operational bugs and noting performance trade‑offs.

Big DataDidiHDFS

0 likes · 11 min read

Erasure Coding Practice in HDFS at Didi: Principles, Implementation, and Lessons Learned

Art of Distributed System Architecture Design

May 22, 2015 · Industry Insights

How Facebook Cuts Power Use with Cold Storage: Inside Their Low‑Energy Data Center Design

This article examines Facebook's cold storage system, detailing how the company redesigned hardware and software to slash power consumption, improve reliability with Reed‑Solomon coding, mitigate bit‑rot, and balance loads while supporting massive photo archives in energy‑constrained data centers.

FacebookReed-Solomoncold storage

0 likes · 8 min read