2 views collected around this technical thread.
In this interview, Didi data governance lead Liu Chao discusses his career journey, the unique technical architecture of Didi’s big‑data governance system, cost‑driven pricing models, metadata management, lineage extraction, automation practices, and offers practical advice for enterprises seeking effective data governance.
Didi’s Elasticsearch platform, built on ES 7.6 and deployed on physical machines with containerized gateway and control layers, provides a multi‑tenant, high‑performance search service—featuring a user console, operational controls, ZGC‑based latency reductions, cost‑saving compression, custom security, real‑time cross‑datacenter replication, and a roadmap toward ES 8.13.
This article outlines Didi’s best‑practice approach to constructing a global ride‑hailing data metric system, covering business scenarios, metric categories, pain points such as definition and technical challenges, and a comprehensive solution involving organizational structure, processes, model design, tooling, timezone handling, and governance.
The article reviews Didi's late‑November 2023 service disruption, detailing the timeline of failures, official apologies, and expert analyses of six possible technical causes—including software bugs, server issues, third‑party failures, DDoS, other attacks, and ransomware—while highlighting the role of a Kubernetes upgrade and cost‑cutting pressures.
To achieve disaster-recovery and cross-data-center resilience, Didi progressed from a simple proxy double-write scheme to a sophisticated MQ-free multi-active Redis design that uses a dedicated syncer, shard-based loop prevention, op-id replay protection, conflict detection, and incremental AOF durability, ensuring low latency, no data loss, and consistent availability.
The article explains observability as the ability to infer any system state from external data, contrasts it with traditional monitoring, outlines challenges of high‑dimensional, high‑cardinality data and storage costs, and describes Didi’s hybrid MTL architecture that separates low‑ and high‑cardinality logs and metrics while linking them via TraceIDs to provide detailed, cost‑effective insight and streamlined debugging.
At Didi, observability and stability engineering combine automated, AI‑driven alarm generation, distributed tracing, and ChatOps‑based fault handling to manage micro‑service complexity, massive traffic spikes, and cross‑region operations, emphasizing systematic investment, AIOps evolution, and a recruitment call for backend and test engineers.
The article details how Didi’s ride‑hailing team built a full‑link load‑testing simulation‑degree measurement system that quantifies test coverage across five dimensions—interface, scenario, category, link, and module—using normalized metrics, traffic prediction, and scoring formulas to identify gaps, improve stability, and guide future capacity‑planning enhancements.
Didi hardened its massive Elasticsearch deployment—spanning 66 clusters and thousands of nodes—by adding a custom security plugin that authenticates requests at the cluster level, implementing a one‑click toggle and staged rolling upgrades, ultimately enabling authentication across all clusters and dramatically reducing data‑leak risk.
Didi’s ride‑hailing backend evolved from a monolithic service to a DDD‑split architecture and the DuKang framework, then standardized components, rendering gateways, function access, and logging to solve multi‑terminal inconsistency, complex rendering logic, cross‑process integration, and data visibility, creating a unified cross‑end rendering platform supporting over twenty pages.
At a June 2023 forum in Beijing, Didi partnered with leading Chinese universities to showcase award‑winning projects that applied deep‑learning positioning, long‑term pricing models, high‑precision two‑wheel navigation, in‑vehicle voice separation, and data‑driven autonomous‑driving, illustrating how academia‑industry collaboration turns cutting‑edge research into commercial value.
Didi balances ride‑hailing supply and demand by forecasting regional needs with time‑series and deep‑learning models, then optimally repositioning drivers through integer programming and refining policies via imitation and offline reinforcement learning, ultimately enhancing passenger experience and platform efficiency.
The article introduces Didi’s large‑scale ride‑hailing marketplace, explaining how mechanism design, decision intelligence, operations research, reinforcement learning, and causal inference are combined to create fair, efficient, data‑driven matching, routing, and incentive systems that tackle massive two‑sided market challenges.
This article chronicles Cheng Wei’s path from a modest upbringing and early setbacks through his tenure at Alibaba to founding Didi, detailing the company’s early struggles, rapid growth, fierce competition with Uber, strategic mergers, and its eventual dominance in China’s ride‑hailing industry.
DiDi applied the epsilon‑greedy bandit algorithm integrated with its CMS to optimize ad placement across 600 slots, using quality scores, traffic sampling, and a drag‑and‑drop UI, which boosted CTR from 1.35% to 13.43% and unique visitors by 686%, demonstrating data‑driven growth beyond simple A/B testing.
Didi migrated HDFS to Hadoop 3.2 and implemented erasure coding—using XOR and Reed‑Solomon RS(6,3) striping—to replace three‑replica storage for cold data, building back‑ported clients, automated conversion tools, and cross‑datacenter backup pipelines, while addressing operational bugs and noting performance trade‑offs.
The Youzan Big Data Technology Salon brought together Youzan, NetEase and Didi to share practical approaches for cutting data‑infrastructure costs, building an Apache Iceberg‑based data lake, scaling Flink real‑time workloads, and creating a data‑driven growth platform that leverages tracking, A/B testing and analytics.
This article presents a comprehensive catalog of more than 40 open‑source projects released by Didi, covering AI runtimes, speech and NLP engines, big‑data loaders, middleware, mobile frameworks, frontend UI libraries and various operational tools, each with a brief description and a GitHub link.
DiDi’s route engine, handling over 40 billion daily requests, replaces static graph algorithms with a deep‑reinforcement‑learning system that first learns intersection decisions via behavior‑cloning LSTM models and then refines them through self‑play Q‑learning, using beam‑search decoding to produce globally optimal, low‑deviation routes for ride‑hailing.
This article explains Didi's comprehensive methodology for constructing, operating, and productizing a metric system, covering metric definitions, lifecycle stages, OSM and AARRR modeling, scenario‑based design, management challenges, metadata governance, and tool support for data‑driven decision making.