Official Didi technology account
The article explains Flink’s metrics architecture—core concepts, reporter interfaces, built‑in and custom metric types, elastic plugin design, and scheduled reporting—illustrated with a consumption‑latency example, and shows how Didi uses these metrics for real‑time UI curves, alerts, and intelligent task diagnosis.
Didi replaced its Elasticsearch‑based log platform with ClickHouse, redesigning architecture into isolated Log and Trace clusters, using hourly‑partitioned MergeTree tables and aggregating views to handle petabyte‑scale writes, diverse low‑latency queries, and high QPS, achieving over 400 nodes, 40 GB/s throughput, 30 % cost savings and four‑fold query latency reduction.
Didi leverages StarRocks materialized views with a global dictionary and bitmap aggregation to perform real‑time, high‑cardinality precise deduplication, automatically rewriting queries and refreshing views, cutting query latency by ~80%, reducing resource use ~95%, and boosting concurrent QPS up to 100‑fold, while planning further automation and bitmap optimizations.
The article describes MPX’s partial runtime capability‑enhancement design, which adds modern web‑framework features such as props, default slots and event forwarding to Didi’s recompilation‑based mini‑program framework by enabling on‑demand runtime rendering for selected components, generating minimal base templates, respecting the native dual‑thread event model, and exploring server‑component‑inspired mixed rendering while keeping performance overhead low.
Didi’s new Ray‑native XGBoost training platform replaces the fault‑prone Spark solution with a fully Pythonic, fault‑tolerant architecture that leverages Ray’s autoscaling and gang‑scheduling, delivering 2–6× speedups, reduced failure rates, efficient sparse‑vector handling, scalable hyper‑parameter search, and improved resource utilization for large‑scale machine‑learning workloads.
XIAOJUSURVEY is an open-source, lightweight yet secure enterprise-grade survey platform derived from Didi’s internal system, offering end-to-end questionnaire creation, distribution, collection, analysis and reporting, with customizable question types, schema-based architecture, pluggable security, and future extensions for collaboration and market-place features.
Mpx 2.9 adds built‑in atomic‑class (utility‑first) CSS, server‑side rendering for web output, and a suite of bundle‑size optimizations—including deterministic IDs, template compression, empty‑module removal, and render‑function pruning—while splitting atomic CSS into sub‑packages for mini‑programs and outlining a roadmap for further compile‑time reductions, Vite support, and cross‑platform extensions.
Apache Pulsar, a cloud‑native distributed messaging platform, solves Didi Big Data’s DKafka bottlenecks by separating compute and storage, using sequential log writes, heterogeneous disks, multi‑level caching, bundle‑based load balancing and automatic scaling, dramatically improving stability while introducing richer monitoring complexity.
The Java service crashed when the Linux OOM killer terminated its process after a developer opened a 37 GB log file with Vim, which loaded the entire file into the container’s 8 GB memory, triggering a port alarm and prompting investigation and recommendations to use streaming tools instead of Vim.
An in‑depth, timeline‑driven investigation of a production Redis memory leak revealed that the custom 3.2.8 build’s getKeysInSlot function failed to free a temporary key‑array after traversing the radix‑tree, causing hundreds of megabytes of leaked SDS strings, which was fixed by adding a single free call and highlighted the need for functional code reviews and early leak detection.
