This article presents a comprehensive design for a layered storage and unified scheduling platform that archives MySQL historical data, reduces storage costs, ensures high‑performance queries, and enables efficient data analysis through tiered hot, warm, and cold storage using big‑data technologies.
This article explains how to extend a Flink CDC job that already syncs an entire MySQL database to Doris so that newly created tables are automatically created in Doris in real time, using the CdcTools utility, side‑output streams, and asynchronous I/O.
Using a 600k‑record IP range dataset, we built identical tables in ClickHouse and Doris, and a Redis skip‑list store, then ran three Flink‑Kafka streaming jobs to compare query latency across the three databases under varying traffic rates, revealing Redis as fastest, ClickHouse second, Doris slowest.
This article explains how to use Flink CDC together with the CdcTools utility to automatically capture newly created MySQL tables and synchronize both their schema and data to a Doris database in real time, covering the required code, side‑output handling, async execution, and a special delete‑sign field.
Facing a ClickHouse cluster shutdown, the author explores three migration methods—using Doris’s ClickHouse catalog, exporting to files with Broker/Stream Load, and Spark—to transfer ~10 billion rows to Doris, evaluating each for simplicity, bugs, and performance, and sharing detailed steps, code snippets, and benchmark results.
When a ClickHouse cluster needed to be decommissioned, the author evaluated three migration approaches—using Doris' ClickHouse catalog, exporting to files with Broker/Stream Load, and leveraging Spark—to move roughly 100 billion rows to Doris, comparing their complexity, reliability, and performance.
This article presents a systematic benchmark comparing ClickHouse, Doris, and Redis for IP‑range dimension lookups using Flink‑Kafka pipelines, detailing test design, result table schema, query interfaces, and performance results across varying data rates, concluding that Redis offers the fastest and most stable query latency.
This article reviews how Paimon and Doris combine to solve unified storage, data visibility, and performance challenges in modern lakehouse architectures, detailing their complementary features, integration capabilities, and real‑world use cases from leading companies.
This article presents a complete Java‑based architecture for high‑throughput Doris stream loading, covering project structure, Maven dependencies, configuration properties, field‑mapping annotations, automatic mapper utilities, a robust parallel loader with retry and compression, plus performance tuning recommendations.
This guide walks through the complete architecture, Maven dependencies, configuration classes, annotation‑driven field mapping, utility mappers, a parallel Stream Load core, response handling, and performance tuning for integrating Apache Doris with a Spring Boot backend.
This article summarizes how the Doris‑Paimon lakehouse architecture leverages Doris' high‑performance OLAP engine to accelerate lake queries, provides a unified data analysis gateway, supports unified data integration, and enables open, layered data modeling for modern big‑data workloads.
This article reviews how major tech firms such as Alibaba, Tencent, ByteDance, and Kuaishou tackled lakehouse challenges—including architecture fragmentation, cost, scalability, and complex multimodal data—by adopting real‑time lakehouse solutions like Flink + Paimon, Iceberg + StarRocks, Hudi + LAS, and Doris + Alluxio, and outlines the resulting performance and cost gains.
This article presents a lightweight algorithm service solution that combines an offline pre‑computed tag knowledge base with a real‑time data warehouse using Flink, Doris, Hive SQL and Python to achieve short development cycles, agile iteration, low cost, and scalable deployment for classification and clustering tasks.
This article provides a comprehensive walkthrough of Doris's SQL execution pipeline, covering the query optimizer's parsing, rewriting, and plan generation, the scheduler's fragment distribution, and the executor's fragment processing, including code examples of expression rewrite rules, join strategies, and data flow between FE and BE nodes.
This article reviews the emerging lakehouse architecture, outlines its core frameworks such as Hudi, Iceberg, Paimon, Flink, and Doris, discusses their storage‑compute separation, read‑write optimizations, and highlights how companies of different sizes adopt these technologies based on cost, efficiency, and specific business scenarios.
This article explains how Apache Doris achieves high‑concurrency OLAP and point‑query performance through MPP architecture, columnar storage, partition‑bucket pruning, various indexes, materialized views, vectorized execution, runtime filters, short‑circuit planning, and prepared‑statement caching.
This article explains Doris's data partitioning, the four join methods (Broadcast, Shuffle, Bucket Shuffle, and Colocate), runtime filter mechanisms and types, join reorder strategies, and practical tuning recommendations to improve query performance in the Doris MPP database.
This article presents Kuaishou's experience of designing and implementing a Doris‑driven lakehouse integrated analytics system, covering the current OLAP landscape, challenges of data duplication and governance, the new architecture with caching and auto‑materialization, implementation details, performance impact, and future work.
This guide explains how to enable row store, configure short query path, and use PreparedStatement in Doris to reduce I/O and CPU overhead for high‑concurrency primary‑key point queries, including DDL examples, JDBC usage, row cache settings, performance tips, and verification methods.
This article compares Doris and ClickHouse across architecture, table creation, ecosystem integration, management tools, query performance, and join capabilities, offering practical guidance on how to choose the right database based on your specific data processing and operational requirements.
This article examines the trade‑offs of using Flink versus Doris/StarRocks for real‑time data pipelines, highlighting Flink's strengths and pain points, and proposes shifting computation to the OLAP layer with Doris to reduce development and operational costs while maintaining near‑real‑time performance.
This article presents a comprehensive overview of precise deduplication in Meituan's Doris database, detailing the underlying bitmap data structures, aggregation bottlenecks, and a series of optimizations—including memory management, fast union, orthogonal encoding, and vectorized engine integration—that together achieve significant performance gains in high‑cardinality scenarios.
This article presents a comprehensive overview of Zhihu's Bridge Platform internal marketing module, detailing its background, business logic, product components such as CDP, activity and delivery platforms, architectural layers, performance bottlenecks, optimization techniques—including distributed transactions, bitmap indexing, and vectorized query execution—and future directions toward marketing automation and intelligence.
After upgrading a Doris cluster from version 1.2.3 to 2.0.2, the author runs a series of SQL benchmarks—including PK lookups, top‑client queries, distinct counts on low‑ and high‑cardinality columns, minute‑level session analysis, and full‑table deduplication—to measure execution times, revealing mixed performance gains and regressions across the seven test scenarios.
This article presents a systematic performance comparison between Doris and ClickHouse, covering data ingestion speed, SQL syntax differences, hardware impact, and detailed query benchmarks across multiple scenarios, ultimately revealing that each system excels in different use cases.
The article explains why Doris needed a new optimizer, describes its architecture—including rule‑based and cost‑based stages, early data‑size reduction techniques, dynamic‑programming join‑reorder methods, and practical challenges such as statistics errors and runtime filters—while sharing performance results and a Q&A session.
This article explains Doris's various data import methods, focuses on the routine load approach for Kafka streams, describes how to handle mixed‑schema topics using the max_error_number parameter, and compares query performance of a 130 million‑row dataset against ClickHouse, highlighting each system's strengths and limitations.
This article details Huolala's big‑data architecture and the practical measures—ranging from background analysis and stability challenges to case studies, discovery mechanisms, capacity planning, high‑availability, and automation—that the company employs to guarantee Doris's reliability and performance across its rapidly growing logistics platform.
This article shares the practical architecture, technology selection, implementation details, and evolution of an e‑commerce platform’s offline and real‑time data warehouses, covering data modeling, processing pipelines, system components such as Hive, Spark, Flink, ClickHouse, Doris, and Hudi, and the lessons learned from multiple production deployments.
This article presents HuoLala's practical approach to guaranteeing the stability of the Doris OLAP engine within its large‑scale big data platform, covering background, challenges, case studies, capability building, process standards, and future planning.
This article demonstrates how to ingest billions of Kafka records into Doris using Routine Load, compares the import process and query performance with ClickHouse’s Kafka engine tables, discusses configuration challenges such as field‑count filtering, and presents practical workarounds and benchmark results.
The article reviews Doris 2.0's major enhancements—including point‑query concurrency, log‑analysis capabilities, cold‑hot data separation, lakehouse integration, and various performance and usability upgrades—explaining how these changes benefit OLAP workloads and simplify data‑engineering pipelines.
The article examines Doris‑based real‑time data warehousing, outlining why teams choose this approach, comparing its low‑threshold development and operational simplicity to Flink’s high‑cost streaming, and highlighting latency, scale limits, and the strict monitoring required for production use.
Using the TPC‑DS benchmark’s 99 queries on a 1 TB dataset, this study evaluates the performance of four open‑source OLAP engines—ClickHouse, Doris, Presto, and ByConity—across basic, join, aggregation, subquery, and window‑function scenarios, revealing ByConity’s superior speed and the limitations of ClickHouse.
This article provides a comprehensive overview of OLAP technology evolution, covering its origins, modern requirements for massive and real‑time data, detailed comparisons of major open‑source OLAP engines such as Druid, Elasticsearch, Kylin, Doris/StarRocks, and ClickHouse, core architectural and storage techniques, and emerging trends like federated queries, hybrid storage, and lakehouse integration.
This article details Huolala's journey from a MySQL‑centric analytics pipeline to a multi‑engine OLAP platform built on Doris, covering system architecture, data flow, stage‑wise evolution, engine selection, POC validation, performance tuning, stability measures, and future roadmap for self‑service analytics.
This technical presentation details NIO's evolution of OLAP solutions—from Druid and TiDB to Doris—explaining the selection criteria, Doris's advantages as a unified OLAP warehouse, its role in the CDP platform, practical deployment experiences, and lessons learned from real‑world usage.
This article presents a comprehensive overview of Zhihu's Data Management Platform (DMP), covering its business background, three core business modes, detailed architecture, offline and real‑time data pipelines, feature storage design, performance optimization techniques, and future iteration directions.
This presentation details Huolala's journey from its initial OLAP architecture to a multi‑engine platform, describing background, data‑flow layers, technical research, engine selection (Druid, ClickHouse, Doris), POC validation, performance tuning, stability measures, production rollout, problem analysis, and future roadmap.
This article explains Doris's fundamental concepts such as rows, columns, tablets, and partitions, provides guidelines for column definition, partitioning and bucketing strategies, and details table creation syntax and property settings for optimal big‑data storage and query performance.
This article explains Doris's compaction mechanism, covering its producer‑consumer architecture, tablet scoring, permission control, cumulative and base compaction processes, parameter tuning, monitoring metrics, and manual compaction commands to help optimize performance and resource usage.
This article provides a comprehensive overview of Doris's architecture—including its FE and BE components, metadata management, data organization, execution planning—and details its major features such as adaptive join aggregation, vectorized execution, materialized views, and Elasticsearch integration, supplemented with example DDL and query code.
This article explains Doris database enhancements including Rollup indexes, Broadcast/Shuffle/Colocation joins, dynamic partition management, bitmap aggregation, and materialized view creation, providing syntax examples and operational principles for each feature.
This article details JD Logistics' journey from fragmented, massive‑scale data to a unified, real‑time analytics platform, covering business needs, pain points, tool evaluation, a new Doris‑based architecture, table management, data import procedures, automation scripts, and future roadmap for data engineering.
JD.com’s customer service team leverages the open‑source MPP database Doris to power real‑time and offline OLAP dashboards, detailing data ingestion pipelines, full‑link monitoring, dual‑stream high‑availability design, dynamic partition management, multi‑layer caching strategies, and performance optimizations applied during the 2020 11.11 shopping festival.
This article shares practical experiences with Doris, covering materialized view concepts, typical use cases, index principles, performance optimizations, and real‑world scenarios such as order analysis, PV/UV aggregation, and detailed queries, while also providing operational tips and Q&A insights.
This article shares practical experiences with Doris, covering materialized view concepts, typical use cases, advantages, creation syntax, prefix index principles, performance‑boosting scenarios such as order analysis, PV/UV counting, detail queries, and operational tips for high‑throughput and low‑latency workloads.
The article details JD.com’s large‑scale OLAP strategy using ClickHouse as the primary engine and Doris as a secondary engine, covering application scenarios, component selection criteria, cluster deployment models, high‑availability architecture, fault‑handling procedures, performance tuning, and future cloud‑native plans.
Meituan Waimai’s data intelligence team outlines a universal real‑time data‑warehouse methodology that combines a production platform with an interactive analytics engine, detailing scenarios, technology choices, architectural designs, platformization, SLA management, and a practical Lambda‑style case study.
This article introduces a universal method for building a real-time data warehouse at Meituan Waimai, covering streaming technologies, architecture choices such as Lambda and Kappa, component design, feature production, SLA management, and practical OLAP solutions using Flink, Storm, and Doris.
This article details JD's OLAP implementation using ClickHouse as the primary engine and Doris as a secondary engine, covering business scenarios, selection criteria, multi‑tenant deployment, high‑availability architecture, encountered challenges, and future roadmap for cloud‑native, scalable analytics.
The article presents a real‑time quantile solution using the TDigest data structure, which clusters data into centroids and stores digests in Redis or Doris, pre‑computes quantiles for all dimension combinations, and provides a reusable API that delivers fast, accurate, low‑memory quantile statistics for diverse business scenarios.
This article presents a comprehensive case study of how Baidu’s senior R&D engineer Zhao Yuyang built a Doris‑based user‑segmentation system for mini‑programs, detailing the product’s private‑domain fine‑grained operation capabilities, the four technical challenges, the architecture and solutions—including global dictionaries, bitmap storage, partitioning, tag optimization, dynamic‑static query handling, and rapid user‑package generation—along with future roadmap plans.
This article describes how JD.com leverages the open‑source MPP analytical database Doris for real‑time and offline OLAP on customer‑service data, covering data ingestion pipelines, dual‑stream high‑availability design, dynamic partition management, multi‑level caching, monitoring with Prometheus‑Grafana, and performance optimizations applied during major sales events.
This article details JD Search's journey from a Storm‑based pipeline to a Flink‑driven architecture backed by Apache Doris, covering business requirements, technical challenges, design trade‑offs, performance optimizations for massive traffic spikes, and future plans for their real‑time OLAP data warehouse.
This article examines Meituan’s data warehouse evolution, detailing the limitations of MOLAP with Kylin, the adoption of Doris‑driven ROLAP using MPP technology, and the practical optimizations—such as join predicate pushdown, concurrent execution, colocate join, and bitmap aggregation—that improve real‑time analytics and reduce costs.
This article introduces Apache Doris, a massively parallel processing (MPP) distributed database, covering its background, core architecture, key technologies such as reliability, maintenance, MySQL compatibility, materialized views, and real‑world applications at Baidu and beyond.
