This article presents the technical solution and key processes of JD's mini‑program custom data analysis service, covering business background, ClickHouse‑based storage design, real‑time processing pipelines, dynamic rule parsing, table architecture, monitoring mechanisms, and future outlook for large‑scale data analytics.
This article explains the design of a high‑performance event analysis platform that ingests billions of daily logs, supports event, funnel, and retention queries, and leverages ClickHouse for storage, efficient writes, and fast analytical queries across massive datasets.
This article explains what an APM system is, compares logs, traces and metrics, reviews popular tools, and then details the design and implementation of the in‑house Dog APM platform—including client data models, Kafka pipelines, processing pipelines, storage in ClickHouse/Cassandra, and UI visualizations.
This article presents the Future Cloud Business Monitoring real-time dashboard solution, detailing its technical architecture, key challenges in massive log processing, storage choices, product considerations, experience sharing, future plans, and concrete case studies such as live classroom monitoring.
This article compares Elasticsearch and ClickHouse for log analytics, detailing their architectures, presenting a Docker‑Compose test setup that streams synthetic syslog data to both stacks, executing equivalent queries, measuring response times, and concluding that ClickHouse generally delivers faster performance, especially for aggregations.
This article explains how ByteDance leverages ClickHouse for ad audience estimation, profiling, and analytics, detailing the challenges of massive user‑level set operations, the evolution from a simple tag‑uid table to Bitmap64 with RoaringBitmap, and the extensive engineering optimizations that cut query latency, storage, and CPU usage dramatically.
This article compares Elasticsearch and ClickHouse for log analytics, detailing their architectures, node roles, query capabilities, and performance through Docker‑Compose stacks and synthetic syslog data, concluding that ClickHouse generally outperforms Elasticsearch in speed and aggregation efficiency.
This article compares Elasticsearch and ClickHouse as log analytics solutions, detailing their architectures, deployment configurations, query capabilities, and performance benchmarks across various query types, and demonstrates that ClickHouse generally outperforms Elasticsearch in speed and aggregation efficiency.
ClickHouse is an MPP column‑oriented DBMS that combines full DBMS functionality, advanced columnar storage with high compression, SIMD‑based vectorized execution, a rich relational SQL interface, diverse table engines, multi‑master clustering, and flexible sharding and distributed query capabilities, making it exceptionally fast for analytical workloads.
This article compares ClickHouse and Elasticsearch for API log storage, detailing development activity, schema handling, query performance, statistical functions, MySQL integration, new features, and practical drawbacks, while providing concrete SQL examples and migration tips.
This article details the design and multi‑stage optimization of an A/B testing metric system, describing its product architecture, Spark‑based computation engine, ClickHouse OLAP layer, cumulative calculation improvements, and batch processing techniques that reduced processing time from hours to a few minutes for hundreds of experiments and metrics.
This article provides a comprehensive overview of ClickHouse, covering its columnar OLAP architecture, key features such as data compression, vectorized processing, distributed query handling, and SQL support, followed by detailed step‑by‑step instructions for deploying a multi‑node ClickHouse cluster with MergeTree and ReplicatedMergeTree engines, configuration files, and Java MyBatis integration.
This article details Youzan's end‑to‑end experience building a real‑time data warehouse, covering background, application scenarios, layered architecture, streaming ETL techniques, data validation, recovery processes, and three production projects that illustrate the solution in action.
This article provides a side‑by‑side technical comparison of HBase, Kudu, and ClickHouse, covering their installation dependencies, architectural designs, read/write workflows, query capabilities, real‑world use cases at Didi, NetEase, and Ctrip, and practical operational tips.
This article details Xiaohongshu’s multi‑stage evolution of its OLAP infrastructure—from Redshift to Presto, ClickHouse, and finally DorisDB—describing the data pipeline, tool comparisons, advertising use‑case implementation, and the resulting performance and operational benefits.
ClickHouse is a high‑performance MPP column‑store DBMS that combines complete DBMS functions, column‑oriented storage with aggressive compression, SIMD‑based vectorized execution, flexible table engines, multithreading, distributed processing, a multi‑master architecture, and SQL compatibility to deliver fast online analytical queries on massive data sets.
This article details JD's OLAP implementation using ClickHouse as the primary engine and Doris as a secondary engine, covering business scenarios, selection criteria, multi‑tenant deployment, high‑availability architecture, encountered challenges, and future roadmap for cloud‑native, scalable analytics.
This article details Vipshop's OLAP evolution, describing how Presto, Kylin, and ClickHouse are integrated, the deployment architecture with HAproxy and chproxy, containerization on Kubernetes, and the Flink‑ClickHouse pipeline that enables self‑service analysis of hundred‑billion‑row datasets while addressing performance challenges and future roadmap.
ClickHouse, born from Yandex’s Metrica and now a top‑50 open‑source analytics engine, achieves exceptional speed through a vectorized compute engine, column‑store architecture, and an active community, powering real‑time workloads at companies like Tencent Music, Sina, Bilibili, and Suning while introducing features such as column merging, projections, and storage‑compute separation for future scalability.
The article describes a real‑time user behavior analysis platform built on a ClickHouse cluster, detailing its architecture, Hive‑to‑ClickHouse data ingestion with user‑ID routing, table designs for behavior and group data, and five analytical methods—event, funnel, path, retention, and attribution—leveraging shard‑level parallelism and custom functions for high efficiency.
This article details BTC.com’s journey from a legacy batch‑oriented analytics stack to a modern real‑time OLAP architecture using Flink, ClickHouse, Kafka, and Kubernetes, highlighting the business drivers, technical choices, architectural evolution, optimizations, and future directions.
This article explains how to break the barrier between heterogeneous databases by synchronizing MySQL to ClickHouse using QingCloud MySQL Plus and the MaterializeMySQL engine, covering MySQL replication history, architecture details, HTAP scenarios, and providing practical code examples for setup and operation.
This article details BTC.com’s journey in building a real‑time OLAP platform for blockchain data, covering business background, challenges, architectural evolution, technology choices such as Flink and ClickHouse, optimization techniques, monitoring, and future directions.
This article shares a step‑by‑step practice of constructing an observability‑driven monitoring and alerting platform, covering data source selection, collection pipelines, storage choices, metric systems, visualization tools, and alert visualisation, with concrete code snippets and real‑world trade‑offs.
To meet iQIYI video production’s thousands‑QPS, petabyte‑scale, frequently‑updated data and large‑table join requirements, the team built a Spark‑plus‑ClickHouse real‑time warehouse that streams Kafka changes, joins HBase dimensions, and writes to ClickHouse, reducing reporting development time from days to hours while supporting both offline and real‑time analytics.
This article presents a comprehensive design of a real‑time security monitoring and alert platform, detailing challenges in high‑concurrency risk control, an architecture that replaces OLAP polling with scalable compute services, event‑time processing, dynamic thresholding using fbprophet, and practical optimizations with Redis and ClickHouse.
The article explains how to perform efficient user retention, funnel, and session analysis on large ClickHouse datasets by replacing costly multi‑table joins with bitmap compression, the built‑in retention function, windowFunnel, and high‑order array functions, providing practical SQL examples and performance insights.
This article introduces ClickHouse, explains its column‑oriented architecture versus row‑oriented databases, details a MySQL‑to‑ClickHouse migration, shows a performance benchmark that reduces query time from minutes to seconds, and shares practical deployment tips and common pitfalls.
The article details the design and engineering of a high‑performance path analysis model for internet data operations, explaining session handling, Sankey visualizations, adjacency‑table storage, multi‑granular session partitioning, Spark‑to‑ClickHouse pipelines, and optimizations that enable billion‑scale user‑path queries in about one second.
This article details ByteDance's evolution from offline ClickHouse ingestion to a robust real‑time analytics pipeline, covering external transaction handling, risks of direct INSERTs, recommendation and ad‑delivery use cases, Kafka Engine design, multi‑threaded consumption, fault‑tolerance improvements, platform tooling, and future roadmap.
This article explains in detail how ClickHouse processes a user‑submitted SQL query, covering the server’s request handling, parsing, query rewrite, optimization, interpreter execution, and result transmission, while illustrating key source code snippets and architectural components.
This article provides a comprehensive overview of ClickHouse's table engines, explaining the purpose and characteristics of each engine series—including Log, Integration, Special, and the various MergeTree families—while offering detailed SQL examples and guidance on when to use each engine for different data workloads.
This article explains how Suning's big‑data team incorporated ClickHouse into their end‑to‑end monitoring ecosystem, detailing the architecture, trace‑ID propagation, slow‑query tracking, MergeTree health checks, replica delay analysis, and the role of Chproxy in delivering comprehensive observability for high‑performance OLAP workloads.
The article compares the popular audio‑social platform Clubhouse with the high‑performance analytical database ClickHouse, explaining why their names cause confusion, outlining Clubhouse’s product features and growth drivers, and detailing ClickHouse’s column‑store architecture, scalability, and the supporting RTC, CDN, AI, and security technologies.
This article describes how JD's Energy Management Platform leverages ClickHouse as a high‑performance, MPP‑based OLAP engine to provide real‑time, multi‑dimensional analytics on IoT energy data, covering business background, technology selection, system architecture, data ingestion, storage, replication, and a generic query interface with code examples.
This article explains how JD’s Energy Management Platform uses ClickHouse as an MPP‑based OLAP engine to ingest, store, and provide multi‑dimensional real‑time analytics on energy consumption data, covering architecture decisions, data pipelines, replication, sharding, and a generic query interface.
The article explains ClickHouse’s high‑performance columnar OLAP design, its vectorized execution, sparse primary‑key indexes and MergeTree engines, contrasts it with ROLAP/MOLAP approaches, and details Youzan’s large‑scale deployment—including dual‑replica clusters, ingestion pipelines, routing architecture, current challenges, and future container‑based expansion plans.
ClickHouse is an open‑source column‑oriented OLAP database developed by Yandex that offers real‑time queries, high performance compared to many traditional databases, a rich set of features, and a modular architecture including columns, data types, blocks, parsers, functions, and various table engines.
This article provides a comprehensive introduction to ClickHouse, an open‑source column‑oriented OLAP database, covering its high‑performance benchmarks, core architectural components, query processing model, deployment patterns, Java client usage, and real‑world implementations at large enterprises.
This article introduces ClickHouse as a column‑oriented OLAP database, explains its architectural differences from row‑based systems, details a real‑world migration from MySQL with code examples, showcases performance gains, and shares practical tips and pitfalls for production use.
eBay’s ad data platform, originally built on a custom SQL engine and later migrated to Druid, was re‑engineered to use ClickHouse, highlighting challenges such as massive data volume, atomic offline replacements, schema design, compression, and operational simplifications, and demonstrating performance and scalability gains for advertisers.
This article recounts the author’s journey building a real‑time data warehouse using Flink, Kafka, Redis, and ClickHouse, describing the initial batch‑oriented setup, successive architectural evolutions, challenges with wide tables and dimension data, and the final OLAP‑centric solution with secondary caching.
This article explains how JD’s JUST platform leverages the open‑source columnar database ClickHouse to store, query and analyze massive time‑series datasets, covering data modeling, lifecycle management, cluster architecture, write and query processes, scaling strategies and future enhancements.
This article describes how 58.com introduced ClickHouse to handle massive daily user‑behavior logs, detailing its features, multi‑layer architecture, configuration management, monitoring, performance optimizations, and operational automation to build a high‑availability, low‑cost analytics platform.
This article examines the core data models and indexing mechanisms of three leading OLAP engines—Kylin, Druid, and ClickHouse—comparing their architectures, query performance characteristics, and suitability for aggregation versus detailed query workloads in modern big data environments.
This article details how Ctrip's senior engineering manager leveraged ClickHouse to build a high‑availability, sub‑second response data platform handling nearly 700 billion rows, describing the motivations, architecture, data synchronization processes, performance gains, challenges, and practical recommendations for large‑scale analytics.
This article details JD's Golden Eye business's adoption of ClickHouse for offline and real‑time traffic data analysis, covering system architecture, data ingestion pipelines, high‑availability design, monitoring, performance optimizations, and practical trade‑offs, offering insights for large‑scale analytical database deployments.
Suning’s senior architect Yang Zhaohui explains how his team rebuilt the tag platform with ClickHouse, using RoaringBitmap and custom optimizations to achieve second-level queries on billions of user tags, dramatically cutting response time, reducing hardware costs, and enabling real-time marketing insights.
This article describes how 58.com designed and optimized a ClickHouse‑based OLAP platform for massive user‑behavior data, covering the reasons for choosing ClickHouse, its key features, multi‑layer architecture, configuration management, automation scripts, monitoring, performance benchmarks, and future improvement plans.
This article provides a comprehensive technical comparison of HBase, Kudu, and ClickHouse—covering installation dependencies, architecture, basic read/write and query operations, real‑world use cases at Didi, a Kudu‑based real‑time data warehouse, and ClickHouse log‑analysis practices—highlighting each system’s strengths and trade‑offs for big‑data workloads.
This article explains how Alibaba's traffic analysis platform evolved to require real‑time analytics, outlines the business background, data model and metric system, discusses the difficulties of big‑data processing, and describes why ClickHouse was chosen and how its features solve those challenges.
This article compares Apache Kylin and ClickHouse, two popular big‑data OLAP engines, by examining their technical principles, storage structures, optimization techniques, and ideal application scenarios to help readers make an informed technology selection.
This article explains how Suning built a high‑performance user‑tag platform on ClickHouse, replacing Elasticsearch with bitmap‑based storage and a new architecture that delivers sub‑second profiling queries for over 600 million users, detailing the design, implementation, and future enhancements.
This article explains how Suning built a massive user‑tag platform using ClickHouse and RoaringBitmap to enable second‑level user profiling for over 600 million users, covering tag generation, storage, query architecture, and performance optimizations for marketing campaigns.
This article examines the data models and index structures of the three leading OLAP engines—Kylin, Druid, and ClickHouse—offering technical explanations, code examples, and a concise comparison to help readers understand their strengths, limitations, and suitable use cases.
This article explains how to build a real‑time click‑stream data warehouse using Flink for stream processing and ClickHouse for near‑real‑time OLAP, covering click‑stream characteristics, dimensional modeling, layered warehouse design, async dimension joins, sink implementation, and data rebalancing strategies.
This article introduces ClickHouse, compares column‑store and row‑store databases, shows how migrating a 50‑million‑row MySQL table to ClickHouse reduced query time from minutes to under one second, and shares practical installation, migration, performance testing, and synchronization tips.
This article explains the concept of sparse indexes in database systems, compares them with dense indexes, and demonstrates their use in Kafka log files and ClickHouse MergeTree tables, highlighting implementation details, lookup procedures, and configuration parameters.
This article introduces ClickHouse, compares column‑ and row‑oriented storage, explains a real‑world migration from MySQL to ClickHouse that reduced a 3‑minute query to under one second, details installation, migration methods, performance results, synchronization options, and common pitfalls.
Ctrip’s hotel data team tackled a 3 TB daily data load by building a ClickHouse cluster on VMware, creating custom sync and execution tools, applying query optimizations, and handling merge and memory errors, ultimately achieving over 400% performance gains across multiple reporting themes.
This article explains what ClickHouse is, compares column‑store and row‑store databases, shows how to migrate large MySQL tables to ClickHouse, presents performance test results, discusses data synchronization methods, highlights why ClickHouse is fast, and shares common migration pitfalls.
This article explains how JD.com’s JUST platform leverages the open‑source columnar database ClickHouse to store, query and analyze massive time‑series data, covering data modeling, lifecycle management, system goals, technology selection, cluster architecture, deployment, scaling and future enhancements.
This article dissects ClickHouse's Distributed table write path, detailing how the Distributed engine creates tables, decides between synchronous and asynchronous insertion, manages local and remote shards, handles sharding keys, and ensures atomic data distribution across the cluster.
This article explains how ClickHouse’s column‑oriented storage, vectorized execution engine, rich DBMS capabilities, flexible table engines, and carefully designed distributed architecture enable it to handle massive workloads with sub‑second query latency, making it a standout OLAP solution.
This article introduces ClickHouse as an OLAP columnar database, explains the differences between row‑ and column‑oriented storage, details a real‑world migration from MySQL using a CREATE TABLE AS SELECT approach, presents performance comparisons, discusses synchronization methods, and shares common pitfalls and solutions.
This article explains how JD Cloud’s JCHDB, built on ClickHouse, achieves millisecond‑level queries on billions of rows through columnar storage, distributed multi‑master architecture, SIMD vector engine, sparse indexing, and specialized table engines, and outlines the ideal use cases and deployment details.
Facing growing log volumes and high Elasticsearch costs, Ctrip migrated its 200 TB daily logs to ClickHouse, achieving up to 38× faster queries, 60 % lower server resources, and simplified operations through columnar storage, sharding, and custom dashboards, while detailing deployment, tuning, and common pitfalls.
This article explains why ClickHouse, the high‑performance columnar OLAP database, can return results on billions of rows within seconds, detailing its columnar storage, MergeTree engine, and how JD Cloud deploys and optimizes it on Kubernetes for scalability and reliability.
This article introduces ClickHouse, a column‑store OLAP database, explains its advantages over row‑store systems, guides installation on macOS, details migration strategies from MySQL, presents performance benchmarks showing 200‑fold speedups, and discusses synchronization methods, pitfalls, and practical tips.
This article details Ctrip's practical implementation of ClickHouse for its hotel data warehouse, covering background, technical selection, integration tooling, code optimization techniques, exception and server fault handling, and the resulting performance gains.
This article introduces ClickHouse as a high‑performance columnar OLAP database, compares it with row‑based systems, details migration methods from MySQL, presents performance benchmarks showing dramatic speed gains, discusses data synchronization strategies, explains why ClickHouse is fast, and shares common pitfalls and solutions.
This article explains ClickHouse's strengths as a high‑performance MPP database, details hardware selection, read/write separation, shard expansion steps, batch‑size tuning, and presents a three‑layer monitoring model, while also describing its practical application in Tencent's game analytics platform.
This article introduces ClickHouse as an OLAP columnar database, compares its storage model with row‑based databases, details migration from MySQL, presents performance benchmarks showing query speeds under one second for 50 million rows, and shares practical deployment tips and pitfalls.
This guide walks through deploying clickhouse-exporter, configuring Prometheus to scrape its metrics, and importing a Grafana dashboard to monitor ClickHouse single‑node or cluster performance, providing a practical monitoring solution for the database.
Tencent’s game marketing analysis system “EAS” evolved from inefficient TDW HiveSQL jobs and file‑heavy real‑time pipelines to a scalable ClickHouse‑based deduplication service that processes hundreds of thousands of daily activity counts in sub‑second time, offering fast, reliable, and maintainable participant deduplication for massive marketing campaigns.
JD’s Algorithmic Intelligence team built a ClickHouse‑based real‑time analytics engine that ingests Kafka and offline data, uses MergeTree tables with strategic partitioning and sorting, and employs batch writes, materialized views, and monitoring to achieve second‑level queries over billions of rows.
This article explains how Suning's big‑data center leverages ClickHouse’s columnar OLAP engine and a full‑chain monitoring platform to achieve real‑time query tracing, slow‑query analysis, cluster health checks, and resource‑level alerts across diverse business scenarios.
This article explains how ClickHouse's MergeTree engine creates partition directories based on a partition key, details the naming convention PartitionID_MinBlockNum_MaxBlockNum_Level, and describes the automatic and manual merge processes that consolidate partitions for efficient storage.
Tencent Cloud’s managed ClickHouse service now includes an automated data‑balancing feature that, after user authorization and bandwidth configuration, creates migration plans to redistribute tables across new or decommissioned nodes, eliminating manual rebalancing, reducing operational overhead, and ensuring balanced storage during elastic scaling.
This article explains ClickHouse's four composite data types—Array, Tuple, Enum, and Nested—detailing their definitions, type inference rules, practical SQL examples, and common pitfalls to help developers model complex data structures efficiently.
This tutorial walks through building a three‑node ClickHouse cluster, configuring host mappings and XML files, setting up ZooKeeper for internal replication, creating local and distributed tables, testing data insertion and queries, and demonstrating how to achieve high availability with replicated tables.
This article demonstrates how to leverage ClickHouse’s parametric aggregate and higher‑order functions to perform path matching, intelligent path detection, ordered funnel conversion, retention calculation, and session statistics for user behavior analysis in a big‑data environment.
This article details ByteDance's engineering practices for using ClickHouse to ingest, store, and query massive real‑time recommendation and advertising data, covering early external‑transaction mechanisms, the risks of direct INSERTs, the design and evaluation of Kafka Engine versus Flink pipelines, and a series of performance and reliability improvements implemented to support high‑frequency workloads.
This article describes how a data management platform (DMP) at Beike leverages ClickHouse bitmap structures and Spark pipelines to generate global numeric user IDs, design tag-specific bitmap rules for enum, continuous, and date attributes, handle boundary cases, and produce high‑performance bitmap SQL for real‑time user group estimation and complex segment logic.
This article compares the data modeling and indexing techniques of three mainstream OLAP engines—Kylin, Druid, and ClickHouse—explaining their conversion processes, storage formats, query execution paths, and summarizing their strengths and limitations for aggregation and detail query scenarios.
This article explains how ClickHouse's MergeTree engine creates partition directories, the naming convention for partitions, and the automatic and manual merging processes that consolidate parts while tracking block numbers and merge levels.
This article introduces ClickHouse, an open‑source column‑oriented OLAP database, detailing its key features, appropriate use cases, installation steps, performance benchmark queries, and how it compares with other columnar storage solutions while highlighting its adoption by major internet companies.
This article details how ByteDance leverages ClickHouse to power large‑scale advertising audience estimation, profiling, and statistical analysis, describing the challenges of massive data, strict latency requirements, and the evolution from a simple tag‑uid table to a bitmap‑based architecture with extensive parallel and cache optimizations.
This article explains how ClickHouse can act as a MySQL replica, covering full and incremental synchronization, supported MySQL versions, DDL compatibility, handling of DELETE/UPDATE events, the underlying binlog mechanism, and provides complete code examples to set up both MySQL master and ClickHouse slave.
QQ Music replaced its slow Hive warehouse with a massive ClickHouse cluster, achieving sub‑second to ten‑second query latency on petabyte‑scale data, enabling real‑time analytics for non‑technical users, and following five operational best practices—ZooKeeper planning, idempotent writes, sensible partitions, read‑write separation, and localized joins—while leveraging Tencent Cloud’s managed ClickHouse service.
This article provides a comprehensive technical overview of ClickHouse, covering its complete DBMS capabilities, column‑oriented storage and compression, vectorized execution engine, relational SQL support, diverse table engines, multi‑master clustering, sharding, and the design philosophies that make it exceptionally fast for large‑scale analytical workloads.
The article describes how a lightweight big‑data analysis platform called MVP was built from scratch—using a User‑Event‑Config model, HDFS + ClickHouse + Spark, and four modules for metric monitoring, root‑cause alerts, deep growth analysis, and A/B testing—enabling real‑time insights in seconds instead of days and dramatically accelerating product‑growth operations.
This article describes how JD.com’s Algorithmic Intelligence team built a ClickHouse‑based real‑time analytics engine, covering ClickHouse fundamentals, MergeTree table design, Kafka‑Flink data pipelines, JDBC batch loading, query‑optimization techniques, and monitoring for handling billions of rows with sub‑second response times.
This article explains how FunTouTiao builds a high‑performance real‑time analytics pipeline using Flink, Hive, and ClickHouse, covering business scenarios, hour‑level and second‑level Flink‑to‑Hive architectures, streaming file sink mechanics, multi‑user permissions, ClickHouse performance tricks, and future roadmap for unified stream‑batch storage.
This article describes how a company replaced its Zabbix‑based monitoring with a scalable, Prometheus‑driven alert engine that leverages ClickHouse for storage, remote‑storage integration via Prom2Click, and materialized views to provide flexible, SQL‑based business metric alerts.
QuTouTiao leverages Flink and ClickHouse to build a high‑performance real‑time analytics platform that supports hourly Hive pipelines and sub‑second ClickHouse queries, achieving sub‑second response for 80% of requests through streaming ingestion, exactly‑once semantics, multi‑cluster coordination, and optimized ClickHouse storage and connector designs.