BestHub
Sign in
Tagged articles

ClickHouse

511 articles · Page 2 of 6
JD Tech
JD Tech
Jul 2, 2024 · Big Data

Real‑Time Monitoring Dashboard for Logistics Supply Chain: Architecture, Data Modeling, and Stability Design

This article presents the design and implementation of a high‑availability, real‑time logistics supply‑chain monitoring dashboard, covering its data processing pipeline with Flink, storage choices between Elasticsearch and ClickHouse, multi‑layer architecture, metric consistency, stability mechanisms, extensibility configurations, and monitoring practices.

ClickHouseData PipelineElasticsearch
0 likes · 11 min read
Real‑Time Monitoring Dashboard for Logistics Supply Chain: Architecture, Data Modeling, and Stability Design
DataFunTalk
DataFunTalk
Jun 28, 2024 · Big Data

Accelerating Spark with ClickHouse: Native Optimization Techniques and Performance Evaluation

This article presents a comprehensive technical overview of using ClickHouse as a native backend to accelerate Spark SQL execution, covering Spark performance bottlenecks, ClickHouse's CPU‑level optimizations, the design and implementation of the Spark‑Native integration, and detailed TPC‑DS benchmark results demonstrating up to 3.5× speedup.

ClickHousePerformance OptimizationSpark
0 likes · 33 min read
Accelerating Spark with ClickHouse: Native Optimization Techniques and Performance Evaluation
Baidu Geek Talk
Baidu Geek Talk
Jun 24, 2024 · Big Data

Accelerating Spark with ClickHouse Native Techniques: Design, Implementation, and Performance Evaluation

The paper presents a Spark acceleration framework that replaces Java‑based task operators with a ClickHouse native library, converting plans via Protobuf and JNI, leveraging columnar storage, SIMD and JIT to achieve up to 3× speed‑up on TPC‑DS workloads while providing fallback mechanisms to ensure no performance loss.

ClickHouseNative AccelerationSQL Engine
0 likes · 31 min read
Accelerating Spark with ClickHouse Native Techniques: Design, Implementation, and Performance Evaluation
Baidu Intelligent Cloud Tech Hub
Baidu Intelligent Cloud Tech Hub
Jun 24, 2024 · Big Data

Boost Spark Performance with ClickHouse: Native Acceleration Techniques

This article presents a detailed technical overview of accelerating Spark's compute engine using ClickHouse as a native backend, covering Spark performance background, ClickHouse's advantages, the design and implementation of a Spark‑Native acceleration solution, and extensive performance evaluation results.

ClickHouseNative AccelerationPerformance Optimization
0 likes · 34 min read
Boost Spark Performance with ClickHouse: Native Acceleration Techniques
DataFunTalk
DataFunTalk
Jun 9, 2024 · Big Data

Optimizing ClickHouse Performance in WeChat: Observation Tools, Lakehouse Reading, Bitmap Acceleration, and AI Integration

This article details how the WeChat team leverages ClickHouse at massive scale, introduces a suite of performance observation tools, describes lakehouse reading and bitmap optimizations, and explains the integration of AI workloads, demonstrating overall query speedups of up to tenfold across diverse scenarios.

BitmapClickHouseLakehouse
0 likes · 10 min read
Optimizing ClickHouse Performance in WeChat: Observation Tools, Lakehouse Reading, Bitmap Acceleration, and AI Integration
ITPUB
ITPUB
Jun 9, 2024 · Databases

Doris vs ClickHouse: Which Database Fits Your Workload?

This article compares Doris and ClickHouse across architecture, table creation, ecosystem integration, management tools, query performance, and join capabilities, offering practical guidance on how to choose the right database based on your specific data processing and operational requirements.

ClickHouseDorisSQL
0 likes · 10 min read
Doris vs ClickHouse: Which Database Fits Your Workload?
ITPUB
ITPUB
May 26, 2024 · Cloud Native

Containerizing Elasticsearch & ClickHouse on Kubernetes: Bilibili’s Scalable, Low‑Cost Solution

This article details Bilibili’s journey of containerizing Elasticsearch and ClickHouse on Kubernetes, covering the challenges of stateful services, architectural decisions, custom operators, storage and network solutions, deployment steps, observability enhancements, and the resulting cost, quality, and efficiency gains.

ClickHouseElasticsearchKubernetes
0 likes · 38 min read
Containerizing Elasticsearch & ClickHouse on Kubernetes: Bilibili’s Scalable, Low‑Cost Solution
ITPUB
ITPUB
May 21, 2024 · Databases

Can ClickHouse Distributed Tables Outperform Single-Node Tables? A Real-World Benchmark

This article presents a systematic benchmark comparing ClickHouse local (single‑node) tables and distributed tables across three data volumes—≈60 billion, 5 billion and 50 million rows—using a variety of aggregation and filter queries, and reveals that distributed tables dominate at large scale while the gap narrows as the dataset shrinks.

BenchmarkClickHouseDistributed Tables
0 likes · 13 min read
Can ClickHouse Distributed Tables Outperform Single-Node Tables? A Real-World Benchmark
Past Memory Big Data
Past Memory Big Data
Apr 23, 2024 · Big Data

ByConity Replaces ClickHouse for OLAP, Cutting Resource Costs Over 50%

MetaApp replaced ClickHouse with the open‑source cloud‑native warehouse ByConity, achieving more than 50% reduction in resource costs while delivering comparable or faster OLAP query performance across distinct, retention, conversion, and point‑lookup workloads, thanks to compute‑storage separation, read/write isolation, and minute‑level elastic scaling.

ByConityClickHouseOLAP
0 likes · 15 min read
ByConity Replaces ClickHouse for OLAP, Cutting Resource Costs Over 50%
vivo Internet Technology
vivo Internet Technology
Apr 17, 2024 · Big Data

Retention Analysis Model Practice Based on ClickHouse

The article explains retention analysis models, their importance for user loyalty, outlines offline Hive architecture, then shows how ClickHouse’s retention() function and columnar storage dramatically speed up multi‑day retention calculations, providing SQL examples and practical guidance for product analytics.

ClickHouseData ModelingHive
0 likes · 17 min read
Retention Analysis Model Practice Based on ClickHouse
ITPUB
ITPUB
Apr 11, 2024 · Big Data

Query 100K Items from 10M+ Records: CK, ES Scroll, HBase, RediSearch

When faced with a business requirement to filter up to 100 000 records from a pool of tens of millions and then sort and de‑duplicate them, this article explores four technical solutions—multithreaded ClickHouse pagination, Elasticsearch scroll‑scan, a combined Elasticsearch‑HBase approach, and RediSearch with RedisJSON—detailing their design, implementation, performance testing, and trade‑offs.

ClickHouseElasticsearchHBase
0 likes · 12 min read
Query 100K Items from 10M+ Records: CK, ES Scroll, HBase, RediSearch
NetEase Cloud Music Tech Team
NetEase Cloud Music Tech Team
Apr 11, 2024 · Backend Development

Design and Implementation of an Online Configurable Data Consumption Service for NetEase Cloud Music Frontend Performance Monitoring (Corona)

The article details NetEase Cloud Music’s end‑to‑end, online‑configurable data‑consumption service and schema‑driven visualization platform that transform raw client logs into ClickHouse records, automatically generate tables and dashboards, and provide observability, dramatically reducing manual effort while supporting over twenty performance metrics for frontend monitoring.

ClickHouseData PipelinePerformance Monitoring
0 likes · 17 min read
Design and Implementation of an Online Configurable Data Consumption Service for NetEase Cloud Music Frontend Performance Monitoring (Corona)
dbaplus Community
dbaplus Community
Apr 8, 2024 · Cloud Native

Containerizing Elasticsearch & ClickHouse on Kubernetes: Challenges & Solutions

Facing the complexities of running stateful services like Elasticsearch and ClickHouse in production, Bilibili’s infrastructure team detailed their migration to Kubernetes, describing the architectural design, custom operators, storage provisioning with LVM, network configuration, high‑availability strategies, observability, and the resulting cost, quality, and efficiency gains.

ClickHouseCloudNativeElasticsearch
0 likes · 37 min read
Containerizing Elasticsearch & ClickHouse on Kubernetes: Challenges & Solutions
Practical DevOps Architecture
Practical DevOps Architecture
Apr 4, 2024 · Databases

ClickHouse Training Course Overview and Curriculum

This article introduces a comprehensive ClickHouse training program that covers fundamental concepts, architecture, installation, distributed cluster design, data import, performance tuning, and includes a detailed list of 33 video modules and additional recommended reading resources for large‑scale data analytics.

ClickHouseColumnar DatabaseData Analytics
0 likes · 4 min read
ClickHouse Training Course Overview and Curriculum
Tencent Cloud Developer
Tencent Cloud Developer
Apr 2, 2024 · Backend Development

tRPC Scaffolding Tooling and Observability Best Practices for Tencent Docs Backend

By introducing the unified tRPC scaffolding tool trpcx and embedding OpenTelemetry‑generated observability configurations, the Tencent Docs backend team streamlined service creation, standardized directory structures, migrated metrics and logs to ClickHouse for cost‑effective performance, and established best‑practice workflows that dramatically improve development speed and fault‑diagnosis efficiency.

ClickHouseOpenTelemetrybackend development
0 likes · 18 min read
tRPC Scaffolding Tooling and Observability Best Practices for Tencent Docs Backend
dbaplus Community
dbaplus Community
Mar 19, 2024 · Big Data

How JD’s Mini‑Program Data Center Powers Real‑Time Analytics and Monitoring

JD’s Mini‑Program Data Center integrates data collection, storage, and real‑time analysis using Flink, ClickHouse, and Elasticsearch to provide comprehensive monitoring, user behavior insights, and scalable analytics for mini‑programs across JD’s ecosystem, enabling precise operations and future AI‑driven enhancements.

ClickHouseElasticsearchJD Mini-Program
0 likes · 19 min read
How JD’s Mini‑Program Data Center Powers Real‑Time Analytics and Monitoring
dbaplus Community
dbaplus Community
Mar 12, 2024 · Databases

How Didi Scaled Log Search by Replacing Elasticsearch with ClickHouse

Facing PB‑scale daily logs and costly Elasticsearch bottlenecks, Didi redesigned its log‑search architecture by migrating to ClickHouse, detailing the challenges, storage redesign, cluster upgrades, performance optimizations, stability fixes, and the resulting cost reduction and query speed gains.

ClickHouseLog Storagedistributed systems
0 likes · 15 min read
How Didi Scaled Log Search by Replacing Elasticsearch with ClickHouse
Bilibili Tech
Bilibili Tech
Mar 12, 2024 · Cloud Native

Containerizing Elasticsearch and ClickHouse on Kubernetes: Architecture, Implementation, and Benefits

Bilibili migrated its Elasticsearch and ClickHouse clusters to Kubernetes using custom operators, CRDs, LVM‑based local storage, MacVLAN networking, and pod anti‑affinity, achieving higher resource utilization, isolation, and automation that cut server count, reduced latency spikes, and lowered operational costs dramatically.

ClickHouseElasticsearchKubernetes
0 likes · 38 min read
Containerizing Elasticsearch and ClickHouse on Kubernetes: Architecture, Implementation, and Benefits
Linux Code Review Hub
Linux Code Review Hub
Mar 11, 2024 · Databases

How Didi Built a Next‑Gen Log Storage System with ClickHouse

Didi migrated its massive PB‑scale log data from Elasticsearch to ClickHouse, redesigning storage with separate Log and Trace clusters, optimizing partition and sorting keys, introducing native TCP connectors, and revamping HDFS cold‑hot separation, achieving up to four‑fold query speed gains and 30% lower hardware costs.

ClickHouseFlinkHDFS
0 likes · 15 min read
How Didi Built a Next‑Gen Log Storage System with ClickHouse
Didi Tech
Didi Tech
Mar 5, 2024 · Databases

Migrating Didi's Log Retrieval from Elasticsearch to ClickHouse: Architecture, Challenges, and Performance Optimizations

Didi replaced its Elasticsearch‑based log platform with ClickHouse, redesigning architecture into isolated Log and Trace clusters, using hourly‑partitioned MergeTree tables and aggregating views to handle petabyte‑scale writes, diverse low‑latency queries, and high QPS, achieving over 400 nodes, 40 GB/s throughput, 30 % cost savings and four‑fold query latency reduction.

ClickHouseData StorageElasticsearch
0 likes · 15 min read
Migrating Didi's Log Retrieval from Elasticsearch to ClickHouse: Architecture, Challenges, and Performance Optimizations
Volcano Engine Developer Services
Volcano Engine Developer Services
Feb 29, 2024 · Big Data

How MetaApp Cut Data Warehouse Costs by 50% with ByConity

MetaApp replaced ClickHouse with the open‑source cloud‑native data warehouse ByConity, achieving over 50% cost reduction and faster, more stable OLAP queries by separating storage and compute, simplifying scaling, and improving resource utilization across a range of analytics workloads such as deduplication, retention, conversion and point‑lookup.

ByConityClickHouseOLAP
0 likes · 13 min read
How MetaApp Cut Data Warehouse Costs by 50% with ByConity
Pupu Technology
Pupu Technology
Feb 7, 2024 · Databases

How Pupu APM Boosted Write Speed and Cut Storage Costs with ClickHouse

Facing soaring data volumes and rising Elasticsearch costs, Pupu's platform team migrated its APM storage to ClickHouse, achieving up to three million spans per second writes, 38% storage savings, and faster query performance through materialized views and careful deployment tuning.

APMClickHousecloud-native
0 likes · 11 min read
How Pupu APM Boosted Write Speed and Cut Storage Costs with ClickHouse
Architect
Architect
Feb 1, 2024 · Backend Development

Design and Optimization of Trace2.0: A High‑Performance Backend Tracing System

Trace2.0 is an OpenTelemetry‑based application monitoring system that processes petabyte‑scale trace data using multi‑channel client protocols, gRPC, load‑balancing optimizations, ZSTD compression, Kafka pipelines, ClickHouse storage, and a JDK 21 upgrade with virtual threads, achieving significant performance and cost improvements.

ClickHouseJDK21OpenTelemetry
0 likes · 15 min read
Design and Optimization of Trace2.0: A High‑Performance Backend Tracing System
Efficient Ops
Efficient Ops
Jan 31, 2024 · Databases

Why ClickHouse Beats Elasticsearch for High‑Performance Log Analytics

Facing data security and cost challenges in SaaS, the author evaluates ClickHouse versus Elasticsearch, highlighting ClickHouse’s superior write throughput, query speed, lower storage and CPU usage, and provides detailed deployment guides for Zookeeper, Kafka, FileBeat, and ClickHouse to build a cost‑effective private analytics platform.

ClickHouseDatabase deploymentElasticsearch
0 likes · 8 min read
Why ClickHouse Beats Elasticsearch for High‑Performance Log Analytics
dbaplus Community
dbaplus Community
Jan 23, 2024 · Operations

How We Built a Scalable Real‑Time Log Center with ClickHouse and ELK

Facing massive data volumes, the team at Kuaidi100 redesigned their logging platform, moving from a file‑based system to an ELK stack and finally to a ClickHouse‑based architecture, achieving real‑time, scalable, cost‑effective log collection, analysis, and alerting while addressing storage, performance, and maintenance challenges.

ClickHouseELKLog Management
0 likes · 12 min read
How We Built a Scalable Real‑Time Log Center with ClickHouse and ELK
Bilibili Tech
Bilibili Tech
Jan 23, 2024 · Databases

Unique Engine Design and Implementation in ClickHouse for Bilibili Live Guild Data

Bilibili migrated its live‑guild analytics from MySQL to ClickHouse, creating a custom ReplicatedUniqueMergeTree engine that uses delete‑on‑insert, min‑max and hash‑bucketed indexes with delete bitmaps to achieve 10‑20× faster queries and scalable near‑real‑time reporting despite higher write latency.

ClickHouseUnique Enginedata engineering
0 likes · 18 min read
Unique Engine Design and Implementation in ClickHouse for Bilibili Live Guild Data
JD Tech
JD Tech
Jan 18, 2024 · Databases

Understanding ClickHouse: Architecture, Principles, and Performance

This article introduces ClickHouse, an open‑source columnar OLAP database, explains its architecture—including columnar storage, block processing, LSM, indexing and vectorized execution—highlights its performance advantages over other engines, and discusses its limitations such as write‑amplification, concurrency constraints, and ZooKeeper dependency.

ClickHouseColumnar DatabaseOLAP
0 likes · 12 min read
Understanding ClickHouse: Architecture, Principles, and Performance
Efficient Ops
Efficient Ops
Jan 17, 2024 · Operations

How We Built a Scalable Cloud‑Native Log Center with ClickHouse

This article details a courier company's evolution from a simple file‑based logging system to a cloud‑native log center, describing the limitations of the original architecture, the migration to an ELK stack, subsequent challenges, and the final redesign using ClickHouse for high compression, low cost, and improved query performance.

ClickHouseELKLog Management
0 likes · 12 min read
How We Built a Scalable Cloud‑Native Log Center with ClickHouse
MaGe Linux Operations
MaGe Linux Operations
Jan 3, 2024 · Big Data

ClickHouse vs Elasticsearch: Faster, Cheaper Log Analytics Explained

This article compares ClickHouse and Elasticsearch for log analytics, highlighting ClickHouse's superior write throughput, query speed, and lower server costs, then provides a detailed, cost‑effective deployment guide covering Zookeeper, Kafka, FileBeat, ClickHouse installation, and visualization with ClickVisual, plus optimization tips.

ClickHouseElasticsearchLog Analytics
0 likes · 15 min read
ClickHouse vs Elasticsearch: Faster, Cheaper Log Analytics Explained
DataFunTalk
DataFunTalk
Jan 3, 2024 · Databases

ClickHouse 2024 Core New Features and Product Development Directions

This article introduces ClickHouse, an open‑source columnar OLAP database, outlines its architecture, advantages, self‑hosted and cloud deployment models, highlights recent product features such as async inserts, JSON support, Parquet acceleration, query caching, and summarizes a Q&A covering semi‑structured data, MPP, virtual columns, and future roadmap.

ClickHouseColumnar DatabaseOLAP
0 likes · 12 min read
ClickHouse 2024 Core New Features and Product Development Directions
Efficient Ops
Efficient Ops
Dec 27, 2023 · Big Data

Why ClickHouse Beats Elasticsearch for Log Analytics – Performance, Cost & Deployment

This article compares ClickHouse and Elasticsearch for log analytics, highlighting ClickHouse’s superior write throughput, query speed, and lower server costs, then details a cost‑effective deployment architecture—including Zookeeper, Kafka, FileBeat, and ClickHouse setup—and shares optimization tips and visualization using ClickVisual.

ClickHouseElasticsearchKafka
0 likes · 13 min read
Why ClickHouse Beats Elasticsearch for Log Analytics – Performance, Cost & Deployment
dbaplus Community
dbaplus Community
Nov 20, 2023 · Operations

Can VictoriaLogs Really Beat Elasticsearch, Loki, and ClickHouse? A Deep Dive

VictoriaLogs, a log‑storage system marketed as a cost‑effective, high‑performance alternative, is compared against Elasticsearch/OpenSearch, Grafana Loki, and ClickHouse, highlighting its lower RAM and disk usage, faster queries, simplified setup, and specialized features such as LogSQL, Bloom filters, and custom compression.

ClickHouseGrafana LokiLog Management
0 likes · 9 min read
Can VictoriaLogs Really Beat Elasticsearch, Loki, and ClickHouse? A Deep Dive
dbaplus Community
dbaplus Community
Nov 14, 2023 · Databases

How Didi Cut ClickHouse CPU Usage by 90%: Optimizing BgMoveProcPool Threads

This article details how Didi identified excessive CPU consumption by ClickHouse's BgMoveProcPool threads, traced the root cause to unnecessary part‑move checks, introduced a simple early‑exit guard in selectPartsForMove, and achieved a dramatic reduction in CPU load while contributing the fix upstream.

Background ThreadsCPUClickHouse
0 likes · 10 min read
How Didi Cut ClickHouse CPU Usage by 90%: Optimizing BgMoveProcPool Threads
DataFunTalk
DataFunTalk
Nov 7, 2023 · Big Data

Comprehensive Guide to User Crowd Analysis: Distribution, Metrics, Drill‑down, Cross, and Comparative Methods with Implementation Details

This article explains the concepts, analytical methods, visualizations, and SQL implementation of user crowd analysis—including distribution, metric, drill‑down, cross, and comparative analyses—while also covering trend monitoring, TGI calculation, and handling of array‑type tags in ClickHouse and Hive.

ClickHouseData VisualizationSQL
0 likes · 17 min read
Comprehensive Guide to User Crowd Analysis: Distribution, Metrics, Drill‑down, Cross, and Comparative Methods with Implementation Details
Inke Technology
Inke Technology
Oct 31, 2023 · Operations

How We Re‑engineered Our Log Platform to Cut Costs by 60% with ClickHouse

This article details the redesign of a company’s logging infrastructure—from an ELK‑based solution to a ClickHouse‑powered architecture—highlighting the motivations, key requirements, component choices, configuration examples, performance optimizations, and the resulting cost and storage benefits.

ClickHouseVectorbig data
0 likes · 13 min read
How We Re‑engineered Our Log Platform to Cut Costs by 60% with ClickHouse
NetEase Cloud Music Tech Team
NetEase Cloud Music Tech Team
Oct 27, 2023 · Databases

Corona Technical Series: Time-Series Databases in Corona

The article explains how Corona leverages three time‑series databases—InfluxDB for storing pre‑aggregated user metrics and platform health data, ClickHouse for real‑time multidimensional log analysis with aggregations, and ElasticSearch for full‑text searchable log monitoring—detailing their schema designs and query examples.

ClickHouseCoronaInfluxDB
0 likes · 19 min read
Corona Technical Series: Time-Series Databases in Corona
Architect
Architect
Oct 26, 2023 · Big Data

Design and Optimization of Bilibili Log Service 2.0 Using ClickHouse and OpenTelemetry

This article details Bilibili's evolution of its log system from an Elastic Stack‑based solution to a ClickHouse‑backed architecture with OpenTelemetry, describing the challenges of cost, stability, and scalability, the new components such as Log‑Agent, Log‑Ingester, and a custom visualization platform, and the performance gains and future directions.

ClickHouseOpenTelemetrylarge-scale logging
0 likes · 26 min read
Design and Optimization of Bilibili Log Service 2.0 Using ClickHouse and OpenTelemetry
dbaplus Community
dbaplus Community
Oct 18, 2023 · Databases

Doris vs ClickHouse: Which Database Delivers Faster Writes and Queries?

This article presents a systematic performance comparison between Doris and ClickHouse, covering data ingestion speed, SQL syntax differences, hardware impact, and detailed query benchmarks across multiple scenarios, ultimately revealing that each system excels in different use cases.

ClickHouseDorisPerformance
0 likes · 15 min read
Doris vs ClickHouse: Which Database Delivers Faster Writes and Queries?
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 7, 2023 · Big Data

Comprehensive Guide to OLAP Optimization and ClickHouse Performance Tuning

This article explains how to optimize OLAP workloads by balancing normalization and denormalization, applying data sharding, replication, indexing, partitioning, materialized views, columnar storage, compression, and lifecycle management, and provides practical ClickHouse SQL examples for index creation, partitioning, and query plan analysis.

ClickHouseData ModelingIndexing
0 likes · 15 min read
Comprehensive Guide to OLAP Optimization and ClickHouse Performance Tuning
dbaplus Community
dbaplus Community
Sep 19, 2023 · Cloud Native

How REDck Transformed ClickHouse into a Scalable Cloud‑Native Real‑Time Data Warehouse

REDck, a cloud‑native real‑time data warehouse built on open‑source ClickHouse, overcomes the original MPP architecture’s scaling and maintenance limits by separating compute and storage, introducing unified metadata, multi‑level caching, bucket‑based sharding, and distributed transaction support, delivering petabyte‑scale, 99.9% availability and ten‑fold cost and performance gains for Xiaohongshu’s diverse workloads.

ClickHouseCompute-Storage SeparationReal-time OLAP
0 likes · 22 min read
How REDck Transformed ClickHouse into a Scalable Cloud‑Native Real‑Time Data Warehouse
DataFunTalk
DataFunTalk
Sep 17, 2023 · Cloud Native

REDck: A Cloud‑Native Real‑Time Data Warehouse Built on ClickHouse

REDck is a cloud‑native, storage‑compute separated real‑time OLAP data warehouse derived from ClickHouse that addresses scalability, operational cost, and reliability challenges through a unified metadata service, object‑storage optimizations, multi‑level caching, distributed task scheduling, and two‑phase commit transactions.

ClickHouseReal-time OLAPcloud-native
0 likes · 18 min read
REDck: A Cloud‑Native Real‑Time Data Warehouse Built on ClickHouse
ITPUB
ITPUB
Sep 15, 2023 · Databases

Importing Billions of Kafka Rows into Doris and Benchmarking Against ClickHouse

This article explains Doris's various data import methods, focuses on the routine load approach for Kafka streams, describes how to handle mixed‑schema topics using the max_error_number parameter, and compares query performance of a 130 million‑row dataset against ClickHouse, highlighting each system's strengths and limitations.

ClickHouseDorisKafka
0 likes · 10 min read
Importing Billions of Kafka Rows into Doris and Benchmarking Against ClickHouse
Architect
Architect
Sep 11, 2023 · Databases

How eBay Scaled ClickHouse with Read/Write Separation and Keeper

This article details eBay's event monitoring platform architecture, explains the challenges of high‑load OLAP workloads on ClickHouse clusters, describes the design and implementation of read/write separation and multi‑shard Keeper coordination, and shares concrete configuration snippets, performance observations, and production lessons learned.

ClickHouseDatabase ScalingKeeper
0 likes · 20 min read
How eBay Scaled ClickHouse with Read/Write Separation and Keeper
ITPUB
ITPUB
Sep 11, 2023 · Cloud Native

How REDck Transforms ClickHouse into a Scalable Cloud‑Native Real‑Time Data Warehouse

Xiaohongshu built REDck, a cloud‑native, storage‑compute separated real‑time OLAP warehouse on ClickHouse, addressing scaling, cost, and reliability challenges through a unified metadata service, object‑storage optimizations, multi‑level caching, distributed task scheduling, bucketing, and exactly‑once transaction support.

CachingClickHouseObject Storage
0 likes · 21 min read
How REDck Transforms ClickHouse into a Scalable Cloud‑Native Real‑Time Data Warehouse
Xiaohongshu Tech REDtech
Xiaohongshu Tech REDtech
Sep 6, 2023 · Databases

REDck: A Cloud‑Native Real‑Time OLAP Data Warehouse Built on ClickHouse

REDck is a cloud‑native, real‑time OLAP data warehouse built on ClickHouse that adds elastic compute and storage scaling, object‑storage optimizations, multi‑level caching, and exactly‑once ingestion, delivering petabyte‑scale interactive analytics with ten‑fold CPU efficiency, ten‑fold cost reduction, and 99.9% availability.

ClickHouseReal-time OLAPbig data
0 likes · 21 min read
REDck: A Cloud‑Native Real‑Time OLAP Data Warehouse Built on ClickHouse
JD Retail Technology
JD Retail Technology
Sep 4, 2023 · Big Data

JD Mini Program Data Center: Architecture, Milestones, and Real‑time Analytics Solutions

The article details the JD Mini Program platform, its data‑center development milestones, comprehensive business panorama, technical architecture, data collection, storage, and analysis pipelines—including Flink‑based real‑time monitoring, ClickHouse custom analytics, and Elasticsearch user‑behavior insights—while outlining current challenges and future AI‑driven enhancements.

ClickHouseElasticsearchFlink
0 likes · 16 min read
JD Mini Program Data Center: Architecture, Milestones, and Real‑time Analytics Solutions
DataFunTalk
DataFunTalk
Aug 28, 2023 · Big Data

Practical Experience of an E‑commerce Platform’s Offline and Real‑time Data Warehouse

This article shares the practical architecture, technology selection, implementation details, and evolution of an e‑commerce platform’s offline and real‑time data warehouses, covering data modeling, processing pipelines, system components such as Hive, Spark, Flink, ClickHouse, Doris, and Hudi, and the lessons learned from multiple production deployments.

ClickHouseDorisE‑commerce
0 likes · 18 min read
Practical Experience of an E‑commerce Platform’s Offline and Real‑time Data Warehouse
JD Retail Technology
JD Retail Technology
Aug 21, 2023 · Artificial Intelligence

ChatGPT-4 Enhances Data Analysis Efficiency and Insight Across Big Data Scenarios

This article examines how ChatGPT-4, as an advanced natural‑language‑processing model, can streamline data analysis tasks—from generating Hive table definitions and sample data to crafting complex HiveSQL queries, visualizing results, and implementing ClickHouse and Flink solutions—thereby improving efficiency, insight, and problem‑solving in big‑data environments.

Artificial IntelligenceChatGPT-4ClickHouse
0 likes · 7 min read
ChatGPT-4 Enhances Data Analysis Efficiency and Insight Across Big Data Scenarios
dbaplus Community
dbaplus Community
Aug 15, 2023 · Databases

Why ClickHouse Outperforms MySQL, Elasticsearch, and HBase for Massive Event Data

This article examines the massive data storage and real‑time analysis needs of an activity platform, evaluates MySQL, sharded MySQL, Elasticsearch and HBase, and explains why ClickHouse—with its columnar storage, MergeTree engine, vectorized execution, and distributed architecture—offers the best balance of write performance, query speed, and scalability for billions of records.

ClickHouseColumnar DatabaseDistributed Architecture
0 likes · 31 min read
Why ClickHouse Outperforms MySQL, Elasticsearch, and HBase for Massive Event Data
DataFunSummit
DataFunSummit
Aug 10, 2023 · Databases

ClickHouse Deployment in Lenovo Manufacturing: Architecture, Data Integration, and Performance Optimization

This article details Lenovo's implementation of ClickHouse in a manufacturing environment, covering the current data landscape, cluster architecture, integration challenges, performance enhancements, and solutions such as Seatunnel and query pre‑aggregation, illustrating how OLAP engines can address real‑time analytics and concurrency issues in production data pipelines.

ClickHouseData IntegrationOLAP
0 likes · 11 min read
ClickHouse Deployment in Lenovo Manufacturing: Architecture, Data Integration, and Performance Optimization
MaGe Linux Operations
MaGe Linux Operations
Aug 5, 2023 · Databases

Elasticsearch vs ClickHouse: Architecture, Queries, and Performance

This article compares Elasticsearch and ClickHouse by examining their underlying architectures, node roles, query languages, and performance through a series of benchmark tests using Docker‑compose, Vector data pipelines, and Python SDKs, revealing ClickHouse’s superior speed in most query scenarios despite lacking advanced search features.

ClickHouseElasticsearch
0 likes · 12 min read
Elasticsearch vs ClickHouse: Architecture, Queries, and Performance
dbaplus Community
dbaplus Community
Aug 3, 2023 · Databases

Scaling eBay’s Sherlock.io ClickHouse Platform with Read/Write Separation and Keeper

The article details how eBay’s Sherlock.io event monitoring platform, built on ClickHouse, faced scaling and performance challenges due to ZooKeeper bottlenecks, and explains the design and implementation of read/write separation, shard‑level Keeper coordination, and related operational fixes to improve reliability and latency.

ClickHouseKeeperRead-Write Separation
0 likes · 19 min read
Scaling eBay’s Sherlock.io ClickHouse Platform with Read/Write Separation and Keeper
Efficient Ops
Efficient Ops
Aug 2, 2023 · Databases

Why ClickHouse Outperforms Elasticsearch in Real‑World Queries

This article compares Elasticsearch and ClickHouse across architecture, query capabilities, and performance using Docker‑compose stacks and Python SDK tests, demonstrating that ClickHouse often delivers superior speed, especially in aggregation and regex queries, while highlighting each system’s design trade‑offs.

ClickHouseDocker ComposeElasticsearch
0 likes · 13 min read
Why ClickHouse Outperforms Elasticsearch in Real‑World Queries
Top Architect
Top Architect
Jul 27, 2023 · Big Data

Performance Comparison of Elasticsearch and ClickHouse for Log Search

This article compares Elasticsearch and ClickHouse as log‑search solutions, detailing their architectures, Docker‑compose deployments, data‑ingestion pipelines with Vector, query syntax differences, and benchmark results that show ClickHouse generally outperforms Elasticsearch in speed and aggregation efficiency.

ClickHouseDockerElasticsearch
0 likes · 13 min read
Performance Comparison of Elasticsearch and ClickHouse for Log Search
DataFunSummit
DataFunSummit
Jul 27, 2023 · Backend Development

Building a High‑Availability ClickHouse Cluster with RaftKeeper

This article explains how RaftKeeper leverages the Raft consensus algorithm to create a high‑availability, high‑performance ClickHouse cluster across multiple data centers, covering project background, architecture, core features, performance optimizations, and real‑world deployment results.

ClickHouseCross-DataCenterDistributed Consensus
0 likes · 17 min read
Building a High‑Availability ClickHouse Cluster with RaftKeeper
dbaplus Community
dbaplus Community
Jul 26, 2023 · Databases

Mastering ClickHouse with Flink: Table Engine Choices, Performance Tuning, and Real‑World Lessons

This article details how JDQ+Flink+Elasticsearch was extended with ClickHouse for real‑time reporting, covering table‑engine selection, Flink sink implementation, query optimization techniques, high‑CPU shard analysis, and practical strategies to ensure high concurrency and stable performance in production.

ClickHouseDistributedTablesFlink
0 likes · 46 min read
Mastering ClickHouse with Flink: Table Engine Choices, Performance Tuning, and Real‑World Lessons
JD Cloud Developers
JD Cloud Developers
Jul 19, 2023 · Databases

Why ClickHouse Is the Ideal Choice for Massive Data Storage and Real‑Time Analytics

This article examines the massive‑scale data requirements of an activity‑tracking platform, compares MySQL, Elasticsearch and HBase, and explains why ClickHouse—with its columnar storage, MergeTree engine, vectorized execution, and distributed architecture—offers the best combination of storage capacity, write performance, real‑time analysis, and query speed for billions of records.

ClickHouseColumnar DatabaseMergeTree
0 likes · 31 min read
Why ClickHouse Is the Ideal Choice for Massive Data Storage and Real‑Time Analytics
dbaplus Community
dbaplus Community
Jul 17, 2023 · Big Data

How Bilibili Built Billions 3.0: A Low‑Cost, Scalable Log Platform with ClickHouse, Iceberg, and Trino

This article details Bilibili's evolution from the ClickHouse‑based Billions 2.0 log system to the Billions 3.0 architecture, explaining how they reduced storage costs, improved troubleshooting, adopted a lake‑house design with Iceberg on HDFS, leveraged ClickHouse for acceleration, and integrated Trino as the unified query engine.

ClickHouseiceberglog platform
0 likes · 37 min read
How Bilibili Built Billions 3.0: A Low‑Cost, Scalable Log Platform with ClickHouse, Iceberg, and Trino
Architect
Architect
Jul 17, 2023 · Databases

Performance Comparison of Elasticsearch and ClickHouse for Log Search and Analytics

This article compares Elasticsearch and ClickHouse by describing their architectures, presenting Docker‑based test stacks, showing code snippets for deployment, data ingestion, and queries, and reporting performance results that demonstrate ClickHouse generally outperforms Elasticsearch in log‑analytics scenarios.

ClickHouseDockerElasticsearch
0 likes · 12 min read
Performance Comparison of Elasticsearch and ClickHouse for Log Search and Analytics
Didi Tech
Didi Tech
Jul 5, 2023 · Databases

Performance Optimization of ClickHouse: Identifying and Fixing High CPU Usage in BgMoveProcPool Threads

By adding a guard that skips the costly part‑scan in MergeTreePartsMover::selectPartsForMove when disk usage is below the threshold and no MoveTTL is set, Didi reduced BgMoveProcPool thread CPU consumption from about 30 % to under 4 %, halving overall node CPU load and improving ClickHouse’s PB‑scale performance.

BgMoveProcPoolCPUClickHouse
0 likes · 10 min read
Performance Optimization of ClickHouse: Identifying and Fixing High CPU Usage in BgMoveProcPool Threads
NetEase Cloud Music Tech Team
NetEase Cloud Music Tech Team
Jun 29, 2023 · Cloud Native

KubeCost: Kubernetes-Based Resource Cost Analysis and Allocation System

KubeCost, developed by NetEase Cloud Music, is a low‑intrusion, scalable Kubernetes cost analysis system that allocates resource expenses using peak‑or‑usage billing models, supports hybrid‑multi‑cloud pricing, aggregates per‑pod CPU/memory/GPU costs, and stores data efficiently in ClickHouse for reliable, business‑oriented financial insight.

ClickHouseCloud Cost ManagementCost Optimization
0 likes · 10 min read
KubeCost: Kubernetes-Based Resource Cost Analysis and Allocation System
Bilibili Tech
Bilibili Tech
Jun 20, 2023 · Big Data

Design and Evolution of Bilibili's Billions 3.0 Log Platform: A Lakehouse Architecture with ClickHouse, Iceberg, and Trino

Bilibili evolved its log platform from ClickHouse‑based Billions 2.0 to Billions 3.0 lakehouse using Iceberg, HDFS, Trino, retaining ClickHouse for acceleration; this reduces storage cost by over 20%, improves observability, solves the compute‑storage mismatch, adds flexible indexing, and supports complex ETL while staying open‑source.

ClickHouseIndexingLakehouse
0 likes · 36 min read
Design and Evolution of Bilibili's Billions 3.0 Log Platform: A Lakehouse Architecture with ClickHouse, Iceberg, and Trino
Didi Tech
Didi Tech
Jun 14, 2023 · Big Data

Real-Time Data Development Practices and Component Selection at Didi

Didi’s unified real‑time data stack outlines best‑practice component choices for four key scenarios—metric monitoring, BI analysis, online services, and feature/tag systems—detailing pipelines from source to sink, resource‑usage guidelines, and a one‑stop development platform to build stable, high‑performance streaming solutions.

ClickHouseDruidFlink
0 likes · 17 min read
Real-Time Data Development Practices and Component Selection at Didi
JD Cloud Developers
JD Cloud Developers
May 30, 2023 · Big Data

ClickHouse & Flink: Choosing Engines, Tuning Queries, and Scaling Concurrency

This article details how JDQ, Flink, and ClickHouse were integrated to replace Elasticsearch for real‑time reporting, covering table‑engine selection, Flink sink implementation, performance bottlenecks, CPU hot‑spots, query optimization techniques, and strategies for handling high concurrency while ensuring data consistency and system stability.

ClickHouseFlinkSQL optimization
0 likes · 46 min read
ClickHouse & Flink: Choosing Engines, Tuning Queries, and Scaling Concurrency
DataFunSummit
DataFunSummit
May 30, 2023 · Big Data

DataFunCon Conference – OLAP, StarRocks, ClickHouse, and ByteHouse Technical Sessions

The DataFunCon conference showcases leading experts from Ctrip, Didi, Bilibili, and ByteDance presenting next‑generation OLAP technologies such as StarRocks, ClickHouse, and ByteHouse, covering architecture, materialized views, ELT practices, and performance optimization to guide practitioners in big‑data platform selection and implementation.

ByteHouseClickHouseOLAP
0 likes · 7 min read
DataFunCon Conference – OLAP, StarRocks, ClickHouse, and ByteHouse Technical Sessions
Java Backend Technology
Java Backend Technology
May 30, 2023 · Databases

Why count(*) Slows Down MySQL and How to Optimize It

This article explains why MySQL count(*) can become a performance bottleneck on InnoDB tables, compares different count() variants, and presents practical optimization techniques such as Redis caching, second‑level caches, parallel execution, reducing joins, and offloading analytics to ClickHouse.

COUNTClickHouseInnoDB
0 likes · 10 min read
Why count(*) Slows Down MySQL and How to Optimize It
ByteDance Data Platform
ByteDance Data Platform
May 29, 2023 · Databases

Which Open‑Source OLAP Engine Wins the TPC‑DS Benchmark? A Deep Performance Comparison

Using the TPC‑DS benchmark’s 99 queries on a 1 TB dataset, this study evaluates the performance of four open‑source OLAP engines—ClickHouse, Doris, Presto, and ByConity—across basic, join, aggregation, subquery, and window‑function scenarios, revealing ByConity’s superior speed and the limitations of ClickHouse.

ByConityClickHouseDoris
0 likes · 12 min read
Which Open‑Source OLAP Engine Wins the TPC‑DS Benchmark? A Deep Performance Comparison
WeChat Backend Team
WeChat Backend Team
May 24, 2023 · Databases

Boost ClickHouse Bitmap Queries 10x with BitBooster: Techniques & Results

This article explains how the BitBooster suite accelerates ClickHouse bitmap (BitMap) queries by up to tenfold, covering background, performance bottlenecks, single‑node and read optimizations, layout and instruction‑set enhancements, encoding dictionaries, multi‑node scaling, and real‑world benchmark results.

BitmapClickHouseOptimization
0 likes · 23 min read
Boost ClickHouse Bitmap Queries 10x with BitBooster: Techniques & Results
ITPUB
ITPUB
May 15, 2023 · Big Data

Why ClickHouse Outperforms Elasticsearch in Log Analytics: A Practical Comparison

This article compares Elasticsearch and ClickHouse for log analytics by detailing their architectures, setting up Docker‑Compose stacks, ingesting synthetic syslog data with Vector, running equivalent queries, and measuring performance, revealing ClickHouse’s superior speed in most scenarios.

ClickHouseDockerElasticsearch
0 likes · 13 min read
Why ClickHouse Outperforms Elasticsearch in Log Analytics: A Practical Comparison
Efficient Ops
Efficient Ops
May 7, 2023 · Databases

Elasticsearch vs ClickHouse: Performance Comparison for Log Analytics

This article compares Elasticsearch and ClickHouse as log‑analytics solutions, detailing their architectures, node roles, data ingestion pipelines, query capabilities, and benchmark results, ultimately showing ClickHouse’s superior performance in most tested scenarios.

ClickHouseDockerElasticsearch
0 likes · 13 min read
Elasticsearch vs ClickHouse: Performance Comparison for Log Analytics
DataFunTalk
DataFunTalk
May 5, 2023 · Big Data

NetEase Cloud Music Real-Time Data Warehouse Architecture and Low-Code Platform Practices

This article presents NetEase Cloud Music's real-time data warehouse architecture, covering its streaming and batch scenarios, layered design (ODS, CDM, ADS), technology stack choices, consistency mechanisms, the FastX low-code platform, and future development plans, offering a comprehensive technical overview for data engineers and architects.

ClickHouseFlinkKafka
0 likes · 18 min read
NetEase Cloud Music Real-Time Data Warehouse Architecture and Low-Code Platform Practices
Top Architect
Top Architect
Apr 26, 2023 · Databases

Comparative Performance and Feature Analysis of Elasticsearch vs ClickHouse

This article presents a practical comparison between Elasticsearch and ClickHouse, detailing their architectures, Docker‑Compose deployment, data ingestion pipelines, a series of representative queries, and benchmark results that show ClickHouse generally outperforms Elasticsearch in basic search and aggregation scenarios.

ClickHouseDocker ComposeElasticsearch
0 likes · 14 min read
Comparative Performance and Feature Analysis of Elasticsearch vs ClickHouse
Architect
Architect
Apr 23, 2023 · Big Data

Performance Comparison of Elasticsearch and ClickHouse for Log Analytics

This article compares Elasticsearch and ClickHouse by describing their architectures, demonstrating a Docker‑compose test environment, executing equivalent queries via both systems, and presenting performance results that show ClickHouse generally outperforms Elasticsearch in basic search and aggregation scenarios for log data.

ClickHouseElasticsearchPerformance
0 likes · 12 min read
Performance Comparison of Elasticsearch and ClickHouse for Log Analytics
JD Cloud Developers
JD Cloud Developers
Apr 20, 2023 · Operations

How to Build a Cost‑Effective, High‑Throughput Log Collection System with ClickHouse

This article examines the challenges of scaling log storage and retrieval for high‑traffic services, analyzes the cost and performance limits of traditional ELK‑based pipelines, and presents a streamlined, UDP‑driven architecture using ClickHouse that dramatically reduces hardware expenses while handling hundreds of gigabytes per second.

ClickHouseCost OptimizationLog Collection
0 likes · 16 min read
How to Build a Cost‑Effective, High‑Throughput Log Collection System with ClickHouse
Code Ape Tech Column
Code Ape Tech Column
Apr 19, 2023 · Databases

Comparative Analysis of Elasticsearch and ClickHouse: Architecture, Query Performance, and Practical Benchmarks

This article compares Elasticsearch and ClickHouse by outlining their architectures, detailing deployment configurations, presenting benchmark queries and performance results, and concluding that ClickHouse generally outperforms Elasticsearch in many basic search and aggregation scenarios, while also noting each system's strengths and limitations.

ClickHouseElasticsearchbig data
0 likes · 13 min read
Comparative Analysis of Elasticsearch and ClickHouse: Architecture, Query Performance, and Practical Benchmarks
Selected Java Interview Questions
Selected Java Interview Questions
Mar 12, 2023 · Big Data

Design and Optimization of Querying 100K Records from Tens of Millions of Data Using ClickHouse, Elasticsearch, HBase, and RediSearch

This article presents a comprehensive design and performance‑optimization study for extracting up to 100 000 records from a pool of tens of millions, comparing multithreaded ClickHouse pagination, Elasticsearch scroll‑scan, ES + HBase, and RediSearch + RedisJSON solutions, and provides practical recommendations based on measured latency and throughput.

ClickHouseHBaseQuery Optimization
0 likes · 11 min read
Design and Optimization of Querying 100K Records from Tens of Millions of Data Using ClickHouse, Elasticsearch, HBase, and RediSearch