Tagged articles
MPP
59 articles · Page 1 of 1
Fifteen years after an internship at Nanda Tongyong, the author returns as an external expert to reflect on personal growth, the evolution of the GBase product line—from the early MPP database to AI‑native HTAP solutions—highlighting technical innovations, benchmark results, and the broader impact on the domestic database ecosystem.
AICloud Data WarehouseGBase
0 likes · 17 min read

The article breaks down a big‑data project into four essential phases—data ingestion, preprocessing, storage, and analysis—explaining common pitfalls, practical tool choices, and why aligning technology with business needs is crucial for success.
Data IngestionData StorageHadoop
0 likes · 10 min read

The article outlines the full lifecycle of data warehouse modeling—from business requirements and system architecture considerations through conceptual, logical, and physical design, illustrated with a procurement bidding case study and a comparison of MPP and Hadoop solutions.
ETLMPPbig data
0 likes · 13 min read

This article explains ClickHouse’s core architecture—including its storage‑compute integration, MPP parallelism, columnar storage, vectorized execution, data pre‑sorting, table engines, sparse and auxiliary indexes, and the two‑stage aggregation pipeline—then walks through the exact GROUP BY execution flow for both local and distributed tables, illustrating each step with diagrams, SQL demos, and code snippets.
ClickHouseDistributed QueryHashMap
0 likes · 29 min read

BaikalDB, Baidu's distributed storage system, evolves from an OLTP‑focused engine to a hybrid HTAP architecture by introducing a vectorized query engine and a massively parallel processing (MPP) layer, addressing compute and resource bottlenecks for large‑scale analytical workloads while preserving transactional guarantees.
BaikalDBDatabase ArchitectureHTAP
0 likes · 18 min read

Tencent Cloud's TDSQL shattered world records in both TPC‑DS (OLAP) and TPC‑C (OLTP) benchmarks, achieving a 7260 M QphDS score at a cost of 37.52 CNY/kQphDS, and the article explains the three self‑developed technologies—MPP execution, parallel execution framework, and columnar‑vectorized engine—that made this performance possible.
Database PerformanceMPPOLAP
0 likes · 7 min read

Shopee’s deployment of StarRocks across DataService, DataGo, and DataStudio demonstrates that its vectorized engine, cost‑based optimizer, and materialized‑view caching can query Hive, Iceberg, Delta Lake and Hudi up to 20,000× faster than Presto, cutting CPU usage and delivering consistently lower latency for complex analytics.
HiveMPPMaterialized View
0 likes · 11 min read

This article compares ClickHouse and MySQL by examining benchmark results, MPP architecture, columnar storage, compression techniques, vectorized execution, and index designs, showing why ClickHouse delivers dramatically higher query performance on massive data sets.
ClickHouseMPPMySQL
0 likes · 8 min read


ITPUBSep 11, 2024 · Big Data
The article examines the concepts of storage‑compute separation and the lake‑warehouse (lakehouse) model, tracing their evolution from physical Hadoop clusters to containerized compute and object storage, and argues that true separation requires MPP systems to adopt open standards, effectively merging lake and warehouse architectures.
HadoopMPPStorage-Compute Separation
0 likes · 7 min read

Pinterest migrated its Partner Insights analytics from Druid to StarRocks, achieving a 50% reduction in p90 latency, a six‑fold cost‑performance improvement, and simplified data ingestion, illustrating the benefits of a modern MPP database for real‑time ad analytics.
AnalyticsMPPPinterest
0 likes · 6 min read

StarRocks, an open‑source, high‑performance MPP analytical database under the Linux Foundation, offers vectorized engines, CBO optimizer, materialized views, and storage‑compute separation, integrates with BI tools and data lakes, and demonstrates superior query speed in benchmark tests against ClickHouse, Druid, and Trino.
MPPStarRocksanalytical database
0 likes · 10 min read

LibraDB, the column‑store replica of TDSQL MySQL, has evolved its execution engine from a simple scatter‑gather model to a vectorized SMP pipeline that integrates MPP parallelism, asynchronous I/O, SIMD‑accelerated aggregation and join operators, work‑stealing, and runtime filters, thereby fully exploiting CPU, memory, network and disk resources for both OLTP and analytical queries.
Execution EngineHash JoinMPP
0 likes · 22 min read

This article explains ClickHouse’s columnar, MPP‑based design, block compression, LSM pre‑sorting, sparse and skip‑list indexing, and vectorized execution, while also discussing its high‑frequency write challenges, concurrency limits, and production‑grade issues such as Zookeeper load and resource management.
ClickHouseColumnar DatabaseLSM
0 likes · 11 min read

This article provides a detailed technical comparison between StarRocks and Trino, covering their shared MPP architecture, cost‑based optimizer, pipeline execution, ANSI SQL support, differences in vectorized execution, materialized view capabilities, caching systems, data source connectors, benchmark results, high‑availability designs, join algorithms, and real‑world user case studies.
CacheMPPStarRocks
0 likes · 20 min read

The article presents Turing Data Analysis (TDA), a self‑service BI platform that replaces fragile traditional pipelines with a unified DWD‑based data model, drag‑and‑drop analytics, multi‑engine query optimization and caching, delivering sub‑10‑second queries on billions of rows, fine‑grained permissions, and rapid dashboard creation, while reporting significant usage growth and outlining AI‑driven future enhancements.
BIMPPSelf-Service Analytics
0 likes · 15 min read

Apache Doris is an open‑source MPP OLAP database that combines a minimalist architecture, ease of use, rich features such as partition‑bucket pruning, materialized views, and bitmap indexes, and provides high‑performance, scalable, and reliable data warehousing for big‑data analytics.
Apache DorisMPPOLAP
0 likes · 19 min read

This article presents an in‑depth overview of the YouShu BI product, outlines the high‑concurrency performance challenges faced by enterprise BI, and details the multi‑layer performance architecture—including front‑end, back‑end, data engine, and data source layers—along with smart caching, MPP acceleration, materialized views, and the Data Doctor operations that together ensure low‑latency, reliable analytics for large‑scale users.
BIMPPSmart Caching
0 likes · 16 min read

This article provides a comprehensive overview of Apache Doris, detailing its origins, MPP‑based analytical capabilities, data‑lake integration techniques, recent architectural enhancements, performance optimizations, community growth, and upcoming development plans, while also addressing common user questions.
Apache DorisMPPSQL
0 likes · 20 min read


ITPUBMar 14, 2023 · Big Data
This article explains why disaster‑recovery and active‑active architectures are essential for OLAP MPP data‑warehouse clusters, outlines the specific RPO/RTO requirements for batch and real‑time workloads, and compares several data‑synchronization techniques and active‑active deployment models with their advantages and drawbacks.
Active-ActiveHigh AvailabilityMPP
0 likes · 12 min read

MPP (Massive Parallel Processing) databases, designed for large‑scale analytical workloads, use distributed, shared‑nothing architectures with multiple control and compute nodes, offering high scalability, diverse data‑sharding strategies, and powerful SQL compatibility, as illustrated by vendors like Teradata, Vertica, Greenplum, and emerging open‑source solutions.
Data ShardingGreenplumMPP
0 likes · 15 min read


ITPUBFeb 13, 2023 · Databases
Based on a DTCC2022 presentation, this article explains Apache Doris's high‑performance MPP architecture, its cloud‑native extensions in SelectDB, and how they solve large‑scale log storage and analysis with superior write throughput, storage efficiency, and interactive query speed.
Apache DorisMPPSelectDB
0 likes · 11 min read


ITPUBJan 3, 2023 · Databases
The article details the design, core features, and real‑world performance of the DragonF MPP DB, a cloud‑native, compute‑storage‑separated database that overcomes traditional MPP limitations, supports millions of daily jobs, and outlines its future roadmap for ultra‑large‑scale data platforms.
MPPbig datacloud native
0 likes · 11 min read

This article presents a comprehensive overview of StarRocks within the modern data stack, covering the evolution of MPP architectures, typical industry use cases, core features, performance benchmark comparisons, real‑time data‑warehouse construction methods, CDP and lakehouse analytics, as well as short‑term roadmap plans and a brief Q&A.
CDPMPPStarRocks
0 likes · 11 min read

This article explains the core concepts, architecture, and code logic of StarRocks' Pipeline execution framework, covering ExecPlan, PlanFragment, Fragment Instance, ExecNode, SourceOperator, SinkOperator, PipelineDriver scheduling, asynchronous handling of blocking operations, and the roles of FE and BE in MPP scheduling.
Execution EngineMPPPipeline
0 likes · 13 min read

The article presents a comprehensive technical overview of MercsDB—formerly HermesDB—including its background, storage and indexing designs, native and Presto computation engines, vectorization optimizations, benchmark results, real‑world applications, and future development plans.
MPPMercsDBbig data
0 likes · 20 min read

This article details Xiaomi's three‑year journey of adopting Apache Doris across dozens of internal services, describing the transition from a Spark‑SQL‑based Lambda architecture to a unified MPP database, performance benchmarks, data ingestion pipelines, compaction tuning, two‑phase commit, single‑replica writes, monitoring, and community contributions.
Apache DorisMPPReal‑Time Analytics
0 likes · 19 min read

This article explains ClickHouse’s MPP columnar design, complete DBMS capabilities, columnar storage, vectorized execution, multi‑master architecture, real‑time queries, sharding, and performance‑focused hardware and algorithm choices that together deliver its superior speed.
ClickHouseColumnar DatabaseDatabase Architecture
0 likes · 17 min read

This article introduces Apache Doris 1.0, detailing its simplified architecture, high‑concurrency support, MPP execution engine, vectorized engine, memory‑controlled stability, multi‑source integration, upcoming lake‑house unification, storage‑compute separation, real‑time ingestion, and community growth.
Apache DorisMPPVectorized Engine
0 likes · 18 min read

This article introduces Apache Doris, a high‑performance, real‑time analytical database built on an MPP architecture, covering its simple FE/BE design, three data‑model types (Aggregate, Unique, Duplicate), partitioning and bucketing strategies, rollup tables, and the limitations and best practices of row‑level updates.
Apache DorisData ModelingMPP
0 likes · 28 min read

This article explains the three main commercial server architectures—SMP, NUMA, and MPP—detailing their structures, performance characteristics, scalability limits, and suitability for OLTP versus data‑warehouse workloads, while also covering practical considerations such as virtualization and real‑world examples.
MPPNUMAParallel Processing
0 likes · 16 min read

The Qunhe Technology data team replaced their legacy Hadoop and Presto clusters with a StarRocks MPP database, achieving up to three times faster queries, supporting billion‑row tables and sub‑second latency for both real‑time and analytical workloads on a daily 1TB data influx.
MPPOLAPStarRocks
0 likes · 10 min read

This article provides a comprehensive overview of Doris's architecture—including its FE and BE components, metadata management, data organization, execution planning—and details its major features such as adaptive join aggregation, vectorized execution, materialized views, and Elasticsearch integration, supplemented with example DDL and query code.
Database ArchitectureDorisElasticsearch
0 likes · 7 min read

This article presents the cloud‑native redesign of ClickHouse, covering its current technical limitations in storage and computation, the proposed storage‑compute separation with DDL task management, multi‑replica and CommitLog mechanisms, and a new MPP query layer to meet future data‑warehouse demands such as real‑time analytics, flexibility, high throughput, low cost, and support for semi‑structured data.
ClickHouseDistributed QueryMPP
0 likes · 15 min read

This article explains how StarRocks converts a SQL statement into an optimal distributed physical execution plan, schedules the plan across compute nodes, and runs it using MPP, pipeline parallelism, and vectorized execution to achieve near‑linear performance scaling.
CBO OptimizerDistributed DatabaseMPP
0 likes · 15 min read

This article explains StarRocks' streamlined architecture, cost‑based optimizer, massively parallel processing and vectorized engine, and how they enable high‑performance queries over data stored in Hive, Iceberg, Hudi and other lake formats, backed by benchmark results and future roadmap details.
CBOMPPStarRocks
0 likes · 19 min read

This article explains how StarRocks manages petabyte‑level user behavior logs, ads and orders through a shared‑nothing architecture, tablet‑based data distribution, MPP compute, high‑availability metadata, real‑time mini‑batch ingestion, and online schema changes, enabling 24/7 analytical services for diverse internet companies.
MPPOnline Schema ChangeStarRocks
0 likes · 11 min read

This article explains TiDB's HTAP architecture, covering industry challenges, the row‑store TiKV and column‑store TiFlash design, MPP integration in TiDB 5.0, and a range of real‑time use cases such as dashboards, reporting, and data‑warehouse pipelines.
Database ArchitectureHTAPMPP
0 likes · 16 min read

The article details how China Construction Bank's fintech arm designed, deployed, and operated a cloud‑native, three‑layer MPP data warehouse spanning 16,000 servers, covering architectural choices, performance gains, operational automation, and high‑availability strategies for ultra‑large scale workloads.
Database ArchitectureMPPcloud native
0 likes · 10 min read

This article explains the design and implementation of a cloud‑native MPP query layer for ClickHouse, detailing its architecture, core features, execution flow, performance advantages, SQL compatibility, and future development plans to create a high‑performance, multi‑source OLAP data platform.
ClickHouseMPPSQL Compatibility
0 likes · 13 min read


ITPUBSep 13, 2021 · Big Data
A team of engineers at MBI debates the merits of MapReduce, MPP, and Hive for their KeepS global data‑warehouse, discussing technical differences, scalability, concurrency, and the feasibility of mixed batch engines while navigating budget and operational constraints.
Cluster ComputingGrid ComputingHive
0 likes · 20 min read

This article explains the various classifications of OLAP data warehouses—including MOLAP, ROLAP, HOLAP, and HTAP—based on data volume and modeling, reviews common open‑source ROLAP products, and details performance‑boosting techniques such as MPP architecture, cost‑based optimization, vectorized execution, and storage optimizations.
Cost-Based OptimizationMPPOLAP
0 likes · 27 min read

This article details how KuJiaLe's big data team replaced their legacy ADB and Presto clusters with a DorisDB MPP database, achieving sub‑second query latency, unified real‑time and offline analytics, simplified ETL pipelines, and significant cost savings while supporting billion‑row tables and high‑QPS workloads.
DorisDBETLMPP
0 likes · 9 min read

The article examines the challenges big data brings to traditional technologies and surveys major OLAP solutions—MPP, batch processing, and pre‑computation—including Greenplum, Druid, Kylin, and Hadoop‑based engines, then outlines Youku’s specific use‑case selections for real‑time APIs, BI reporting, and ad‑hoc analysis.
MPPOLAPPrecomputation
0 likes · 13 min read

This article explores the challenges big data brings to traditional data technologies and reviews various OLAP solutions—including MPP, batch processing, pre‑computation, and Hadoop‑based engines—while detailing Youku’s specific business scenarios and how different OLAP engines are selected to meet performance, scalability, and real‑time analysis requirements.
AnalyticsMPPOLAP
0 likes · 14 min read

Tencent’s game marketing analysis system “EAS” evolved from inefficient TDW HiveSQL jobs and file‑heavy real‑time pipelines to a scalable ClickHouse‑based deduplication service that processes hundreds of thousands of daily activity counts in sub‑second time, offering fast, reliable, and maintainable participant deduplication for massive marketing campaigns.
ClickHouseLevelDBMPP
0 likes · 10 min read

This article examines Meituan’s data warehouse evolution, detailing the limitations of MOLAP with Kylin, the adoption of Doris‑driven ROLAP using MPP technology, and the practical optimizations—such as join predicate pushdown, concurrent execution, colocate join, and bitmap aggregation—that improve real‑time analytics and reduce costs.
DorisMOLAPMPP
0 likes · 19 min read

This article provides a comprehensive overview of OLAP data warehouses, covering classification by data volume and modeling, detailed explanations of MOLAP, ROLAP, HOLAP and HTAP, common open‑source implementations, and a deep dive into performance‑boosting techniques such as MPP architectures, cost‑based optimization, vectorized execution, dynamic code generation, storage compression, runtime filters and resource management.
Cost-Based OptimizationDynamic Code GenerationMPP
0 likes · 25 min read

At a Tsinghua University forum, Hulu presented a comprehensive overview of its big‑data solutions for advertising and streaming, covering challenges of massive, complex data, the limits of MySQL, and advanced techniques using HBase, Protobuf, Redis batch pipelines, and its own MPP engine Nesto for high‑performance, scalable analytics.
AdvertisingHBaseMPP
0 likes · 6 min read

Tencent’s iData analysis center selected Spark as its new computing platform because, unlike ElasticSearch, TiDB, and other MPP solutions, Spark offers iterative processing, shuffle support, robust SQL and DAG scheduling, and flexible SMP‑style data exchange, enabling efficient OLAP on billions of game‑user records.
MPPOLAPSMP
0 likes · 13 min read

This article introduces Apache Doris, a massively parallel processing (MPP) distributed database, covering its background, core architecture, key technologies such as reliability, maintenance, MySQL compatibility, materialized views, and real‑world applications at Baidu and beyond.
Distributed DatabaseDorisKafka Integration
0 likes · 12 min read

This article examines why traditional relational databases like MySQL struggle with analytical workloads, compares ROLAP and MOLAP approaches, explains Spark’s architecture and its advantages for OLAP, and details how Alibaba Cloud’s DRDS HTAP leverages a Spark‑based engine to deliver real‑time distributed query processing.
HTAPMPPOLAP
0 likes · 11 min read

This article examines the evolution and classification of distributed databases for OLAP workloads, comparing traditional RDBMS, MPP solutions such as Teradata and Greenplum, Hadoop‑based ecosystems, and newer architectures like ClickHouse and Palo, while highlighting their architectural traits, strengths, and limitations.
ClickHouseHadoopMPP
0 likes · 17 min read

This article explains the concepts of MPI, OpenMPI, and OpenMP, compares three major server architectures—SMP, NUMA, and MPP—and discusses their performance characteristics, scalability limits, and typical application scenarios in high‑performance computing.
HPCMPIMPP
0 likes · 13 min read

This article explains Greenplum's MPP architecture, master‑segment design, high‑availability, interconnect network, rich management tools, parallel query planning, data loading techniques, and additional capabilities such as LDAP authentication and resource queues, demonstrating why it is a strong next‑generation big‑data query engine.
GreenplumMPPOperations
0 likes · 15 min read

This article explains what Greenplum is, its MPP shared‑nothing architecture, and provides concrete table‑design principles, distribution‑column strategies, indexing guidance, vacuum and table‑rebuilding techniques, as well as SQL, join, insert, update/delete, and resource‑queue optimizations for better performance.
Distributed TablesGreenplumMPP
0 likes · 9 min read

Greenplum is an open‑source, massively parallel processing (MPP) database built on PostgreSQL, offering ANSI‑SQL compliance, distributed ACID transactions, linear scalability, polymorphic storage, advanced optimizers, and extensive ecosystem integrations, making it suitable for large‑scale data warehousing, analytics, and big‑data workloads.
GreenplumMPPPolymorphic Storage
0 likes · 15 min read


ITPUBJun 29, 2016 · Big Data
The article explains how traditional OLTP systems cannot satisfy modern big‑data analytics needs and compares OLAP, Hadoop, and MPP architectures, highlighting their data processing models, scalability, cloud‑based managed services, and practical recommendations for building effective data warehouses.
HadoopMPPOLAP
0 likes · 21 min read

This article delivers a comprehensive guide to Greenplum deployment, covering early architecture planning, data‑model design, daily maintenance best practices, system‑table management, diagnostic tools like gpcheckcat, and detailed troubleshooting techniques for persistent tables and other common issues.
Data ModelingGreenplumMPP
0 likes · 13 min read

Since the 1970s databases have become essential middleware, and modern large‑scale parallel databases, designed for extreme parallelism on clustered hardware, face trade‑offs in performance, scalability, and fault tolerance, prompting a shift toward cloud‑native, micro‑service architectures and new hardware such as SSDs and memory‑centric designs.
In-MemoryMPPSSD
0 likes · 23 min read
