The article examines the challenges big data brings to traditional technologies and surveys major OLAP solutions—MPP, batch processing, and pre‑computation—including Greenplum, Druid, Kylin, and Hadoop‑based engines, then outlines Youku’s specific use‑case selections for real‑time APIs, BI reporting, and ad‑hoc analysis.
Meituan’s sales system “Qingtian” boosted OLAP performance by migrating Apache Kylin’s build engine from MapReduce to Spark, consolidating Hive files, refining dictionary creation, applying a By‑layer algorithm, and bulk‑loading cuboid files to HBase, cutting resource consumption and halving build time, ultimately reaching a 100 % SLA.
This article details the three‑stage evolution of BEIKE's OLAP platform—from the early Hive‑to‑MySQL phase, through a Kylin‑based architecture, to a flexible multi‑engine design—explaining metric modeling, engine selection, performance trade‑offs, and future roadmap for supporting Druid, ClickHouse, Doris and real‑time analytics.
This article presents Sohu Intelligent Media's data warehouse construction practice, covering fundamental concepts, batch and real‑time processing, OLAP theory, multidimensional modeling, workflow management, data quality, metadata lineage, and security, with a focus on Apache Doris and a Lambda‑style architecture.
This article examines the data models and index structures of the three leading OLAP engines—Kylin, Druid, and ClickHouse—offering technical explanations, code examples, and a concise comparison to help readers understand their strengths, limitations, and suitable use cases.
This article explores the challenges big data brings to traditional data technologies and reviews various OLAP solutions—including MPP, batch processing, pre‑computation, and Hadoop‑based engines—while detailing Youku’s specific business scenarios and how different OLAP engines are selected to meet performance, scalability, and real‑time analysis requirements.
This article introduces ClickHouse, compares column‑store and row‑store databases, shows how migrating a 50‑million‑row MySQL table to ClickHouse reduced query time from minutes to under one second, and shares practical installation, migration, performance testing, and synchronization tips.
This article introduces ClickHouse, compares column‑ and row‑oriented storage, explains a real‑world migration from MySQL to ClickHouse that reduced a 3‑minute query to under one second, details installation, migration methods, performance results, synchronization options, and common pitfalls.
This article details JD.com's transition from a custom ad‑reporting system to Apache Doris, describing the background, challenges with the legacy platform, selection criteria, implementation of data import, pre‑aggregation, on‑site computation, and the resulting performance and operational benefits during regular operation and major sales events.
This article details how Beike's engineering team scaled Apache Kylin to handle tens of millions of daily queries by optimizing HBase configurations, reducing region count, improving data locality, addressing IO and JVM GC bottlenecks, and implementing comprehensive slow‑query detection and active‑defense mechanisms.
The article details Xiaomi's growth analytics platform evolution from a Lambda architecture using SparkSQL, Kudu, and HDFS to a streamlined MPP solution with Apache Doris, covering performance gains, real‑time data ingestion, query tuning, and operational improvements for large‑scale analytics.
This article details ZTO Express's journey of adopting Apache Kylin for OLAP, comparing it with Presto, describing platform architecture, performance gains, integration with scheduling and monitoring systems, and the practical optimizations and future plans that enabled sub‑second query responses on massive daily data volumes.
This article explains what ClickHouse is, compares column‑store and row‑store databases, shows how to migrate large MySQL tables to ClickHouse, presents performance test results, discusses data synchronization methods, highlights why ClickHouse is fast, and shares common migration pitfalls.
ClickHouse is a PB‑scale, open‑source columnar OLAP database that uses a ZooKeeper‑coordinated sharded cluster, columnar storage, vectorized execution, advanced compression, data‑skipping indexes, and materialized views to deliver high‑performance interactive analytics, yet it requires manual shard management, lacks a mature MPP optimizer, and handles real‑time single‑row writes poorly.
This article presents a comprehensive overview of NetEase's use of Impala as an OLAP query engine, detailing its architectural advantages, performance benefits, enhancements such as management servers, metadata synchronization, high‑availability via Zookeeper, expanded storage support, and real‑world deployment cases in the "Mammoth" platform and NetEase Cloud Music.
This article describes Meituan's massive OLAP workloads, the specific challenges of data scale, complex schemas, and precise counting, explains how Apache Kylin was integrated using wide tables and bitmap deduplication, compares its performance and features with Presto, Druid and other engines, and outlines future improvements.
This article explains why real‑time data warehouses are needed, outlines their core principles, compares them with offline warehouses, describes typical use cases such as real‑time OLAP, dashboards, feature generation and monitoring, and provides a step‑by‑step guide to designing, implementing, and operating a Flink‑based streaming warehouse with Kafka, HBase, and metadata management.
This article explains how ClickHouse’s column‑oriented storage, vectorized execution engine, rich DBMS capabilities, flexible table engines, and carefully designed distributed architecture enable it to handle massive workloads with sub‑second query latency, making it a standout OLAP solution.
This article explains how to evaluate business architecture, data volume, data types, and team expertise to select the most suitable database—whether relational with strong ACID guarantees for OLTP, analytical OLAP systems, or various NoSQL options—while highlighting Huawei Cloud offerings for different scenarios.
This article introduces ClickHouse as an OLAP columnar database, explains the differences between row‑ and column‑oriented storage, details a real‑world migration from MySQL using a CREATE TABLE AS SELECT approach, presents performance comparisons, discusses synchronization methods, and shares common pitfalls and solutions.
This article systematically examines the core technologies of OLAP systems, covering storage models, columnar formats, indexing, distributed storage architectures, query execution steps, optimizer designs, and emerging trends such as real‑time analytics, HTAP, cloud‑native deployment, and hardware acceleration.
This article explains the classification of OLAP data warehouses by data volume and modeling approach, compares MOLAP, ROLAP, HOLAP and HTAP, reviews popular open‑source ROLAP systems, and details advanced performance‑boosting techniques such as MPP architectures, cost‑based optimization, vectorized execution, dynamic code generation, and runtime filtering.
This article explains why ClickHouse, the high‑performance columnar OLAP database, can return results on billions of rows within seconds, detailing its columnar storage, MergeTree engine, and how JD Cloud deploys and optimizes it on Kubernetes for scalability and reliability.
This article introduces ClickHouse, a column‑store OLAP database, explains its advantages over row‑store systems, guides installation on macOS, details migration strategies from MySQL, presents performance benchmarks showing 200‑fold speedups, and discusses synchronization methods, pitfalls, and practical tips.
This article explains the core differences between OLTP and OLAP, evaluates whether traditional OLTP databases like MySQL can handle analytical workloads, introduces benchmark queries, and provides a comprehensive overview of data‑warehouse concepts such as data sources, fact and dimension tables, multi‑dimensional modeling, and common cube operations.
Beike’s OLAP platform has progressed from a basic Hive‑MySQL batch pipeline to a Kylin‑based single‑engine solution, and now to a flexible multi‑engine architecture that uses a query‑engine layer to route metrics across Kylin, Druid, ClickHouse and Doris, dramatically cutting cube‑build times, supporting real‑time ingestion, and paving the way for further engine consolidation and automated performance routing.
This article introduces ClickHouse as a high‑performance columnar OLAP database, compares it with row‑based systems, details migration methods from MySQL, presents performance benchmarks showing dramatic speed gains, discusses data synchronization strategies, explains why ClickHouse is fast, and shares common pitfalls and solutions.
This article introduces ClickHouse as an OLAP columnar database, compares its storage model with row‑based databases, details migration from MySQL, presents performance benchmarks showing query speeds under one second for 50 million rows, and shares practical deployment tips and pitfalls.
Tencent’s game marketing analysis system “EAS” evolved from inefficient TDW HiveSQL jobs and file‑heavy real‑time pipelines to a scalable ClickHouse‑based deduplication service that processes hundreds of thousands of daily activity counts in sub‑second time, offering fast, reliable, and maintainable participant deduplication for massive marketing campaigns.
This article presents Alibaba's search‑recommendation real‑time data warehouse, describing its business background, typical use cases, key requirements, the evolution from architecture 1.0 to 2.0 with Flink and Hologres, best‑practice patterns such as row/column storage, stream‑batch integration, high‑concurrency updates, and future directions like real‑time joins and persistent dimension storage.
This article explains how Suning's big‑data center leverages ClickHouse’s columnar OLAP engine and a full‑chain monitoring platform to achieve real‑time query tracing, slow‑query analysis, cluster health checks, and resource‑level alerts across diverse business scenarios.
This article explains the fundamentals of Kylin Cube construction—including dimensions, measures, Cuboid generation, layer-by-layer and in‑memory building algorithms, storage mechanisms, and various optimization strategies such as derived dimensions, aggregation groups, row‑key design, and concurrency granularity—providing a comprehensive guide for big‑data OLAP practitioners.
This document explains the differences between OLTP and OLAP workloads, introduces the CSTORE columnar storage engine architecture, its core technologies, performance advantages, typical use cases, benchmark results, and future development plans for MySQL 8.0.
This article compares the data modeling and indexing techniques of three mainstream OLAP engines—Kylin, Druid, and ClickHouse—explaining their conversion processes, storage formats, query execution paths, and summarizing their strengths and limitations for aggregation and detail query scenarios.
This article introduces ClickHouse, an open‑source column‑oriented OLAP database, detailing its key features, appropriate use cases, installation steps, performance benchmark queries, and how it compares with other columnar storage solutions while highlighting its adoption by major internet companies.
The article details iQIYI’s migration from a Hive + MySQL OLAP stack to Apache Kylin, describing the system’s architecture, typical use cases, performance gains, independent HBase deployment, service platform for monitoring, and future plans such as automated cube building and clustering.
This article provides a detailed analysis of Apache Druid V1's column‑oriented storage format, covering dimension dictionaries, variable‑length encoded values, bitmap inverted indexes, array handling, and the physical metadata layout that enables sub‑second OLAP queries on massive datasets.
ClickHouse, the high‑performance columnar OLAP DBMS behind Yandex.Metrica, combines complete DBMS capabilities, column‑oriented storage with compression, vectorized execution, flexible table engines, multi‑master clustering, and extensive SQL support, offering fast online queries and scalable distributed processing for massive data workloads.
QQ Music replaced its slow Hive warehouse with a massive ClickHouse cluster, achieving sub‑second to ten‑second query latency on petabyte‑scale data, enabling real‑time analytics for non‑technical users, and following five operational best practices—ZooKeeper planning, idempotent writes, sensible partitions, read‑write separation, and localized joins—while leveraging Tencent Cloud’s managed ClickHouse service.
This article provides a comprehensive overview of Apache Kylin, covering its background, core OLAP concepts, technical architecture, installation steps, cube-building methods, real‑world enterprise deployments, and resources for further learning, illustrating how it enables sub‑second query performance on massive datasets.
This article explains how to build a systematic data metric framework—including indicator definitions, dimension management, data dictionary, and data warehouse architecture—to turn massive online‑school data into actionable business insights and support efficient decision‑making and BI visualization.
This article provides a comprehensive overview of OLAP systems, classifying them by data volume and modeling approach, comparing MOLAP, ROLAP, HOLAP and HTAP, reviewing popular open‑source products, and detailing architectural, query‑optimization, vectorization, storage and resource‑management techniques that affect analytical warehouse performance.
This article provides a comprehensive technical overview of ClickHouse, covering its complete DBMS capabilities, column‑oriented storage and compression, vectorized execution engine, relational SQL support, diverse table engines, multi‑master clustering, sharding, and the design philosophies that make it exceptionally fast for large‑scale analytical workloads.
This article provides an in‑depth analysis of Apache Druid V1's column‑oriented storage format, covering dimension structures, dictionaries, variable‑length integer encoding, inverted indexes, array handling, and how these components are used during query execution.
This article provides a comprehensive overview of OLAP data warehouses, covering classification by data volume and modeling, detailed explanations of MOLAP, ROLAP, HOLAP and HTAP, common open‑source implementations, and a deep dive into performance‑boosting techniques such as MPP architectures, cost‑based optimization, vectorized execution, dynamic code generation, storage compression, runtime filters and resource management.
Suning’s 618 shopping festival showcased a massive sales surge backed by its big‑data platform, which processed over 200 billion requests, handled 38.5 PB of daily data, and delivered 31.5 trillion computations, while Kafka and HBase sustained tens of millions of TPS to ensure a seamless consumer experience.
This article details AutoHome's use of Apache Kylin as its core OLAP engine, covering its development timeline, architecture, large‑scale cube deployment, performance optimizations, cluster upgrade experiences, and future directions such as real‑time OLAP and cloud‑native deployment.
This article details Autohome's use of Apache Kylin as its core OLAP engine, covering its architecture, large‑scale Cube deployment, real‑world business applications, a series of performance and operational optimizations, cluster upgrade experiences, and upcoming plans for real‑time OLAP and cloud‑native evolution.
QQ Music’s data team tackled massive PB‑scale, real‑time analytics challenges by migrating from Hive to a ClickHouse‑based OLAP platform integrated with Tencent Cloud EMR and Superset, achieving low‑latency, high‑availability data processing, self‑service visualization, and efficient read/write scaling for billions of daily events.
QQ Music’s new real‑time OLAP platform, built on ClickHouse, Superset and Tencent Cloud EMR, ingests petabyte‑scale streaming and batch data with SSD‑backed ZooKeeper, load‑balanced writes, optimized partitions and read/write separation, delivering second‑level query responses that are several times faster than Hive, Presto or SparkSQL and enabling self‑service BI for thousands of users.
ClickHouse is a column‑oriented, open‑source OLAP database developed by Yandex that offers high‑compression columnar storage, vectorized execution, and massive read/write throughput, making it ideal for large‑scale analytics while having specific usage scenarios and notable limitations such as lack of true transactions and secondary indexes.
This article provides a comprehensive overview of Druid, an open‑source, distributed, column‑store system for real‑time analytics, detailing its features, limitations, typical use cases, core components, external dependencies, data flow, and high‑availability mechanisms.
This article presents an in‑depth overview of Apache Flink's new OLAP engine, covering OLAP fundamentals, the three OLAP models, Flink's unified streaming‑batch‑OLAP architecture, performance optimizations, benchmark results, and future development directions.
The article chronicles Apache Kylin's evolution from an Apache incubator OLAP engine to a comprehensive analytical data warehouse, highlighting its five‑year growth, extensive enterprise adoption, core data‑warehouse features, and the community’s rebranding to better reflect its big‑data capabilities.
The article details iQIYI’s RAP platform, describing its real‑time analytics requirements, architectural evolution from RAP 1.x to 2.x, core design steps, integration of Druid, Spark, Flink, and KIS, and showcases business use cases such as membership monitoring, recommendation evaluation, and smart‑TV alerting.
The article details iQIYI's Real-Time Analysis Platform (RAP), describing its motivation, architecture evolution from RAP 1.x to 2.x, OLAP engine selection, product design workflow, integration of Druid KIS and Flink, enhanced diagnostics, and real-world applications in membership monitoring, recommendation evaluation, and smart TV alerting.
iQIYI’s Real‑Time Analysis Platform (RAP) combines Apache Druid with Spark/Flink to deliver minute‑level, low‑latency multidimensional analytics via a web wizard, supporting hundreds of streaming tasks and thousands of reports across membership, recommendation, and TV monitoring, while simplifying development and maintenance.
This article provides a thorough introduction to data warehousing, covering warehouse creation, the ETL process, OLAP/BI tools, data cube concepts, common OLAP operations, and the three main OLAP architectural models (MOLAP, ROLAP, HOLAP).
This article introduces Apache Kylin, details its deployment at Tongcheng Yilong, explains the design of a large‑scale trajectory model, and provides step‑by‑step optimization techniques—including cube dimension reduction, HBase rowkey tuning, build parameter tweaks, high‑cardinality handling, and query compression disabling—to achieve sub‑second OLAP queries on multi‑terabyte data.
At a Tsinghua University forum, Hulu presented a comprehensive overview of its big‑data solutions for advertising and streaming, covering challenges of massive, complex data, the limits of MySQL, and advanced techniques using HBase, Protobuf, Redis batch pipelines, and its own MPP engine Nesto for high‑performance, scalable analytics.
This article outlines ClickHouse’s columnar OLAP architecture, dual‑center design, storage and write stability strategies, performance testing results, and practical query and system optimizations for handling petabyte‑scale data with high throughput and low latency requirements.
Apache Kylin is an open‑source, Hadoop‑based OLAP engine that provides sub‑second, multi‑dimensional SQL queries on massive datasets, with features such as cube pre‑computation, real‑time analytics, and seamless BI tool integration, and its latest v2.6.4 release adds numerous fixes and improvements.
This article explains how Apache Kylin, an open‑source distributed analytics engine built on Hadoop/Spark, achieves sub‑second OLAP query performance through pre‑computed cubes, a layered cuboid generation algorithm, bitmap‑based distinct counting, dimension optimization techniques, and tight integration with HBase for storage and fast SQL querying.
This article describes Meituan’s large‑scale OLAP requirements, how Apache Kylin was integrated to meet them, the architectural solutions, performance benchmarks against other engines, and future work, providing practical insights for building stable, precise, and high‑performance analytics platforms.
This article reviews the concepts, criteria, and characteristics of major open‑source OLAP engines—including Hive, HAWQ, Spark SQL, Presto, Kylin, Impala, Druid, Greenplum, and ClickHouse—providing guidance on selecting the most suitable solution for various big‑data analytics scenarios.
This article surveys the recent surge in real‑time data warehouse construction, explains why low‑latency analytics are needed, and compares the architectures and technology stacks of Alibaba, Zhihu, Meituan, and NetEase, highlighting the role of open‑source OLAP engines such as Druid and ADB.
This article provides a comprehensive introduction to data warehouses, covering their definition, architectural layers, characteristics, modeling approaches such as Inmon and Kimball, fact and dimension table design, star and snowflake schemas, and best‑practice principles for building scalable, maintainable warehouse solutions in the big‑data ecosystem.
The article explains the two main types of data processing—OLTP for real‑time transactional workloads and OLAP for analytical data‑warehousing tasks—detailing their distinct characteristics, resource demands, typical features, major vendor products, and a five‑point comparison of their design and usage.
This article explains fundamental big‑data concepts by comparing offline (batch) and real‑time (stream) computing, distinguishing real‑time queries from ad‑hoc queries, clarifying OLTP versus OLAP workloads, and outlining the differences between row‑based and column‑based storage architectures.
This article provides an in‑depth overview of Apache Kylin’s pre‑computation architecture, data‑warehouse concepts, step‑by‑step cube creation from Hive tables, and advanced optimization techniques such as derived dimensions, aggregation groups, and HBase row‑key encoding to achieve sub‑second OLAP queries on massive datasets.
This article provides an in‑depth technical comparison of Apache Kylin and Apache Doris, covering their system architectures, aggregation and detail data models, storage engines, data import processes, query execution, deduplication, metadata handling, performance, high availability, maintainability, usability, schema‑change capabilities, features, and community ecosystems.
Tencent’s iData analysis center selected Spark as its new computing platform because, unlike ElasticSearch, TiDB, and other MPP solutions, Spark offers iterative processing, shuffle support, robust SQL and DAG scheduling, and flexible SMP‑style data exchange, enabling efficient OLAP on billions of game‑user records.
Tencent’s iData Analysis Center rebuilt its backend as TGMars, replacing a rigid row‑oriented bitmap store and single‑node MapReduce pipeline with a more extensible architecture that shards user behavior bitmaps, eliminates shuffle overhead, and adds columnar storage, iterative processing and SQL‑like capabilities using Spark to overcome scalability and flexibility limitations.
This article introduces Apache Doris, a massively parallel processing (MPP) distributed database, covering its background, core architecture, key technologies such as reliability, maintenance, MySQL compatibility, materialized views, and real‑world applications at Baidu and beyond.
This article explains core data‑warehouse concepts—including DB, DW, ODS, OLTP, OLAP, BI, and the differing meanings of DM—as well as their relationships, integration examples, and why OLAP cannot replace data mining, providing a concise reference for beginners in data analytics.
This article presents Meituan's real‑world experience with Apache Druid, detailing the platform’s current usage, the usability, security and stability challenges faced, the principles and architecture of Druid SQL, the enhancements made to schema inference, logging, query safety, and the custom security extensions implemented to achieve fine‑grained access control and SSO integration.
This article explains the concepts of OLTP, OLAP and HTAP, describes the DRDS HTAP architecture—including its engine and storage layers, Fireworks Spark‑based engine, optimizer stages, and streaming capabilities—and demonstrates cross‑database MPP queries and streaming joins while outlining suitable use cases and limitations.
This article presents an in‑depth overview of Hulu’s big‑data platform, detailing its multi‑layer architecture, the design and functionality of the Sophon OLAP cache layer, and how Impala is employed for high‑performance query processing and integration with cloud‑native engines.
YouZan adopted MetaMarket’s Druid OLAP platform—featuring millisecond‑level interactive queries, high availability, horizontal scalability, and rich SQL/API query types—by configuring simple ingestion tasks that automatically manage real‑time and batch data, tiered hot/cold storage, and monitoring, while still facing ingestion limits, lack of joins, and occasional latency spikes.
The article details JD's 2018 upgrade of its Business Intelligence platform, describing how unified offline OLAP with ClickHouse, Spark, and Scala, timeliness optimizations, and a React‑based visualization component library together improve data consistency, performance, and user experience for merchants.
This article introduces Kylin, a HBase‑based multidimensional analysis platform, explains its architecture and various performance optimizations—including multi‑tenant support, dimension dictionary handling, and cube size estimation—while showcasing a real‑world deployment and case study at 58.com.
HybridDB for MySQL delivers millisecond‑scale, multi‑dimensional filtering on billions of rows with hundreds of metrics by combining a high‑performance columnar engine, automatic composite indexes, and a fused MPP‑DAG pipeline, turning half‑day push preparation into seconds while supporting full SQL, spatial, and JSON data.
The article reviews the 2018 58 Group technology salon on big‑data OLAP, summarizing Xiaomi’s one‑stop OLAP architecture, 58.com’s challenges and solutions using Kylin, Druid, and UnionSQL, and the practical implementations and optimizations that illustrate modern OLAP practices.
This article examines why traditional relational databases like MySQL struggle with analytical workloads, compares ROLAP and MOLAP approaches, explains Spark’s architecture and its advantages for OLAP, and details how Alibaba Cloud’s DRDS HTAP leverages a Spark‑based engine to deliver real‑time distributed query processing.
The article reviews recent big-data trends—from Hadoop’s evolution and Spark’s in-memory advances to emerging storage like Ozone—while detailing data-warehouse models, query-optimizer techniques, and cloud-native architectures that integrate diverse data sources, enabling scalable, AI-ready analytics and modern data-lake capabilities.
The 2018 NDBC conference in Dalian gathered leading academia and industry experts to showcase cutting‑edge database technologies such as Alibaba's POLARDB, POLARDB X, SolarDB, Huawei Gauss, and AnalyticDB, while discussing cloud‑native challenges, large‑scale deployment, AI integration, and future collaborative research directions.
This article details how Beike Zhaofang leveraged the ClickHouse columnar OLAP database for large‑scale user behavior analysis, covering its architecture, key features, performance benchmarks against other engines, data ingestion pipelines, custom UDFs for funnel and retention metrics, deployment setup, and future enhancements.
This article summarizes a presentation by Ping An Insurance data engineers on Druid’s architecture, core concepts, node roles, tuning strategies, and real-world deployment for insurance analytics, illustrating how Druid enables sub‑second, high‑cardinality OLAP queries and supports both real‑time and batch processing.
This article examines the evolution and classification of distributed databases for OLAP workloads, comparing traditional RDBMS, MPP solutions such as Teradata and Greenplum, Hadoop‑based ecosystems, and newer architectures like ClickHouse and Palo, while highlighting their architectural traits, strengths, and limitations.
This article details Lianjia's journey from a Hadoop‑based 0.0 data platform to a sophisticated 2.0 architecture, describing the three‑layer design, OLAP engine choices, transparent compression techniques, operational challenges, and practical recommendations for building and maintaining large‑scale big data systems.
This article details the evolution of Lianjia's massive‑data platform from its initial 0 version through 1.0 and 2.0, describing architectural challenges, a three‑layer redesign, data‑engine selection (ROLAP, MOLAP, Kylin), transparent compression techniques, and practical lessons for large‑scale data systems.
In this interview, Baidu’s senior big‑data architect Ma Ruyue discusses his career transition from Hadoop to online databases, the design philosophy behind Baidu’s Palo ROLAP system, the future of HTAP, and his views on the evolving big‑data ecosystem including Spark, AI, and containerization.
This article details Chain Home's OLAP platform architecture and Kylin usage, covering the evolution from early ROLAP to MOLAP multi-dimensional engine, Kylin's basic principles, platform structure, application scenarios, usage specifications, capability extensions, and middleware development.
This article introduces Hulu's in‑house OLAP engine Nesto, detailing its near‑real‑time data ingestion, nested data model, TB‑level storage using HBase and Parquet, MPP query execution, custom predicate library, and the overall architecture that enables sub‑second ad‑hoc queries for user analytics.
This article shares a data engineer's three‑year experience using Apache Kylin for real‑time OLAP on petabyte‑scale data, describing the business background, challenges of pre‑computation, cube modeling, dimension reduction, and various optimization techniques such as hierarchy, mandatory, and joint dimensions, as well as precise count‑distinct handling.
This article shares practical experience deploying MariaDB ColumnStore for offline OLAP, covering its MPP‑based architecture, user‑experience benefits, high‑availability options, numerous syntax and data‑type limitations, and detailed scripts for schema conversion and incremental data extraction using pt‑archiver.
This article presents a detailed case study of Baidu Waimai's traffic analysis platform, outlining the data challenges of high dimensionality and volume, the evaluation of OLAP engines, the adoption of Apache Kylin for pre‑computation, the end‑to‑end data modeling, cube construction, incremental builds, and integration with Saiku‑Mondrian reporting, while sharing practical lessons and performance gains.
This article walks through Rubik's OLAP cube reduction techniques—including aggregation groups, required, combined, derived, hierarchical, and partial cubes—by designing and implementing buyers and suppliers cubes with six tables, demonstrating performance gains through pre‑computed queries and SQL examples.
This article explains why raw OLAP Cubes consume exponential storage, then details six practical dimensionality‑reduction methods—Aggregation Group, Mandatory Dimension, Joint Dimension, Derived Dimension, Hierarchy Dimension, and Partial Cube—showing how each can dramatically shrink materialized tables while preserving query performance.
The article explains the Materialized View Based Optimizer (MBO) in Inceptor, detailing its purpose, configuration parameters, supported aggregation, filter, and join scenarios, and shows how combining MBO with Transwarp Rubik cubes can dramatically accelerate OLAP query performance.
