Tagged articles
columnar storage
81 articles · Page 1 of 1
This article dissects VictoriaLogs' performance engineering across four layers—data architecture (LSM-tree, time partitioning, log-stream locality), query optimization (sparse indexes, Bloom filters), columnar storage with type-specific encoding and compression, and Go runtime tuning—showing that reducing unnecessary work via smart data organization outweighs micro-optimizations.
Bloom FilterGo performanceLSM-tree
0 likes · 26 min read

This article details Xiaohongshu's migration of its core real-time indexing pipeline from Kafka to Apache Fluss, covering architectural drivers like read amplification and historical replay contention, Fluss's columnar storage and Lakestream integration with Paimon, and a phased migration approach including baseline testing, shadow dual-run, and gradual cutover.
Apache FlussApache PaimonKafka
0 likes · 50 min read

This article details Apache Fluss's dual-table model where LogTable handles high-throughput append-only streaming with columnar storage while PrimaryKeyTable supports real-time updates, partial merges, and CDC changelogs via integrated RocksDB state storage, sharing unified tiering and lakehouse sinking.
Apache FlussCDCLogTable
0 likes · 19 min read

This article breaks down Dremel's columnar encoding for nested data, detailing the definition‑level and repetition‑level concepts, showing step‑by‑step examples of encoding and reconstructing JSON‑like schemas, and explaining the limits of single‑column reconstruction.
Apache ArrowDremelParquet
0 likes · 9 min read


TonyBaiMar 13, 2026 · Backend Development
The article analyzes why traditional row‑oriented databases struggle with high‑volume analytics, introduces DuckDB as an embedded columnar engine for Go, presents benchmark results of up to 18.6 M rows per second writes and 6 M rows per second scans, walks through the Appender API code, and outlines the trade‑offs and ideal hybrid architecture.
DuckDBGoOLAP
0 likes · 11 min read

AliSQL integrates DuckDB as its storage engine to achieve high‑density columnar compression and fast analytical scans, detailing DuckDB’s multi‑layer storage format, adaptive compression algorithm selection, performance benchmarks versus InnoDB, HBase, ClickHouse, OceanBase, and the engineering optimizations AliSQL adds for throughput and cost reduction.
AliSQLData CompressionDuckDB
0 likes · 12 min read

This article explains the challenges of frequent column changes in AI feature engineering, introduces Paimon’s column‑separation storage with a global continuous Row ID, details its Blob data type for efficient multi‑modal handling, and outlines production results and future roadmap for building an AI‑native data lakehouse.
Apache PaimonBLOBbig data
0 likes · 11 min read

The article explains how Apache Parquet’s columnar layout, multi‑level row‑group structure, projection and predicate push‑down, and advanced compression and encoding make it the high‑performance, space‑efficient storage format that powers modern big‑data ecosystems and tools like Spark, Python pandas, and ClickHouse.
ClickHouseCompressionParquet
0 likes · 11 min read

StarRocks 4.0 introduces FlatJSON, a columnar storage and execution engine that converts high‑frequency JSON fields into native columns, dramatically reducing I/O and CPU costs and enabling JSON queries to run with performance close to that of traditional columnar data.
Database PerformanceFlatJSONJSON
0 likes · 19 min read

This article examines how adaptive structural encoding in columnar formats like Lance dramatically improves random‑access performance on NVMe storage, compares it with Apache Parquet and Arrow, and discusses the trade‑offs between scan speed, memory usage, and compression.
LanceNVMeParquet
0 likes · 17 min read


ITPUBOct 11, 2025 · Databases
This article details OceanBase's evolution from a distributed OLTP system to a unified HTAP database, covering its cost‑based optimizer, vectorized execution, integrated row‑column storage, bypass import, materialized views, external tables, full‑text search, and real‑world use cases for real‑time analytics.
HTAPMaterialized ViewOceanBase
0 likes · 12 min read

This article explores ClickHouse’s high‑performance OLAP architecture, covering its MPP design, columnar storage, vectorized execution, pre‑sorting, table engines, data types, sharding and replication strategies, as well as index designs that together enable rapid analysis of massive datasets.
ClickHouseData Compressioncolumnar storage
0 likes · 15 min read

This article explores ClickHouse’s high‑performance OLAP design—including its MPP architecture, columnar storage, vectorized execution, pre‑sorting, sharding, replication, index strategies, and compute engine—showing how each innovation contributes to ultra‑fast, scalable data analysis in the big‑data era.
ClickHouseOLAPSharding
0 likes · 14 min read

This article explains ClickHouse’s core architecture—including its storage‑compute integration, MPP parallelism, columnar storage, vectorized execution, data pre‑sorting, table engines, sparse and auxiliary indexes, and the two‑stage aggregation pipeline—then walks through the exact GROUP BY execution flow for both local and distributed tables, illustrating each step with diagrams, SQL demos, and code snippets.
ClickHouseDistributed QueryHashMap
0 likes · 29 min read

This article examines ClickHouse’s high‑performance OLAP design, covering its MPP architecture, columnar storage, vectorized execution, pre‑sorting, table engines, extensive data‑type system, sharding and replication strategies, as well as its sparse and skip‑index mechanisms that together enable ultra‑fast analytics on massive datasets.
ClickHouseData PartitioningOLAP
0 likes · 16 min read

ClickHouse is an open‑source, massively parallel, column‑oriented OLAP database that integrates its own columnar storage, vectorized batch processing, pre‑sorted data, diverse table engines, extensive data types, sharding with replication, sparse primary‑key and skip indexes, and a multithreaded query engine, delivering high‑throughput real‑time analytics on massive datasets.
ClickHouseOLAPbig data
0 likes · 15 min read

This article explains how ClickHouse’s columnar storage, bitmap functions, and distributed architecture can be leveraged to store billions of tag bitmaps, combine them efficiently, and support fast group calculations for customer data platforms, while addressing data‑warehouse integration, storage format, and performance challenges.
BitmapOLAPcolumnar storage
0 likes · 10 min read

Tencent Cloud's TDSQL shattered world records in both TPC‑DS (OLAP) and TPC‑C (OLTP) benchmarks, achieving a 7260 M QphDS score at a cost of 37.52 CNY/kQphDS, and the article explains the three self‑developed technologies—MPP execution, parallel execution framework, and columnar‑vectorized engine—that made this performance possible.
Database PerformanceMPPOLAP
0 likes · 7 min read

This article compares ClickHouse and MySQL by examining benchmark results, MPP architecture, columnar storage, compression techniques, vectorized execution, and index designs, showing why ClickHouse delivers dramatically higher query performance on massive data sets.
ClickHouseMPPMySQL
0 likes · 8 min read


ITPUBAug 29, 2024 · Databases
TeleDB’s journey from a centralized MySQL/PostgreSQL‑based system to a native distributed HTAP database showcases innovations such as share‑nothing architecture, columnar storage, vectorized execution, Remote Data Access, global caching, and advanced dead‑lock detection, dramatically improving query performance, storage efficiency, and scalability.
Distributed DatabaseHTAPTeleDB
0 likes · 13 min read


21CTOJul 30, 2024 · Databases
This article surveys two decades of database system architecture innovations—including columnar stores, cloud databases, data lakes, NewSQL, hardware accelerators, and blockchain databases—highlighting their motivations, trade‑offs, and the shifting landscape that shapes modern DBMS design.
DBMSNewSQLcolumnar storage
0 likes · 23 min read

Vivo's unified content platform replaced its unwieldy 60 TB MongoDB store with HBase, then upgraded the cluster, introduced table‑specific connection pools, column‑only reads, tuned compaction, and leveraged multi‑version cells, cutting response times from seconds to under ten milliseconds and dramatically lowering operational costs while boosting read/write performance.
Compaction OptimizationDistributed DatabaseHBase
0 likes · 16 min read

This article explains why new data systems are needed, introduces Apache Arrow and its columnar in‑memory format, describes dynamic read‑time modeling, outlines the system’s execution flow, storage and indexing strategies, and shares practical tips and extensions for building scalable big‑data solutions.
AceroApache ArrowData Systems
0 likes · 20 min read

This article explains why new data systems are needed, introduces Apache Arrow’s columnar in‑memory format and its zero‑copy advantages, describes how to model data at read time, outlines the execution flow with Acero and SQL planning, and shares practical tips and extensions for building robust, dynamic‑schema data platforms.
AceroApache ArrowData Systems
0 likes · 20 min read

The article explains how Apache Arrow’s columnar, cross‑language in‑memory format enables high‑performance, interoperable data systems—replacing traditional row‑oriented databases—by supporting dynamic schemas, zero‑copy data exchange, efficient indexing, Acero‑based query execution, and Flight/ADBC connectivity, while offering practical guidance and highlighting challenges.
Apache ArrowData SystemsMemory Format
0 likes · 20 min read

This article explains why new data systems are needed, introduces Apache Arrow and its columnar in‑memory format, describes read‑time modeling and dynamic schema handling, and shows how Arrow can be used to build a complete data processing pipeline with indexing, SQL planning, and zero‑copy data exchange.
Apache ArrowData SystemsSQL
0 likes · 20 min read

This article explains the nature of semi‑structured data, compares traditional schema‑on‑read and schema‑on‑write approaches, and details MaxCompute's columnar storage solution that balances flexibility, performance, and cost for large‑scale data warehouses.
MaxComputeSemi-structured Databig data
0 likes · 19 min read

The interview with Wu Li, a research engineer at Shanghai Yanhuang Data, explores how columnar storage, JIT compilation, and push-mode processing are reshaping modern database performance, highlighting hardware constraints, software optimizations, and product‑centric goals in the era of big data analytics.
JIT CompilationOLAPcolumnar storage
0 likes · 11 min read

In this technical interview, Wu Li, a research engineer at Shanghai Yanhuang Data, explains how hardware constraints drive database evolution, why columnar storage and SIMD acceleration are crucial for OLAP, and how JIT compilation and push‑mode processing improve query performance and product experience.
JIT CompilationOLAPcolumnar storage
0 likes · 10 min read

The article presents an interview with Wu Li, a senior R&D engineer at Shanghai Yanhuang Data, discussing how columnar storage, JIT compilation, and push‑mode execution are reshaping database performance in the era of big‑data analytics and evolving hardware constraints.
Apache ArrowGandivaJIT Compilation
0 likes · 10 min read

The article presents an interview with Wu Li, a senior engineer at Shanghai Yanhuang Data, discussing how columnar storage, JIT compilation, and push‑mode processing are reshaping database performance and product strategy in the era of large‑scale data analytics.
JIT CompilationOLAPcolumnar storage
0 likes · 9 min read

This article explains why time‑series databases are ideal for massive telemetry data, describes the high‑cardinality problem that degrades performance, and shows how openGemini’s newly introduced columnar engine—combined with sorting and clustering indexes—effectively mitigates those issues while delivering impressive write and query speeds.
columnar storagedatabaseshigh cardinality
0 likes · 7 min read

This article explains the nature of semi‑structured data, compares schema‑on‑read and schema‑on‑write approaches, and shows how Alibaba Cloud MaxCompute leverages columnar storage and dynamic parsing to achieve low‑cost, high‑performance analytics for large‑scale data workloads.
MaxComputeSemi-structured Datacolumnar storage
0 likes · 20 min read

This article explains how a data‑ecosystem team governs and applies behavior‑analysis data by describing common analysis scenarios, data‑warehouse modeling methods and their pros and cons, the concepts and overall architecture of behavior‑centric analytics, key system components, and several concrete analysis examples such as retention, funnel and path analysis.
behavior analysisbig datacolumnar storage
0 likes · 12 min read

This article explains the motivation, design, and implementation of Alibaba Cloud's SLS Schema‑on‑Read scanning mode, showing how it enables SQL analysis on raw log data without pre‑built indexes, improves flexibility for evolving schemas, and reduces storage and index costs in various log‑analysis scenarios.
Cost OptimizationLog AnalyticsSQL
0 likes · 27 min read


ITPUBDec 18, 2022 · Databases
This article explains how ClickHouse achieves high query performance by leveraging storage‑engine designs such as pre‑sorting, columnar layout, and block‑level compression, and by exploiting a vectorized compute engine while avoiding joins and using built‑in functions.
ClickHouseCompressionDatabase Performance
0 likes · 9 min read

This article explains the differences between row-based and column-based storage, comparing their write and read performance, outlining advantages and disadvantages, and describing suitable scenarios such as OLAP queries, column families, compression, and indexing, to help choose the appropriate storage model.
OLAPRow Storagebig data
0 likes · 10 min read

The article presents a comprehensive technical overview of MercsDB—formerly HermesDB—including its background, storage and indexing designs, native and Presto computation engines, vectorization optimizations, benchmark results, real‑world applications, and future development plans.
MPPMercsDBbig data
0 likes · 20 min read

A comprehensive TPC‑H benchmark compares ClickHouse, Oracle, and the open‑source esProc SPL across simple and complex queries, showing ClickHouse excels at single‑table scans, while SPL consistently outperforms both in complex calculations and offers more concise code.
ClickHouseDatabase PerformanceOracle
0 likes · 12 min read

This article benchmarks ClickHouse, Oracle, and the open‑source esProc SPL on the TPC‑H suite, showing ClickHouse excels at simple scans, Oracle handles many complex queries, while SPL consistently outperforms both in speed and code simplicity across a range of workloads.
ClickHouseDatabase PerformanceOracle
0 likes · 12 min read

The article reports the June 2022 China database popularity ranking, highlights TiDB's comeback, introduces OtterTune's new financing, announces PostgreSQL 15 Beta 1, explains Google AlloyDB columnar features, and provides a detailed overview of columnar database concepts, history, advantages, and evolution.
AIDatabase RankingOtterTune
0 likes · 8 min read

This guide explains ClickHouse fundamentals, OLAP versus OLTP concepts, columnar storage benefits, core performance techniques, the MergeTree family and its indexing, specialized table engines, installation on Linux, Docker deployment, and integration with HDFS, MySQL, and Kafka for modern analytical workloads.
ClickHouseData IngestionDocker
0 likes · 30 min read

UniqueMergeTree, a new ClickHouse table engine, addresses real‑time data update challenges by combining upsert semantics, unique key enforcement, and efficient delete‑bitmap handling, offering higher query performance at modest write cost, with detailed design, sharding strategies, conflict resolution, and performance evaluation.
ClickHouseDatabase EngineReal-time Updates
0 likes · 14 min read

The talk outlines IoT data classifications, requirements, and proposes a one‑stop storage product using multi‑model support, columnar formats, compute‑storage separation, tiered storage, and query optimization to achieve ten‑fold cost reduction and ten‑fold performance gains.
IoTcloud nativecolumnar storage
0 likes · 20 min read

After months of using MongoDB for frontend monitoring logs, the data grew to billions of records causing severe query slowdown, prompting a migration to ClickHouse where columnar storage, partitioning, and OLAP capabilities dramatically improved performance and storage efficiency.
ClickHouseDatabase MigrationMongoDB
0 likes · 17 min read

This tutorial walks through setting up a Citus‑enabled PostgreSQL cluster on Kubernetes, creating distributed tables, using co‑location and reference tables for high‑performance joins, and comparing row‑based versus columnar storage with concrete SQL commands and performance metrics.
CitusCo-locationDistributed Tables
0 likes · 11 min read

This article explains how Apache Doris adopts CPU‑level vectorization and columnar storage to boost query performance, details the design and current status of its vectorized engine, and outlines future work such as JOIN acceleration, storage‑layer vectorization, import optimization, and extensive SQL function support.
Apache DorisSIMDSQL Engine
0 likes · 21 min read

This guide shows how to combine PostgreSQL declarative time‑range partitioning with Citus distributed sharding to create a scalable time‑series database, covering table creation, distribution, automatic monthly partition generation, columnar archiving, maintenance with pg_cron, and performance benefits such as faster inserts, queries, and data expiration.
CitusDistributed DatabasePartitioning
0 likes · 11 min read

This article recounts the author's journey from using MongoDB for front‑end monitoring logs to migrating to ClickHouse, detailing the challenges with large‑scale data, optimization attempts, the fundamental differences between row‑ and column‑oriented databases, and the resulting performance and storage improvements.
Database MigrationMongoDBNode.js
0 likes · 19 min read

This article explains how Apache Doris adopts CPU vectorization techniques—such as SIMD, columnar storage, and cache‑friendly designs—to boost query performance, detailing its current vectorized engine architecture, recent benchmarks, ongoing work on JOIN, storage, import, and future enhancements.
Apache DorisDatabase PerformanceSIMD
0 likes · 22 min read

This article provides a comprehensive overview of Tencent's self‑developed MySQL kernel TXSQL, covering its evolution, overall architecture, columnar storage engine, instant DDL capabilities, enterprise‑grade features, high‑availability mechanisms, performance optimizations, and the rigorous development and testing processes behind the product.
Cloud DatabaseHigh AvailabilityMySQL
0 likes · 11 min read

This article provides a comprehensive introduction to Apache Parquet, covering its origins, columnar storage advantages, nested schema support, internal architecture, storage model components, comparison with ORC, and practical tools for inspecting Parquet files.
Data CompressionHadoopORC Comparison
0 likes · 10 min read

Kudu is an open‑source columnar storage engine from Cloudera that combines high‑throughput batch processing with low‑latency random reads, offering features such as C++/Java APIs, Raft‑based replication, flexible consistency, partitioning, and integration with Hadoop, Spark, Impala, and other ecosystem components.
HadoopKuducolumnar storage
0 likes · 64 min read

Tongcheng Yilong’s BaikalDB deployment combines high‑availability multi‑Raft HA, HTAP support, and share‑nothing scalability to deliver over 72K TPS OLTP and ten‑fold faster OLAP queries while cutting operational costs up to a hundredfold through dual‑center, columnar storage and cloud‑native elasticity.
BaikalDBDistributed DatabaseHTAP
0 likes · 27 min read

ClickHouse is an MPP column‑oriented DBMS that combines full DBMS functionality, advanced columnar storage with high compression, SIMD‑based vectorized execution, a rich relational SQL interface, diverse table engines, multi‑master clustering, and flexible sharding and distributed query capabilities, making it exceptionally fast for analytical workloads.
ClickHouseDBMSSQL
0 likes · 21 min read

ClickHouse is a high‑performance MPP column‑store DBMS that combines complete DBMS functions, column‑oriented storage with aggressive compression, SIMD‑based vectorized execution, flexible table engines, multithreading, distributed processing, a multi‑master architecture, and SQL compatibility to deliver fast online analytical queries on massive data sets.
ClickHouseDBMScolumnar storage
0 likes · 21 min read


ITPUBDec 29, 2020 · Databases
This article details how the DTCC2020 guest speaker from Tongcheng‑Elong introduced BaikalDB’s distributed columnar storage, covering internal and external motivations, technology comparison, architecture, implementation tricks, performance gains in production, and future hybrid row‑column research directions.
BaikalDBHTAPcolumnar storage
0 likes · 12 min read

This article provides a comprehensive overview of Apache Parquet, covering its modular architecture, nested data model, striping/assembly and definition level algorithms, file format details, push‑down optimizations, performance benchmarks, and the project's evolution within the big‑data ecosystem.
ParquetPushdown Optimizationcolumnar storage
0 likes · 18 min read

This article explains the ORC (Optimized Row Columnar) file format, its advantages, internal structure, data model, compression mechanisms, and how to create Hive tables and write ORC files using Java, providing practical code examples and reference resources.
HiveJavaORC
0 likes · 15 min read

This article explains what ClickHouse is, compares column‑store and row‑store databases, shows how to migrate large MySQL tables to ClickHouse, presents performance test results, discusses data synchronization methods, highlights why ClickHouse is fast, and shares common migration pitfalls.
ClickHouseOLAPcolumnar storage
0 likes · 7 min read

ClickHouse is a PB‑scale, open‑source columnar OLAP database that uses a ZooKeeper‑coordinated sharded cluster, columnar storage, vectorized execution, advanced compression, data‑skipping indexes, and materialized views to deliver high‑performance interactive analytics, yet it requires manual shard management, lacks a mature MPP optimizer, and handles real‑time single‑row writes poorly.
ClickHouseData CompressionOLAP
0 likes · 18 min read

This article systematically examines the core technologies of OLAP systems, covering storage models, columnar formats, indexing, distributed storage architectures, query execution steps, optimizer designs, and emerging trends such as real‑time analytics, HTAP, cloud‑native deployment, and hardware acceleration.
OLAPcolumnar storagedistributed systems
0 likes · 23 min read

This article explains how JD Cloud’s JCHDB, built on ClickHouse, achieves millisecond‑level queries on billions of rows through columnar storage, distributed multi‑master architecture, SIMD vector engine, sparse indexing, and specialized table engines, and outlines the ideal use cases and deployment details.
ClickHouseJCHDBanalytical database
0 likes · 10 min read

The article explains SAP HANA’s performance advantages by combining four key technologies—high‑speed memory, columnar storage, data compression, and an insert‑only model—detailing their individual pros and cons, how they complement each other, and the trade‑offs involved in scaling and persistence.
CompressionIn-MemoryInsert-Only
0 likes · 19 min read

This document explains the differences between OLTP and OLAP workloads, introduces the CSTORE columnar storage engine architecture, its core technologies, performance advantages, typical use cases, benchmark results, and future development plans for MySQL 8.0.
CStoreDatabase EngineMySQL
0 likes · 14 min read

Apache Parquet is a language‑agnostic, columnar storage format for the Hadoop ecosystem that offers high compression, efficient I/O through column and predicate push‑down, nested‑structure support, and a three‑layer architecture, and is compared with ORC while providing tooling for schema inspection.
Apache HadoopData FormatsORC Comparison
0 likes · 9 min read

This article explains how Alibaba's MaxCompute platform evolved its storage engine from row‑based CFile to the columnar AliORC format, details the technical innovations such as async prefetch, small I/O elimination, adaptive dictionary encoding, and range‑aligned reads, and compares its performance against Apache ORC and Parquet.
AliORCApache ORCMaxCompute
0 likes · 20 min read

This article provides a comprehensive overview of Apache Druid's distributed column‑store architecture, detailing its node types, external dependencies, data flow, and operational mechanisms, and compares Druid's real‑time analytics capabilities with systems such as Impala, Elasticsearch, and Spark.
Apache Druidcolumnar storagedistributed system
0 likes · 12 min read

This article presents a comprehensive overview of Apache Doris, covering its positioning as a distributed MPP analytical database, core architecture with FE and BE nodes, key technologies such as vectorized execution and materialized views, integration with Kafka and Elasticsearch, additional features, roadmap, and detailed case studies from Baidu Statistics and Meituan, illustrating its practical deployment and performance characteristics.
Apache DorisKafka Integrationcolumnar storage
0 likes · 25 min read

This article provides a comprehensive overview of Apache Parquet, covering its purpose, architectural components, nested data model, file structure, practical Hive commands for creating and inspecting Parquet tables, and a brief introduction to the TPC‑DS benchmark for performance testing.
Data ModelingHiveParquet
0 likes · 8 min read

This article explains the differences between row‑oriented and column‑oriented storage, compares their trade‑offs, and introduces HBase’s column‑family architecture, including row keys, column qualifiers, timestamps, cells, and how it maps to a multi‑dimensional map structure.
HBasebig datacolumnar storage
0 likes · 7 min read

The article details a real‑world MySQL performance case where a sudden 100‑fold load increase was mitigated through read/write splitting, replica‑based statistics, limited index tuning, middleware‑driven sharding, and finally a columnar storage layer (Infobright) with scripted dynamic data synchronization, achieving dramatic latency reductions and scalable architecture.
InfobrightMySQLcolumnar storage
0 likes · 12 min read

A MySQL‑based receipt‑tracking service suffered a sudden 100‑fold load increase, prompting a step‑by‑step optimization that combined read‑write splitting, middleware‑less data routing, columnar storage with Infobright, and dynamic scheduling to dramatically lower CPU/IO pressure and restore performance.
MySQLcolumnar storagedatabase scaling
0 likes · 13 min read

HybridDB for MySQL delivers millisecond‑scale, multi‑dimensional filtering on billions of rows with hundreds of metrics by combining a high‑performance columnar engine, automatic composite indexes, and a fused MPP‑DAG pipeline, turning half‑day push preparation into seconds while supporting full SQL, spatial, and JSON data.
HybridDBOLAPSQL
0 likes · 8 min read

This article introduces Hulu's in‑house OLAP engine Nesto, detailing its near‑real‑time data ingestion, nested data model, TB‑level storage using HBase and Parquet, MPP query execution, custom predicate library, and the overall architecture that enables sub‑second ad‑hoc queries for user analytics.
HBaseOLAPQuery Engine
0 likes · 22 min read

Apache CarbonData 1.0.0, now an Apache incubating project, adds over 80 new features and bug fixes—including a new data loading solution, Spark 2.1 integration, update/delete SQL support, adaptive compression for numeric types, B‑Tree LRU cache, V2 format for faster first‑query performance, vectorized reader, bucket‑table joins, off‑heap memory, single‑pass loading, and pre‑generated dictionaries—aimed at delivering faster, more flexible, and efficient columnar storage for big‑data workloads.
Apache CarbonDataData CompressionSpark Integration
0 likes · 8 min read

This article describes Hulu’s real‑world OLAP query optimization, covering the fundamentals of OLAP, comparisons of row‑ and column‑based storage formats, detailed indexing mechanisms of Parquet, ORC and CarbonData, and the specific schema, shuffle, block size, speculation and GC tuning techniques that enabled CarbonData to dramatically accelerate wide‑table queries on SparkSQL.
CarbonDataOLAPSparkSQL
0 likes · 17 min read

This article explains Dameng's columnar storage architecture, the smart index mechanism that leverages zone statistics to reduce I/O, and the adaptive compression algorithms—including dictionary, constant, RLE, and sequence encoding—used to achieve high compression ratios on columnar data.
Damengadaptive compressioncolumnar storage
0 likes · 14 min read

This article explains IBM DB2 BLU Accelerator’s columnar storage, multi‑level compression, TSN‑based logical rows, SIMD processing, intra‑parallel execution, probability‑based caching, and automatic admin features, showing how these technologies together deliver dramatic I/O and performance gains for analytical workloads.
BLU AcceleratorCompressionDB2
0 likes · 15 min read

This article explains four fundamental database data models—key‑value pair, document, column, and graph—detailing their structures, scalability characteristics, and typical implementations such as Redis, MongoDB, HBase, and Neo4j.
columnar storagedatabase modelsgraph database
0 likes · 2 min read

This article provides a comprehensive analysis of SQL‑on‑Hadoop architectures, comparing runtime‑framework‑based engines like Hive with MPP‑style engines such as Impala, detailing core components, compilation pipelines, optimizer strategies, CPU/IO performance tricks, columnar storage formats, and resource management in modern big‑data query platforms.
OptimizationQuery EngineSQL on Hadoop
0 likes · 22 min read
