Tagged articles
materialized views
39 articles · Page 1 of 1
The article analyzes two online travel case studies: Tongcheng Travel migrating from Hudi to Paimon to solve slow large-table real-time writes using LSM-tree storage, and Trip.com adopting StarRocks with materialized views to unify lake queries and eliminate data warehouse copies, detailing migration methodologies, architectural reasoning, and production troubleshooting.
Apache HudiApache PaimonLSM-tree
0 likes · 22 min read

The author details migrating a high-volume logging platform from Elasticsearch to ClickHouse, covering schema design with ReplicatedMergeTree, Spring Boot integration via JDBC and MyBatis, materialized views for pre-aggregation, and optimization techniques like skip indexes, batch inserts, and capacity planning.
ClickHouseElasticsearchKafka
0 likes · 21 min read

The article recounts how an API that originally used OFFSET pagination suffered 2‑3 seconds response time, and after rewriting the query to Keyset (Seek) pagination—adding a secondary sort key for stability and exploring cursor pagination, index‑only OFFSET, and materialized views—the same endpoint now responds in under 200 ms, with detailed benchmark comparisons.
Cursor PaginationKeyset PaginationSQL
0 likes · 7 min read

This article shares production lessons from integrating Spring Boot with Apache Doris for real-time data warehousing, covering connection pool tuning, data model selection, ingestion methods, materialized views, Colocate Join optimization, resource isolation, compaction tuning, BI tool integration, and cluster operations.
Apache DorisCluster OperationsColocate Join
0 likes · 16 min read

This article dissects the full advertising funnel, explains why CTR and eCPM are critical, and demonstrates how StarRocks combined with Flink can deliver minute‑level real‑time monitoring, material selection, anomaly alerts, A/B testing, and a successful migration from Druid for massive ad‑tech workloads.
AdvertisingSQLStarRocks
0 likes · 20 min read

This article explains MySQL view fundamentals, outlines update limitations, demonstrates INSTEAD OF triggers and stored procedures as workarounds, details two methods for simulating materialized views—including scheduled table refreshes and trigger‑based real‑time updates—covers optimization techniques, and shows role‑based permission management for both regular and materialized views.
MySQLmaterialized viewsoptimization
0 likes · 13 min read

Facing Postgres overload and costly Snowflake queries, Fresha rebuilt its analytics platform by introducing StarRocks as a unified SQL entry point, combining federated lakehouse queries with high‑performance internal tables, which reduced homepage query latency to around 200 ms and achieved minute‑level data freshness across real‑time, historical, and search workloads.
StarRockscompute-storage separationdata pipeline
0 likes · 20 min read

This article explains what materialized views are, outlines their advantages such as query acceleration, lightweight ETL, and lake‑warehouse integration, classifies them by sync mode, table count, and refresh strategy, and highlights their limitations and best‑practice recommendations for production use.
Database PerformanceETLdata warehousing
0 likes · 6 min read

Facing minute‑level decision demands and billions of marketing events during Taobao's Flash Sale, the Ele.me data team built a real‑time lakehouse with StarRocks and Paimon, leveraging asynchronous materialized views, RoaringBitmap de‑duplication, and resource isolation to achieve sub‑second query latency, lower storage costs, and stable high‑concurrency.
PaimonRoaringBitmapStarRocks
0 likes · 25 min read

Faced with billions of marketing events and minute‑level decision requirements during Taobao's flash‑sale campaign, the e‑commerce data team built a real‑time lakehouse using StarRocks and Paimon, leveraged asynchronous materialized views and RoaringBitmap deduplication, and achieved sub‑second query latency, massive cost savings, and stable high‑concurrency performance.
PaimonRoaringBitmapStarRocks
0 likes · 26 min read

Grab replaced its fragmented Grafana‑Superset stack with a StarRocks‑backed Iris platform, achieving over ten‑fold query speedups, 40% lower resource usage, and a unified real‑time and historical data store for Spark observability across its Southeast Asian super‑app ecosystem.
KafkaSparkStarRocks
0 likes · 16 min read

This article provides a comprehensive technical overview of StarRocks' query optimizer, covering its evolution, core architecture, multi‑stage optimization pipeline, key optimizations such as multi‑join colocate, low‑cardinality global dictionary, MV union rewrite, and advanced mechanisms like cost‑estimation fixes, query feedback, adaptive execution, runtime filters, join‑reorder strategies, and SQL plan management.
Cost-Based OptimizationOLAPSQL Plan Management
0 likes · 26 min read

NAVER migrated its massive analytics platform from ClickHouse to StarRocks, achieving dramatic improvements in multi‑table JOIN performance, real‑time aggregation speed, and horizontal scalability while simplifying data integration across heterogeneous sources on a Kubernetes‑based architecture.
ClickHouseKubernetesQuery Performance
0 likes · 13 min read

Facing high costs and scaling limits with ClickHouse, we migrated a 4000‑core, 500TB OLAP workload to StarRocks, achieving 40% cost reduction, 50% storage savings, and up to 30× query speedups through storage‑compute separation, materialized‑view rewrites, and extensive performance tuning.
ClickHouseOLAPPerformance Tuning
0 likes · 18 min read

This article introduces RisingWave, an open‑source streaming database, and explains how its SQL‑based interface, compute‑storage separation, UDF support, and materialized views enable efficient real‑time feature engineering, state management, and diverse downstream applications, including the enhancements in RisingWave 2.0.
RisingWaveSQLState Management
0 likes · 17 min read

Apache Doris 3.0 introduces storage‑compute separation, native lakehouse write‑back, optimized Variant handling for semi‑structured data, stronger ETL transaction support, enhanced multi‑table materialized views, and Java UDTF capabilities, providing developers with more flexible, cost‑effective, and high‑performance analytics solutions.
Apache DorisETLJava UDTF
0 likes · 7 min read

CryptoHouse is a free, real‑time blockchain analytics platform built on ClickHouse and Goldsky, offering SQL access to Solana and Ethereum data, with materialized views, fair‑use quotas, deduplication via ReplacingMergeTree, a lightweight UI, and detailed engineering insights.
Blockchain analyticsClickHouseGoldsky
0 likes · 18 min read

This article provides a detailed technical comparison between StarRocks and Trino, covering their shared MPP architecture, cost‑based optimizer, pipeline execution, ANSI SQL support, differences in vectorized execution, materialized view capabilities, caching systems, data source connectors, benchmark results, high‑availability designs, join algorithms, and real‑world user case studies.
CacheMPPStarRocks
0 likes · 20 min read

This article explains the concept of a metrics layer, why StarRocks is suited for building such platforms, and presents detailed case studies from Airbnb, a major bank, and a leading restaurant chain, while comparing three modeling paradigms and outlining the future vision for materialized views.
Data ModelingMetrics PlatformStarRocks
0 likes · 18 min read

Airbnb’s Riverbed framework unifies streaming CDC events and batch Spark jobs behind a GraphQL‑based declarative API to automatically build and maintain distributed materialized views, using Kafka‑partitioned ordering and version control to deliver billions of daily updates with low‑latency reads for features such as payments and search.
AirbnbApache SparkData Engineering
0 likes · 8 min read

Didi leverages StarRocks materialized views with a global dictionary and bitmap aggregation to perform real‑time, high‑cardinality precise deduplication, automatically rewriting queries and refreshing views, cutting query latency by ~80%, reducing resource use ~95%, and boosting concurrent QPS up to 100‑fold, while planning further automation and bitmap optimizations.
OLAPPrecise DeduplicationStarRocks
0 likes · 19 min read

StarRocks’ materialized view feature lets Didi replace costly fuzzy deduplication with precise, high‑concurrency deduplication for real‑time dashboards, using global dictionary mapping, layered ODS/DWD/ADS views, synchronous and asynchronous refreshes, and transparent query rewrite to cut query latency by 80% and boost QPS dramatically.
OLAPPrecise DeduplicationQuery Acceleration
0 likes · 20 min read

This article details Didi's evolution from a multi‑engine OLAP stack to a unified StarRocks solution, explains the design of global dictionaries and materialized views for real‑time dashboard acceleration, and shares performance results, challenges, and future optimization directions.
DidiOLAPStarRocks
0 likes · 19 min read

WeChat evolved its data platform from Hadoop to ClickHouse and finally to a StarRocks‑based lakehouse, solving data fragmentation and storage redundancy while achieving sub‑second to minute‑level query latency, cutting storage costs by over 65%, halving operational tasks, and reducing offline job time by two hours across several business lines.
StarRocksWeChatbig data
0 likes · 16 min read

This article introduces Apache Doris, a real‑time analytical database built on an MPP architecture, explains its suitability for massive data workloads and online high‑concurrency reporting scenarios, and details the core technologies—storage models, vectorized query engine, materialized views, partitioning, indexing, row‑store and prepared statements—that enable sub‑second query latency and high QPS, while also showing a real‑world case study and how to join the Doris community.
Apache Doriscommunitydata warehouse
0 likes · 13 min read

StarRocks, an open‑source MPP analytical database, consolidates BI, interactive, and real‑time analytics into a single engine by evolving from version 1.0 to 3.x, introducing compute‑storage separation, unified catalog, generated columns, operator spill, and advanced materialized views, while outlining its cloud‑native lakehouse roadmap.
MPP databaseStarRockscompute-storage separation
0 likes · 22 min read

StarRocks introduces a spill-to-disk mechanism that writes intermediate results of heavy operators to disk, freeing memory and enabling stable execution of ETL and ad‑hoc queries, while combined with materialized views it dramatically improves query success rates and delivers up to 4.35× faster performance than Spark.
Query PerformanceSpillStarRocks
0 likes · 10 min read

StarRocks 3.1 introduces a cloud‑native, lakehouse‑oriented architecture with enhanced storage‑compute separation, up to 3‑6× faster data‑lake queries than Trino/Presto, expanded Iceberg and Paimon support, richer materialized view capabilities, new random bucketing, expression partitioning, generated columns, and spill‑to‑disk stability, all backed by extensive performance optimizations and open‑source contributions.
SQLStarRocksdata lake
0 likes · 17 min read

Mango TV replaced its fragmented EMR‑Hive‑Kudu‑Presto stack with a unified StarRocks lakehouse, simplifying architecture, cutting operational costs, and achieving more than a ten‑fold increase in query speed while supporting real‑time analytics, materialized views, bitmap indexing, and store‑compute separation.
Bitmap IndexStarRocksStore-Compute Separation
0 likes · 14 min read

This article presents an in‑depth overview of the YouShu BI product, outlines the high‑concurrency performance challenges faced by enterprise BI, and details the multi‑layer performance architecture—including front‑end, back‑end, data engine, and data source layers—along with smart caching, MPP acceleration, materialized views, and the Data Doctor operations that together ensure low‑latency, reliable analytics for large‑scale users.
BIMPPSmart Caching
0 likes · 16 min read

This article details the three‑stage evolution of 360's real‑time data warehouse—from Storm + Druid + MySQL to Flink + Druid + TiDB and finally to Flink + Apache Doris—explaining architectural pain points, the reasons for choosing Doris, and how the new system delivers sub‑second query latency, strong consistency, and simplified operations across advertising scenarios.
Apache DorisData ConsistencyFlink
0 likes · 17 min read

This article presents the design and implementation of Tencent Lighthouse's Fusion Analysis Engine, covering its background, challenges, fusion architecture, kernel optimizations, acceleration techniques, practical outcomes, and future evolution directions for high‑performance data access.
Fusion EngineLighthouseSQL Optimization
0 likes · 12 min read

The article details ByteDance's large‑scale deployment of the Presto OLAP engine for ad‑hoc, BI, and near‑real‑time analytics, describing its architecture, multi‑coordinator high‑availability design, routing gateway, adaptive cancel, history server, materialized‑view support, Hudi connector integration, and how these innovations improve performance, stability, and operational efficiency.
High AvailabilityHudi ConnectorOLAP
0 likes · 11 min read


ITPUBApr 8, 2022 · Big Data
This article explains how a large‑scale advertising platform replaced its slow offline data‑warehouse with a ClickHouse‑based real‑time warehouse, covering data source integration, performance comparison, materialized views, projections, schema management, and cost‑effective hot‑cold storage strategies.
ClickHouseKafka IntegrationProjections
0 likes · 19 min read

ByteDance’s internal Presto platform, supporting nearly one million daily queries across ad‑hoc, BI visualization, and near‑real‑time analytics, achieves high performance and stability through SparkSQL compatibility, multi‑Coordinator architecture, dynamic routing, adaptive query cancellation, History Server, materialized views, and a dedicated Hudi connector.
Hudi ConnectorOLAPdistributed SQL
0 likes · 11 min read

This article presents an in‑depth overview of ClickHouse Projection, covering its background, definition, practical use cases, underlying architecture, query analysis, consistency guarantees, performance comparisons, and real‑world production results, highlighting how it enhances OLAP workloads while maintaining strong data consistency.
ClickHouseOLAPProjection
0 likes · 19 min read

ClickHouse is a PB‑scale, open‑source columnar OLAP database that uses a ZooKeeper‑coordinated sharded cluster, columnar storage, vectorized execution, advanced compression, data‑skipping indexes, and materialized views to deliver high‑performance interactive analytics, yet it requires manual shard management, lacks a mature MPP optimizer, and handles real‑time single‑row writes poorly.
ClickHouseData CompressionOLAP
0 likes · 18 min read

This article shares a detailed case study of how a startup managed a massive cinema‑ticket dataset with PostgreSQL, evaluated cloud and self‑hosted options, refined materialized‑view strategies, built a reliable job queue, and distilled practical takeaways for large‑scale database operations.
PostgreSQLcloud hostingdatabase scaling
0 likes · 27 min read

An expert panel from the High‑Efficiency Operations community answers eleven pressing MySQL 5.7 questions, covering test hardware, GA timeline, GIS speed, virtual column indexing, materialized view plans, 2‑D geo support, performance monitoring, Oracle’s market strategy, geojson handling, multi‑source replication, buffer‑pool locking, and InnoDB Memcached consistency.
Database PerformanceGISInnoDB
0 likes · 7 min read
