Tagged articles
Query Acceleration
11 articles · Page 1 of 1
StarRocks 4.1 introduces an incremental materialized view for Apache Iceberg that ties refresh cost to data changes instead of table size, dramatically cutting refresh time, maintaining low latency, and keeping query results fresh even as tables scale to terabytes or petabytes, with a fallback to partition refresh when needed.
Apache IcebergIncremental Materialized ViewQuery Acceleration
0 likes · 8 min read

This article summarizes how the Doris‑Paimon lakehouse architecture leverages Doris' high‑performance OLAP engine to accelerate lake queries, provides a unified data analysis gateway, supports unified data integration, and enables open, layered data modeling for modern big‑data workloads.
DorisPaimonQuery Acceleration
0 likes · 9 min read

Magnus is Bilibili’s self‑developed intelligent service that continuously optimizes Iceberg tables by scheduling snapshot expiration, orphan‑file cleanup, manifest rewriting, and multi‑dimensional data optimizations—including small‑file merging, sorting, distribution, and index creation—while automatically recommending configurations from real‑time query logs, delivering over 99.9% task success and up to 30% scan‑data reduction.
IcebergIntelligent RecommendationOptimization
0 likes · 15 min read

This article explains the concept, advantages, and drawbacks of materialized views, describes how MaxCompute implements them—including creation syntax, maintenance properties, automatic query rewrite, smart recommendation, and auto‑materialization—and shares performance results and future improvement plans.
Automatic RefreshMaterialized ViewMaxCompute
0 likes · 13 min read

StarRocks’ materialized view feature lets Didi replace costly fuzzy deduplication with precise, high‑concurrency deduplication for real‑time dashboards, using global dictionary mapping, layered ODS/DWD/ADS views, synchronous and asynchronous refreshes, and transparent query rewrite to cut query latency by 80% and boost QPS dramatically.
OLAPPrecise DeduplicationQuery Acceleration
0 likes · 20 min read

This article details Bilibili's implementation of an Iceberg‑based lakehouse platform that unifies storage and analytics, addressing Hive’s performance and latency issues through multidimensional sorting, various file‑level indexes, cube pre‑aggregation, star‑tree structures, and an automated Magnus service for intelligent optimization, achieving near‑second query responses.
IcebergOLAPQuery Acceleration
0 likes · 14 min read

StarRocks 3.0 introduces a lake‑warehouse unified architecture where materialized views reduce data processing complexity, boost query performance, and improve timeliness, offering declarative modeling, transparent acceleration, and incremental computation for enterprise‑scale analytics.
Data ModelingIncremental RefreshLake Warehouse
0 likes · 19 min read

This article explains how to implement real‑time materialized views to accelerate large‑scale time‑series data queries, covering the need for materialized views, their definition, storage, incremental updates, pre‑computation, query partitioning, performance testing, and future directions.
Data PartitioningMaterialized ViewQuery Acceleration
0 likes · 16 min read

This article details Bilibili's implementation of a sub‑second response lakehouse platform using Apache Iceberg, covering background challenges, query acceleration techniques such as multi‑dimensional sorting, indexing, cube pre‑aggregation, and intelligent automated optimizations via the Magnus service, and reports current production metrics.
CubeIcebergMagnus
0 likes · 14 min read

This article details Bilibili's technical practice of constructing a millisecond‑response lake‑warehouse platform using Apache Iceberg, covering the background challenges, unified architecture, multi‑dimensional sorting and indexing for query acceleration, the Magnus service for intelligent optimization, and the current production deployment and performance metrics.
CubeIcebergOLAP
0 likes · 14 min read

This article explains Bilibili's lake‑warehouse integrated architecture, describing how Iceberg, MagnuS, Trino, and Alluxio are used to achieve flexible data storage, high‑performance query acceleration, and automated indexing through Z‑Order, Hilbert curve, Bloom filter, and advanced BitMap techniques.
IcebergQuery AccelerationZ-Order
0 likes · 18 min read
