Tagged articles
incremental computing
15 articles · Page 1 of 1
The article examines the limitations of traditional batch and stream processing, explains how Hologres Dynamic Table combines declarative freshness settings with stateful incremental computation to bridge the gap between low‑cost batch jobs and low‑latency streaming, and presents benchmark results and real‑world case studies.
Cloud Data WarehouseDynamic TableHologres
0 likes · 13 min read

The article analyzes the limitations of traditional batch and stream processing, introduces Hologres Dynamic Table as a declarative, incremental‑compute framework that bridges the gap between low‑cost batch jobs and low‑latency streaming, and validates its performance with benchmarks and real‑world case studies.
Cloud Data WarehouseDynamic TableHologres
0 likes · 13 min read

This article details Xiaohongshu's journey from a ClickHouse‑based batch analytics stack to a unified lakehouse architecture powered by generic incremental computing, showing how the company reduced architecture complexity, resource consumption and development effort each to roughly one‑third while supporting trillions of daily events with sub‑10‑second query latency.
Xiaohongshubig datadata architecture
0 likes · 24 min read

This article details Xiaohongshu's data platform evolution from a simple ClickHouse‑based ad‑hoc system to a Lambda‑style architecture and finally a lakehouse solution, highlighting how the adoption of a new incremental computing model reduced architectural complexity, resource consumption and development effort each to roughly one‑third while delivering sub‑second query performance on petabyte‑scale data.
Performance OptimizationXiaohongshubig data
0 likes · 21 min read

This article analyzes Xiaohongshu's data platform evolution—from a simple ClickHouse‑based analytics layer to a Lambda architecture and finally a lakehouse design—highlighting how adopting a new incremental computing model reduced architecture complexity, resource consumption, and development effort each to roughly one‑third while delivering sub‑second query performance on petabyte‑scale data.
Performance OptimizationXiaohongshubig data
0 likes · 22 min read

This article collection examines Tencent Cloud’s Iceberg batch‑stream integration, AI‑driven game data governance, Apache Gravitino unified metadata and lineage, Xiaohongshu’s multimodal data‑lake evolution, and Volcano Engine’s Data+AI multimodal lake, highlighting architectures, techniques, performance gains, and practical implementations.
AI governanceGravitinoIceberg
0 likes · 7 min read

This article explains how Xiaohongshu, a lifestyle community with over 350 million monthly users, transformed its data platform from a traditional Lambda architecture to a next‑generation incremental computing model, reducing architectural complexity, resource consumption and development effort each by roughly two‑thirds while supporting massive real‑time and offline data demands.
AICloud MigrationXiaohongshu
0 likes · 6 min read

This article details how Xiaohongshu's massive user‑generated data platform evolved from a simple ClickHouse‑based architecture to a multi‑stage lakehouse design, adopting a new incremental computing model that reduced architecture complexity, resource and development costs by one‑third while boosting query performance for petabyte‑scale data.
incremental computinglakehouse
0 likes · 19 min read

This interview with Yunqi Technology CTO Guan Tao explains how the traditional Lambda architecture’s triple‑system complexity drives costs and operational pain, and why the company’s General Incremental Computing (GIC) approach offers a unified, cost‑effective Kappa‑style solution for real‑time, batch, and interactive analytics.
Data EngineeringKappa architectureLambda architecture
0 likes · 13 min read

This article examines how Xiaohongshu replaced its costly Lambda architecture with a real‑time lakehouse built on Iceberg, Paimon, Spark, and StarRocks, achieving minute‑level latency, higher data quality, lower resource consumption, and dramatically faster query performance.
IcebergPaimonStarRocks
0 likes · 7 min read

Facing billions of daily logs and the need for minute‑level experiment metrics, Xiaohongshu partnered with Yunqi Tech to design a generic incremental‑compute solution that delivers near‑real‑time data warehousing with lower cost, higher accuracy, simplified pipelines, and improved query performance.
FlinkIcebergPaimon
0 likes · 24 min read

At the 2025 DA Data+AI Conference in Shanghai, Tencent Cloud unveiled Setats—a unified stream‑batch‑incremental engine that cuts system costs, delivers second‑level data visibility and real‑time changelog generation, and demonstrates measurable performance gains in automotive IoT analytics while integrating tightly with the WeData platform.
batch processingbig data architecturedata lake
0 likes · 5 min read

This article introduces Yunqi's self‑developed Lakehouse product, explaining its cloud‑native, one‑stop data platform architecture, incremental computing that balances freshness, performance and cost, and the autoMV feature that automatically creates materialized views to boost query speed up to nine times.
Auto Materialized Viewbig datadata platform
0 likes · 14 min read

The article explains why unified data architecture is essential to avoid duplication and inefficiency, discusses differing performance trade‑offs among batch, streaming, and interactive analytics, introduces an incremental computation model that unifies these modes, and invites readers to a Dec 19, 2023 technical sharing event.
Lambda architecturebatch processingbig data
0 likes · 3 min read


ITPUBApr 8, 2023 · Big Data
Facing daily terabyte‑scale data ingestion and costly duplicate reads in its ODS‑to‑DWD pipeline, Bilibili introduced a Flink‑based real‑time incremental computation and multi‑level partition shuffling, dramatically reducing read amplification, cutting resource usage by ~20%, improving latency to minutes, and enhancing scalability.
Data PipelineFlinkSpark
0 likes · 19 min read
