Materialized Views

DataFunTalk

Dec 25, 2024 · Databases

Applying RisingWave to Real-Time Feature Engineering: Architecture, Capabilities, and Use Cases

This article introduces RisingWave, an open‑source streaming database, and explains how its SQL‑based interface, compute‑storage separation, UDF support, and materialized views enable efficient real‑time feature engineering, state management, and diverse downstream applications, including the enhancements in RisingWave 2.0.

Materialized ViewsReal-time Feature EngineeringRisingWave

0 likes · 17 min read

Applying RisingWave to Real-Time Feature Engineering: Architecture, Capabilities, and Use Cases

Airbnb Technology Team

Mar 1, 2024 · Big Data

Riverbed: A Scalable Data Framework for Real‑time and Batch Processing at Airbnb

Airbnb’s Riverbed framework unifies streaming CDC events and batch Spark jobs behind a GraphQL‑based declarative API to automatically build and maintain distributed materialized views, using Kafka‑partitioned ordering and version control to deliver billions of daily updates with low‑latency reads for features such as payments and search.

AirbnbApache SparkData Engineering

0 likes · 8 min read

Riverbed: A Scalable Data Framework for Real‑time and Batch Processing at Airbnb

Didi Tech

Feb 27, 2024 · Big Data

Real-time Precise Deduplication Using StarRocks Materialized Views at Didi

Didi leverages StarRocks materialized views with a global dictionary and bitmap aggregation to perform real‑time, high‑cardinality precise deduplication, automatically rewriting queries and refreshing views, cutting query latency by ~80%, reducing resource use ~95%, and boosting concurrent QPS up to 100‑fold, while planning further automation and bitmap optimizations.

Big DataMaterialized ViewsOLAP

0 likes · 19 min read

Real-time Precise Deduplication Using StarRocks Materialized Views at Didi

DataFunSummit

Dec 7, 2023 · Databases

Apache Doris: A High‑Performance Real‑Time Analytical Database for Online High‑Concurrency Reporting

This article introduces Apache Doris, a real‑time analytical database built on an MPP architecture, explains its suitability for massive data workloads and online high‑concurrency reporting scenarios, and details the core technologies—storage models, vectorized query engine, materialized views, partitioning, indexing, row‑store and prepared statements—that enable sub‑second query latency and high QPS, while also showing a real‑world case study and how to join the Doris community.

Apache DorisCommunityData Warehouse

0 likes · 13 min read

Apache Doris: A High‑Performance Real‑Time Analytical Database for Online High‑Concurrency Reporting

DataFunSummit

May 27, 2023 · Big Data

Building and Practicing the Performance Assurance System of YouShu BI

This article presents an in‑depth overview of the YouShu BI product, outlines the high‑concurrency performance challenges faced by enterprise BI, and details the multi‑layer performance architecture—including front‑end, back‑end, data engine, and data source layers—along with smart caching, MPP acceleration, materialized views, and the Data Doctor operations that together ensure low‑latency, reliable analytics for large‑scale users.

BIMPPMaterialized Views

0 likes · 16 min read

Building and Practicing the Performance Assurance System of YouShu BI

DataFunTalk

May 17, 2023 · Databases

Evolution of 360 Commercial Real-Time Data Warehouse and Apache Doris Deployment

This article details the three‑stage evolution of 360's real‑time data warehouse—from Storm + Druid + MySQL to Flink + Druid + TiDB and finally to Flink + Apache Doris—explaining architectural pain points, the reasons for choosing Doris, and how the new system delivers sub‑second query latency, strong consistency, and simplified operations across advertising scenarios.

Apache DorisBig DataFlink

0 likes · 17 min read

Evolution of 360 Commercial Real-Time Data Warehouse and Apache Doris Deployment

DataFunTalk

Feb 4, 2023 · Big Data

Design and Practice of Tencent Lighthouse Fusion Analysis Engine

This article presents the design and implementation of Tencent Lighthouse's Fusion Analysis Engine, covering its background, challenges, fusion architecture, kernel optimizations, acceleration techniques, practical outcomes, and future evolution directions for high‑performance data access.

Big DataFusion EngineMaterialized Views

0 likes · 12 min read

Design and Practice of Tencent Lighthouse Fusion Analysis Engine

Kuaishou Tech

Aug 30, 2021 · Databases

ClickHouse Projection: Design, Implementation, and Production Performance

This article presents an in‑depth overview of ClickHouse Projection, covering its background, definition, practical use cases, underlying architecture, query analysis, consistency guarantees, performance comparisons, and real‑world production results, highlighting how it enhances OLAP workloads while maintaining strong data consistency.

ClickHouseDatabase OptimizationMaterialized Views

0 likes · 19 min read

ClickHouse Projection: Design, Implementation, and Production Performance

Tencent Cloud Developer

Oct 20, 2020 · Databases

ClickHouse: Architecture, Core Features, and Limitations for Interactive Analytics

ClickHouse is a PB‑scale, open‑source columnar OLAP database that uses a ZooKeeper‑coordinated sharded cluster, columnar storage, vectorized execution, advanced compression, data‑skipping indexes, and materialized views to deliver high‑performance interactive analytics, yet it requires manual shard management, lacks a mature MPP optimizer, and handles real‑time single‑row writes poorly.

ClickHouseMaterialized ViewsOLAP

0 likes · 18 min read

ClickHouse: Architecture, Core Features, and Limitations for Interactive Analytics

Architects Research Society

Aug 30, 2020 · Databases

Lessons Learned from Hosting PostgreSQL at Scale and Using It as a Job Queue

This article shares a detailed case study of how a startup managed a massive cinema‑ticket dataset with PostgreSQL, evaluated cloud and self‑hosted options, refined materialized‑view strategies, built a reliable job queue, and distilled practical takeaways for large‑scale database operations.

Cloud HostingDatabase ScalingJob Queue

0 likes · 27 min read

Lessons Learned from Hosting PostgreSQL at Scale and Using It as a Job Queue

Efficient Ops

Jul 24, 2015 · Databases

What MySQL 5.7 InnoDB Experts Reveal: Key Q&A on New Features

An expert panel from the High‑Efficiency Operations community answers eleven pressing MySQL 5.7 questions, covering test hardware, GA timeline, GIS speed, virtual column indexing, materialized view plans, 2‑D geo support, performance monitoring, Oracle’s market strategy, geojson handling, multi‑source replication, buffer‑pool locking, and InnoDB Memcached consistency.

Database PerformanceGISInnoDB

0 likes · 7 min read

What MySQL 5.7 InnoDB Experts Reveal: Key Q&A on New Features