Skip to content
Tagged articles

lakehouse

268 articles · Page 2 of 3
58 Tech
58 Tech
Aug 7, 2025 · Big Data

Transform Real‑Time Data Warehousing with Paimon: From Flink ROW_NUMBER to Streaming Lakehouse

This article details how a real‑time data warehouse built on Flink, Kafka, HBase and MySQL was redesigned using Paimon to eliminate costly deduplication, handle out‑of‑order events, enable streaming reads, simplify aggregation, replace multiple lookup sources, and achieve faster, more reliable batch repairs, resulting in major resource and operational gains.

FlinkPaimondata warehouse
0 likes · 24 min read
Transform Real‑Time Data Warehousing with Paimon: From Flink ROW_NUMBER to Streaming Lakehouse
StarRocks
StarRocks
Jul 23, 2025 · Big Data

How StarRocks Powers Intelligent BI with AI‑Native Lakehouse Architecture

This article explores the evolution of business intelligence toward intelligent BI, detailing traditional BI limitations, agile BI improvements, and how StarRocks' MPP lakehouse engine combined with large language models enables natural‑language analytics, real‑time performance, AI‑driven insights, and scalable enterprise deployments.

AI integrationIntelligent BIStarRocks
0 likes · 19 min read
How StarRocks Powers Intelligent BI with AI‑Native Lakehouse Architecture
DataFunSummit
DataFunSummit
Jul 18, 2025 · Databases

Boosting ClickHouse on WeChat: Performance Tools, Lakehouse Hacks & AI

This article explores how ClickHouse is deployed across WeChat for real‑time analytics, introduces a suite of performance‑monitoring tools, details lakehouse read and bitmap optimizations, and describes the integration of AI‑driven vector search, showcasing substantial speedups and scalability improvements.

AIClickHousebig data
0 likes · 12 min read
Boosting ClickHouse on WeChat: Performance Tools, Lakehouse Hacks & AI
DataFunSummit
DataFunSummit
Jul 18, 2025 · Big Data

Data Lake & Lakehouse Innovations: Real-Time Analytics and Industry Case Studies

This article presents a curated collection of cutting‑edge data lake and lakehouse case studies—including real‑time analytics, cloud‑native architectures, industry implementations from sales platforms to automotive IoT, and the latest advancements in open‑source projects—offering insights into modern big‑data strategies and governance.

Cloud Architecturebig datadata lake
0 likes · 2 min read
Data Lake & Lakehouse Innovations: Real-Time Analytics and Industry Case Studies
DataFunTalk
DataFunTalk
Jul 13, 2025 · Big Data

Unlock Real-Time Multidimensional Insights with Cloud Lakehouse Technology

This guide presents a series of expert case studies and insights on how Cloud Lakehouse solutions enable real‑time, fully managed multidimensional data analysis, improve user data experiences, balance performance and cost, and power large‑scale IoT and big‑data platforms across industries.

IoTdata architecturelakehouse
0 likes · 2 min read
Unlock Real-Time Multidimensional Insights with Cloud Lakehouse Technology
DataFunTalk
DataFunTalk
Jul 9, 2025 · Big Data

How Lakehouse Is Transforming Real‑Time Multi‑Dimensional Analytics

This article compiles a series of expert case studies and insights on real‑time intelligent fully‑managed Lakehouse technology, illustrating how companies such as SalesEasy, Chang’an Auto, Kuaishou, Tencent, and JD.com leverage lakehouse architectures to achieve advanced multi‑dimensional analytics, cost‑performance balance, and effective data governance in the digital economy.

Case Studiesdata architecturedata governance
0 likes · 2 min read
How Lakehouse Is Transforming Real‑Time Multi‑Dimensional Analytics
DataFunTalk
DataFunTalk
Jul 8, 2025 · Big Data

Explore Cutting-Edge Lakehouse Solutions: Real-Time Analytics & Data Governance

This guide presents a curated collection of case studies and insights on cloud-native Lakehouse architectures, real‑time analytics, data‑driven user experiences, and data governance, showcasing implementations from companies like SalesEasy, Changan Auto, TikTok, Tencent, JD.com, and more.

Case Studiesdata governancelakehouse
0 likes · 2 min read
Explore Cutting-Edge Lakehouse Solutions: Real-Time Analytics & Data Governance
DataFunTalk
DataFunTalk
Jul 7, 2025 · Big Data

Unlock Real-Time Analytics with Cloud Lakehouse: A Complete Guide

This article presents a curated list of sessions covering cloud Lakehouse technology for real-time, multidimensional data analysis, including case studies from SalesEasy, Changan Auto, Tencent, and JD, as well as discussions on data lake adoption, streaming lake Paimon, and the relevance of metadata‑driven data governance in the digital economy.

IoTbig datacase study
0 likes · 2 min read
Unlock Real-Time Analytics with Cloud Lakehouse: A Complete Guide
DataFunTalk
DataFunTalk
Jul 6, 2025 · Big Data

How Cloud Lakehouse Is Redefining Real-Time Multi-Dimensional Data Analytics

This article presents a curated list of case studies and insights on cloud Lakehouse technology, covering real-time intelligent analytics, data architecture simplification, IoT big‑data platforms, integrated data platforms, and the evolving role of metadata‑driven data governance in the digital economy.

Case Studiesbig datadata governance
0 likes · 2 min read
How Cloud Lakehouse Is Redefining Real-Time Multi-Dimensional Data Analytics
DataFunSummit
DataFunSummit
Jun 22, 2025 · Databases

Unlocking Apache Doris: How Lakehouse Integration Supercharges Data Analytics

This article walks through Apache Doris’s lakehouse‑in‑one architecture, explains its core value and paradigm, details the system’s components and use cases, examines technical challenges such as file‑format diversity and I/O stability, and presents a suite of optimizations—from predicate push‑down and partition pruning to metadata caching and dynamic scheduling—that dramatically improve query performance and resource utilization, while also outlining future roadmap plans.

Apache Dorisbig datadata warehouse
0 likes · 22 min read
Unlocking Apache Doris: How Lakehouse Integration Supercharges Data Analytics
DataFunSummit
DataFunSummit
Jun 3, 2025 · Big Data

BiFang: A Unified Lake‑Stream Storage Engine for Real‑Time and Batch Data Processing

BiFang is a lake‑stream integrated storage engine that merges Apache Pulsar message‑queue capabilities with Iceberg data‑lake features, providing a single unified data store with full‑incremental queries, sub‑second visibility, exactly‑once semantics, and seamless integration with Flink, Spark, and StarRocks for both real‑time analytics and batch processing.

Apache IcebergApache PulsarData Storage
0 likes · 13 min read
BiFang: A Unified Lake‑Stream Storage Engine for Real‑Time and Batch Data Processing
Big Data Technology & Architecture
Big Data Technology & Architecture
Apr 29, 2025 · Big Data

Big Data Interview Preparation: Data Governance, Iceberg Metadata, Lakehouse Best Practices, and Xiaohongshu HR Updates

The article reports Xiaohongshu’s cancellation of the big‑small week schedule and non‑compete clause, then provides a collection of big‑data interview questions—including data governance, Iceberg metadata management, and lakehouse production best practices—along with concise answers and resources for candidates.

Xiaohongshudata governanceiceberg
0 likes · 7 min read
Big Data Interview Preparation: Data Governance, Iceberg Metadata, Lakehouse Best Practices, and Xiaohongshu HR Updates
Big Data Tech Team
Big Data Tech Team
Mar 25, 2025 · Big Data

How Apache Paimon Transforms Real‑Time Lakehouse Architecture

This article analyzes the limitations of a traditional Flink + Talos + Iceberg real‑time lakehouse, introduces Apache Paimon's lakehouse table format and LSM storage, and demonstrates three practical use cases—partial‑update widening, streaming upsert, and lookup join—showing cost, stability, and performance improvements while outlining future roadmap.

Apache PaimonFlinkStreaming Upsert
0 likes · 16 min read
How Apache Paimon Transforms Real‑Time Lakehouse Architecture
Big Data Technology & Architecture
Big Data Technology & Architecture
Mar 17, 2025 · Big Data

Lakehouse Implementations at Leading Companies: Challenges, Solutions, and Benefits

This article reviews how major tech firms such as Alibaba, Tencent, ByteDance, and Kuaishou tackled lakehouse challenges—including architecture fragmentation, cost, scalability, and complex multimodal data—by adopting real‑time lakehouse solutions like Flink + Paimon, Iceberg + StarRocks, Hudi + LAS, and Doris + Alluxio, and outlines the resulting performance and cost gains.

DorisFlinkPaimon
0 likes · 9 min read
Lakehouse Implementations at Leading Companies: Challenges, Solutions, and Benefits
StarRocks
StarRocks
Feb 20, 2025 · Big Data

How RedBI Boosted Query Speed 3× with StarRocks & Iceberg Lakehouse

The article details how Xiaohongshu's RedBI self‑service analytics platform transformed its architecture by integrating StarRocks and Iceberg, replacing ClickHouse‑based storage with Parquet, introducing DataCache, Z‑Order sorting and intelligent key selection, achieving a three‑fold P90 query speed improvement, sub‑10‑second latency, and halving storage consumption.

DataCacheSelf-Service AnalyticsStarRocks
0 likes · 19 min read
How RedBI Boosted Query Speed 3× with StarRocks & Iceberg Lakehouse
Data Practical Notes
Data Practical Notes
Feb 9, 2025 · Big Data

Data Integration Evolution: 7 Shifts from Tool-Centric to Value-Driven in the Data Element Era

This article traces the evolution of data integration from point-to-point and ETL to modern lakehouse architectures, detailing seven key shifts—tool-to-value orientation, downstream-to-upstream integration, manual-to-automated pipelines, professional-to-citizen access, siloed-to-ecosystem scope, batch-to-real-time processing, and isolated-to-unified storage—driven by data elementization trends.

ETLdata elementizationdata governance
0 likes · 12 min read
Data Integration Evolution: 7 Shifts from Tool-Centric to Value-Driven in the Data Element Era
DataFunSummit
DataFunSummit
Jan 14, 2025 · Big Data

Tencent Real-Time Lakehouse Intelligent Optimization Practice

This presentation details Tencent's real‑time lakehouse architecture and the four key topics—lakehouse design, intelligent optimization services, scenario‑driven capabilities, and future outlook—covering components such as Spark, Flink, Iceberg, Auto‑Optimize Service, indexing, clustering, AutoEngine, and PyIceberg implementations.

Auto OptimizeFlinkSpark
0 likes · 12 min read
Tencent Real-Time Lakehouse Intelligent Optimization Practice
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Jan 14, 2025 · Big Data

How Fluss Unifies Lake and Stream for Real‑Time Analytics: Architecture, Benefits, and Future Roadmap

This article summarizes a talk by Alibaba Cloud senior engineer and Flink Committer Luo Yuxia on the challenges of separating lake and stream storage, introduces the Fluss lake‑stream unified architecture, explains its technical benefits such as second‑level data freshness, unified metadata, efficient changelog generation, and outlines future plans for broader ecosystem integration.

FlinkFlussdata lake
0 likes · 13 min read
How Fluss Unifies Lake and Stream for Real‑Time Analytics: Architecture, Benefits, and Future Roadmap
DataFunSummit
DataFunSummit
Jan 3, 2025 · Big Data

Tencent Real‑Time Lakehouse Intelligent Optimization Practices

This article presents Tencent's end‑to‑end real‑time lakehouse architecture, detailing its three‑layer design, the Auto Optimize Service modules such as compaction, indexing, clustering and engine acceleration, as well as scenario‑driven capabilities like multi‑stream joins, primary‑key tables, in‑place migration and PyIceberg support, and concludes with future optimization directions.

FlinkSparkbig data
0 likes · 11 min read
Tencent Real‑Time Lakehouse Intelligent Optimization Practices
DataFunSummit
DataFunSummit
Dec 27, 2024 · Big Data

Tencent Real-time Lakehouse Intelligent Optimization Practice

This presentation describes Tencent's real-time lakehouse architecture, including data lake compute, management, and storage layers, and details the intelligent optimization services—such as compaction, indexing, clustering, and auto-engine—designed to improve query performance, storage cost, and operational efficiency for large-scale data processing.

AutoEngineFlinkSpark
0 likes · 11 min read
Tencent Real-time Lakehouse Intelligent Optimization Practice
StarRocks
StarRocks
Dec 2, 2024 · Big Data

How Paimon Revamps Lakehouse Management and Supercharges Queries with StarRocks

This article details Tongcheng Travel's migration from Hive/Kudu/Hudi to Paimon for lakehouse integration, highlighting a 30% resource reduction, three‑fold write speed gains, significant query acceleration via StarRocks, the end‑to‑end architecture across ODS‑DWD‑DWS‑ADS layers, and future roadmap plans.

FlinkPaimonStarRocks
0 likes · 18 min read
How Paimon Revamps Lakehouse Management and Supercharges Queries with StarRocks
DataFunSummit
DataFunSummit
Dec 2, 2024 · Big Data

Gravitino Powers TBDS Product Architecture Upgrade with a Unified Metadata Lake

This article explains how Tencent Cloud's TBDS platform evolves its architecture by adopting Apache Gravitino as a unified metadata lake, detailing the challenges of legacy versus new lakehouse designs, storage and compute separation, unified data access, permission management, and the resulting benefits for big‑data and AI workloads.

GravitinoTBDSbig data
0 likes · 15 min read
Gravitino Powers TBDS Product Architecture Upgrade with a Unified Metadata Lake
Big Data Technology & Architecture
Big Data Technology & Architecture
Nov 1, 2024 · Big Data

Real‑Time Lakehouse Architecture at Ximalaya Live: Leveraging Flink, Paimon, and StarRocks

This article details Ximalaya Live's transition from an offline‑centric data warehouse to a real‑time lakehouse using Flink, Paimon, and StarRocks, covering business background, architectural challenges, technology evaluation, implementation steps, encountered issues, performance gains, and future expansion plans.

FlinkPaimonReal-time
0 likes · 12 min read
Real‑Time Lakehouse Architecture at Ximalaya Live: Leveraging Flink, Paimon, and StarRocks
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Oct 31, 2024 · Big Data

How EMR Serverless Spark Powers the Next‑Gen Lakehouse Era

This article traces the evolution of data platforms, explains the rise of lakehouse architecture, and details how Alibaba Cloud's EMR Serverless Spark delivers one‑stop development, high performance, and full ecosystem compatibility, illustrated with real‑world case studies from Midea and Eagle Network.

AIEMR Serverless Sparkbig data
0 likes · 16 min read
How EMR Serverless Spark Powers the Next‑Gen Lakehouse Era
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Oct 25, 2024 · Big Data

How Real-Time Flink Powers Automotive Big Data: Architecture & Case Studies

This article, based on Alibaba Cloud expert Li Lubing’s presentation, examines the rapid growth of China’s new energy vehicle market, outlines typical automotive big‑data architectures, compares Lambda and real‑time lakehouse solutions built with Flink and Apache Paimon, and showcases real‑world customer deployments.

AutomotiveFlinkbig data
0 likes · 18 min read
How Real-Time Flink Powers Automotive Big Data: Architecture & Case Studies
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 22, 2024 · Big Data

Key Frameworks and Characteristics of Lakehouse Architecture: A Ground‑Level Perspective

This article reviews the emerging lakehouse architecture, outlines its core frameworks such as Hudi, Iceberg, Paimon, Flink, and Doris, discusses their storage‑compute separation, read‑write optimizations, and highlights how companies of different sizes adopt these technologies based on cost, efficiency, and specific business scenarios.

DorisFlinkOLAP
0 likes · 6 min read
Key Frameworks and Characteristics of Lakehouse Architecture: A Ground‑Level Perspective
StarRocks
StarRocks
Oct 16, 2024 · Big Data

How to Build a High‑Performance Lakehouse with StarRocks and Apache Hive

This guide walks through the core concepts of Apache Hive, its architecture and key features, then shows how to integrate Hive with StarRocks via the Hive Catalog, construct ODS/DWD/DWS/ADS tables, enable DataCache, use materialized views, and handle automatic partition detection for fast lakehouse analytics.

Apache HiveDataCacheMaterialized View
0 likes · 17 min read
How to Build a High‑Performance Lakehouse with StarRocks and Apache Hive
DataFunTalk
DataFunTalk
Oct 3, 2024 · Big Data

Data Lake Technology Maturity Curve: Architecture, Design Principles, Core Functions, and Open‑Source Solutions

Amid growing data demands, this article explains the data lake technology maturity curve, detailing lake‑warehouse architectural patterns, design principles, core functionalities, and the four leading open‑source solutions (Hudi, Iceberg, Delta Lake, Paimon) to guide enterprises in building flexible, scalable, and governed data platforms.

Delta LakeHudiPaimon
0 likes · 10 min read
Data Lake Technology Maturity Curve: Architecture, Design Principles, Core Functions, and Open‑Source Solutions
DataFunTalk
DataFunTalk
Sep 24, 2024 · Big Data

Data Lake Technology Maturity Curve: Architecture Modes, Design Principles, Core Functions, and Applications

This article explains the rapid growth of data-driven businesses, the challenges of traditional data warehouses, and how modern data lake technologies such as Delta Lake, Hudi, Iceberg, and Paimon form a maturity curve that guides enterprises in architecture choices, design principles, core capabilities, and practical applications.

Delta LakeHudiPaimon
0 likes · 12 min read
Data Lake Technology Maturity Curve: Architecture Modes, Design Principles, Core Functions, and Applications
Sohu Tech Products
Sohu Tech Products
Sep 11, 2024 · Big Data

Tencent Real-time Lakehouse Intelligent Optimization Practice

Tencent’s real‑time lakehouse combines Spark, Flink, StarRocks and Presto compute layers with Iceberg‑based management and HDFS/COS storage, and its Intelligent Optimize Service—comprising Compaction, Expiration, Cleaning, Clustering, Index and Auto‑Engine modules—automatically reduces merge time, improves query performance, enables secondary indexing, and dynamically routes hot partitions, while future plans target cold/hot separation, materialized view acceleration, and AI‑driven optimizations.

IndexingPyIcebergTencent
0 likes · 12 min read
Tencent Real-time Lakehouse Intelligent Optimization Practice
ITPUB
ITPUB
Sep 11, 2024 · Big Data

Is Storage‑Compute Separation the Future? Unpacking the Lakehouse Debate

The article examines the concepts of storage‑compute separation and the lake‑warehouse (lakehouse) model, tracing their evolution from physical Hadoop clusters to containerized compute and object storage, and argues that true separation requires MPP systems to adopt open standards, effectively merging lake and warehouse architectures.

HadoopMPPStorage-Compute Separation
0 likes · 7 min read
Is Storage‑Compute Separation the Future? Unpacking the Lakehouse Debate
StarRocks
StarRocks
Sep 5, 2024 · Big Data

Accelerate Lakehouse Queries: A Hands‑On Guide to StarRocks + Apache Iceberg

This tutorial walks you through the fundamentals of Apache Iceberg, its architecture and key features, explains why it’s advantageous for lakehouse workloads, and provides a step‑by‑step Docker‑Compose setup to integrate Iceberg with StarRocks for fast, ACID‑compliant analytics on real‑world taxi data.

Apache IcebergData EngineeringDocker
0 likes · 15 min read
Accelerate Lakehouse Queries: A Hands‑On Guide to StarRocks + Apache Iceberg
DataFunSummit
DataFunSummit
Aug 26, 2024 · Big Data

Building a Doris‑Based Lakehouse Integrated Analytics System at Kuaishou

This article presents Kuaishou's experience of designing and implementing a Doris‑driven lakehouse integrated analytics system, covering the current OLAP landscape, challenges of data duplication and governance, the new architecture with caching and auto‑materialization, implementation details, performance impact, and future work.

Auto MaterializationDorisOLAP
0 likes · 24 min read
Building a Doris‑Based Lakehouse Integrated Analytics System at Kuaishou
StarRocks
StarRocks
Aug 14, 2024 · Big Data

Mastering StarRocks & Apache Paimon: A Fast‑Track Lakehouse Guide

This guide provides a comprehensive overview of Apache Paimon’s architecture, key features, and advantages, explains how to integrate it with StarRocks for real‑time lakehouse analytics, and walks through a complete quick‑start setup including component installation, Flink and Kafka deployment, data ingestion, table creation, and query execution with time‑travel support.

Apache PaimonData EngineeringFlink
0 likes · 18 min read
Mastering StarRocks & Apache Paimon: A Fast‑Track Lakehouse Guide
DataFunSummit
DataFunSummit
Aug 6, 2024 · Big Data

Implementing a Multi‑Tenant Lakehouse Data Platform for Real‑Time Analytics at a SaaS CRM Company

This article details how a SaaS CRM provider built a cloud‑native Lakehouse platform to support multi‑tenant real‑time analytics, describing data challenges, metadata‑driven architecture, virtual database design, query optimization, BI integration, AI readiness, migration steps, and the resulting performance and scalability gains.

SaaS CRMbig datadata platform
0 likes · 19 min read
Implementing a Multi‑Tenant Lakehouse Data Platform for Real‑Time Analytics at a SaaS CRM Company
Wukong Talks Architecture
Wukong Talks Architecture
Aug 6, 2024 · Databases

Migrating Tencent Music's Data Infrastructure from ClickHouse and Druid to StarRocks: Strategy, Implementation, and Best Practices

This article details how Tencent Music’s data‑infrastructure team migrated thousands of ClickHouse and Druid nodes to a StarRocks compute‑storage‑separated lakehouse, achieving 40‑50% cost reduction while maintaining query performance, and shares the technical challenges, solutions, and best‑practice recommendations gathered during the process.

ClickHouseDruidStarRocks
0 likes · 19 min read
Migrating Tencent Music's Data Infrastructure from ClickHouse and Druid to StarRocks: Strategy, Implementation, and Best Practices
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Jul 22, 2024 · Databases

Why StarRocks Is Redefining Fast Unified OLAP Analytics

StarRocks combines vectorized execution, a new cost‑based optimizer, materialized views, a real‑time storage engine, pipeline execution, and distributed joins to deliver a unified, high‑performance OLAP solution that supports both traditional and lakehouse analytics while reducing operational complexity.

CBOOLAPStarRocks
0 likes · 14 min read
Why StarRocks Is Redefining Fast Unified OLAP Analytics
DataFunSummit
DataFunSummit
Jul 12, 2024 · Big Data

Data Lake Development Trends, Architecture, Integration, Lakehouse Core Capabilities, and Open Design

This article examines the current evolution of data lakes, detailing their overall architecture, batch and real‑time integration methods, Lakehouse core functionalities such as enhanced DML, schema evolution, ACID support, and open‑design principles that enable multi‑cloud deployment and seamless interaction with diverse compute engines.

Open Data Formatsbatch processingbig data architecture
0 likes · 12 min read
Data Lake Development Trends, Architecture, Integration, Lakehouse Core Capabilities, and Open Design
StarRocks
StarRocks
Jul 2, 2024 · Big Data

What’s New in StarRocks 3.3? Deep Dive into Lakehouse‑Optimized Performance and Features

StarRocks 3.3 introduces a comprehensive set of enhancements—including maturity levels, ARM‑optimized performance, advanced caching, materialized‑view rewrites, storage optimizations, and expanded lakehouse ecosystem support—that together boost stability, query speed, and usability for large‑scale analytics workloads.

Materialized ViewQuery PerformanceStarRocks
0 likes · 15 min read
What’s New in StarRocks 3.3? Deep Dive into Lakehouse‑Optimized Performance and Features
DataFunTalk
DataFunTalk
Jul 1, 2024 · Big Data

DataFunCon2024 Beijing: Real‑Time Lakehouse and Big Data Sessions

The DataFunCon2024 Beijing conference on July 5‑6 showcases a series of technical talks about real‑time lakehouse architectures, big‑data analytics, and cloud‑native data warehouses, offering practitioners insights into Apache Paimon, SelectDB, and Doris implementations for faster, more agile data processing.

Apache PaimonSelectDBlakehouse
0 likes · 8 min read
DataFunCon2024 Beijing: Real‑Time Lakehouse and Big Data Sessions
DataFunTalk
DataFunTalk
Jun 10, 2024 · Big Data

Data Lake Development Trends, Architecture, Integration, and Lakehouse Core Capabilities

This article reviews the latest developments in data lakes, including trend analysis, overall architecture, data integration methods, Lakehouse core capabilities, open design principles, stream‑batch unified processing, real‑time OLAP, and lake‑internal warehousing, highlighting how these advances reduce complexity and cost while improving data sharing and performance.

Real-time OLAPStream-Batch Integrationbig data architecture
0 likes · 14 min read
Data Lake Development Trends, Architecture, Integration, and Lakehouse Core Capabilities
DataFunTalk
DataFunTalk
Jun 9, 2024 · Big Data

Optimizing ClickHouse Performance in WeChat: Observation Tools, Lakehouse Reading, Bitmap Acceleration, and AI Integration

This article details how the WeChat team leverages ClickHouse at massive scale, introduces a suite of performance observation tools, describes lakehouse reading and bitmap optimizations, and explains the integration of AI workloads, demonstrating overall query speedups of up to tenfold across diverse scenarios.

BitmapClickHousebig data
0 likes · 10 min read
Optimizing ClickHouse Performance in WeChat: Observation Tools, Lakehouse Reading, Bitmap Acceleration, and AI Integration
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Jun 6, 2024 · Databases

How StarRocks Redefines Lakehouse Architecture with Ultra-Fast Unified Analytics

StarRocks combines extreme query speed and a unified architecture to deliver a lakehouse solution that separates storage and compute, supports multi‑warehouse resource isolation, offers Trino compatibility, materialized‑view acceleration, and cost‑effective scaling, making it suitable for real‑time analytics, data‑lake queries, and traditional OLAP workloads.

StarRocksStorage-Compute Separationbig data
0 likes · 23 min read
How StarRocks Redefines Lakehouse Architecture with Ultra-Fast Unified Analytics
StarRocks
StarRocks
May 14, 2024 · Artificial Intelligence

How Tencent Games Boosted AI‑Generated SQL Accuracy to 89% with a Lakehouse Architecture

Tencent Games tackled the low accuracy of AI‑generated SQL in production by combining large language models with a StarRocks lake‑warehouse, introducing a semantic layer, async materialized views, and an agent‑based multi‑intelligence framework, ultimately raising one‑shot SQL correctness to 89% and cutting delivery time from 2 hours to 0.33 hours.

AIData EngineeringLLM
0 likes · 13 min read
How Tencent Games Boosted AI‑Generated SQL Accuracy to 89% with a Lakehouse Architecture
DataFunSummit
DataFunSummit
May 12, 2024 · Big Data

Practice of Lakehouse‑Integrated Data Platform Architecture in the Financial Innovation Sector

This article presents the evolution of data platform architectures, the specific challenges of financial‑sector information‑technology innovation, and the design, core components, deployment path, and real‑world case studies of the cloud‑native lakehouse solution DataCyber developed by Shuxin Network.

Financial Innovationbig datadata platform
0 likes · 21 min read
Practice of Lakehouse‑Integrated Data Platform Architecture in the Financial Innovation Sector
DataFunSummit
DataFunSummit
May 5, 2024 · Big Data

Alluxio in Lakehouse Architecture: Benefits, Challenges, and Real‑World Use Cases

This article explains how Alluxio enables a unified lake‑warehouse architecture by decoupling compute and storage, outlines its core capabilities, evaluates the cost‑saving and performance benefits, discusses the technical challenges, and presents several practical deployment scenarios in finance and AI workloads.

Alluxiobig datadata orchestration
0 likes · 15 min read
Alluxio in Lakehouse Architecture: Benefits, Challenges, and Real‑World Use Cases
DataFunSummit
DataFunSummit
Apr 25, 2024 · Big Data

Paimon Project Overview: Recent Developments, Core Capabilities, and Future Roadmap

This article presents a comprehensive overview of the Apache‑incubated Paimon project, covering its evolution from Flink Table Store, the current features of primary‑key and log tables, management tools such as snapshots, tags and branches, performance optimizations for Flink and Spark, and a detailed roadmap of upcoming functionalities.

Data ManagementFlinkPaimon
0 likes · 23 min read
Paimon Project Overview: Recent Developments, Core Capabilities, and Future Roadmap
DataFunTalk
DataFunTalk
Apr 23, 2024 · Big Data

Apache Paimon Graduates to Top‑Level Project – Milestones, Core Capabilities, and Community Highlights

Apache Paimon, originally launched as Flink Table Store, has graduated to an Apache Top‑Level Project after a year of incubation, showcasing real‑time lakehouse capabilities, extensive ecosystem integration, and strong adoption by major enterprises, marking a significant milestone for streaming and batch data processing.

Apache PaimonReal-time Data Lakebig data
0 likes · 9 min read
Apache Paimon Graduates to Top‑Level Project – Milestones, Core Capabilities, and Community Highlights
DataFunTalk
DataFunTalk
Apr 20, 2024 · Big Data

Tencent Video Metrics Middle Platform and Lakehouse Integration: Architecture, Governance, and Practices

This article details Tencent Video’s data business, describing the design and implementation of its metrics middle platform and lake‑warehouse integration, covering architecture, governance, consistency, timeliness, usability, cost optimization, and future plans, with insights into technology choices such as Iceberg, StarRocks, and MQL.

Data EngineeringMetrics PlatformStarRocks
0 likes · 18 min read
Tencent Video Metrics Middle Platform and Lakehouse Integration: Architecture, Governance, and Practices
DataFunTalk
DataFunTalk
Mar 16, 2024 · Big Data

Performance Optimization Practices for KwaiBI Big Data Analysis Platform

This article introduces KwaiBI, the internal data analysis product of Kuaishou, outlines its five major functional areas, details the performance challenges of large‑scale analytics, and presents a comprehensive set of optimization techniques—including cache warming, query rewriting, materialized acceleration, and the Bleem lake‑house engine—along with future directions and a brief Q&A.

KwaiBIbig datadata analytics
0 likes · 15 min read
Performance Optimization Practices for KwaiBI Big Data Analysis Platform
DataFunSummit
DataFunSummit
Mar 14, 2024 · Big Data

Tencent Game Data Analysis: Lakehouse Integration Practice

This article presents Tencent Game's comprehensive lakehouse integration practice, detailing the project background, storage‑compute separation, data layering, unified DDL/DML operations, performance optimizations, and future plans, illustrating how StarRocks, Iceberg, and Spark are combined to achieve scalable, cost‑effective analytics for massive game data.

StarRockscompute-storage separationdata warehouse
0 likes · 16 min read
Tencent Game Data Analysis: Lakehouse Integration Practice
DataFunTalk
DataFunTalk
Mar 4, 2024 · Big Data

Design and Implementation of a Lakehouse‑Integrated Data Platform for Financial Innovation by Shuxin Network

This article presents Shuxin Network's practical experience in building a cloud‑native, lakehouse‑integrated data platform for the financial sector, covering architecture evolution, challenges of domestic‑innovation (信创), the DataCyber solution, core components, deployment roadmap, and real‑world case studies.

Financial InnovationSecuritybig data
0 likes · 21 min read
Design and Implementation of a Lakehouse‑Integrated Data Platform for Financial Innovation by Shuxin Network
DataFunSummit
DataFunSummit
Feb 26, 2024 · Big Data

Building a New Lakehouse Analytics Paradigm with StarRocks and Paimon

This article introduces a new lakehouse analytics paradigm by combining StarRocks and Paimon, covering the evolution of data lake technologies, key integration scenarios, core technical mechanisms such as JNI connectors, materialized views, and future roadmap for enhanced lakehouse capabilities.

AnalyticsPaimonSQL Engine
0 likes · 16 min read
Building a New Lakehouse Analytics Paradigm with StarRocks and Paimon
DataFunTalk
DataFunTalk
Feb 9, 2024 · Big Data

Alluxio’s Role in Lakehouse Architecture: Benefits, Challenges, and Real‑World Use Cases

This article explains how Alluxio enables lake‑warehouse integration by providing a data orchestration layer that caches data near compute, reduces storage‑compute separation costs, improves performance, and addresses challenges such as security, scalability, and multi‑cloud deployment, illustrated with several industry case studies.

AIAlluxioCaching
0 likes · 16 min read
Alluxio’s Role in Lakehouse Architecture: Benefits, Challenges, and Real‑World Use Cases
DataFunTalk
DataFunTalk
Jan 27, 2024 · Big Data

JuiceFS: A Cloud‑Native Distributed File System for Data Lake and Lakehouse

This article presents JuiceFS, a cloud‑native distributed file system that bridges the gaps between HDFS and object storage, explaining Data Lake and Lakehouse concepts, comparing storage options, detailing JuiceFS's architecture and performance benefits, and showcasing real‑world user case studies.

JuiceFSObject Storagebig data
0 likes · 23 min read
JuiceFS: A Cloud‑Native Distributed File System for Data Lake and Lakehouse
Past Memory Big Data
Past Memory Big Data
Jan 17, 2024 · Big Data

How WeChat Implements a StarRocks‑Powered Lakehouse Across Multiple Business Scenarios

WeChat evolved its data platform from Hadoop to ClickHouse and finally to a StarRocks‑based lakehouse, solving data fragmentation and storage redundancy while achieving sub‑second to minute‑level query latency, cutting storage costs by over 65%, halving operational tasks, and reducing offline job time by two hours across several business lines.

StarRocksWeChatbig data
0 likes · 16 min read
How WeChat Implements a StarRocks‑Powered Lakehouse Across Multiple Business Scenarios
DataFunSummit
DataFunSummit
Jan 9, 2024 · Big Data

Introducing Yunqi Lakehouse: An Integrated Cloud‑Native Data Platform with Incremental Computing and Auto Materialized Views

This article introduces Yunqi's self‑developed Lakehouse product, explaining its cloud‑native, one‑stop data platform architecture, incremental computing that balances freshness, performance and cost, and the autoMV feature that automatically creates materialized views to boost query speed up to nine times.

Auto Materialized Viewbig datadata platform
0 likes · 14 min read
Introducing Yunqi Lakehouse: An Integrated Cloud‑Native Data Platform with Incremental Computing and Auto Materialized Views
Tongcheng Travel Technology Center
Tongcheng Travel Technology Center
Dec 27, 2023 · Big Data

Recap of Tongcheng Travel’s 7th Big Data Technology Salon – Talks on StarRocks, Paimon, Iceberg, Data+AI, Vector Retrieval, Real‑Time Computing, and Hotel Ranking

The 7th Tongcheng Travel Big Data Technology Salon in Beijing featured a series of expert talks covering StarRocks architecture evolution, lake‑house solutions with Paimon, Iceberg real‑time upsert, Data+AI for travel recommendation, vector retrieval in AI, JD Logistics real‑time computing governance, and multi‑task hotel ranking modeling, providing deep technical insights and future roadmaps.

AIPaimonReal-time Computing
0 likes · 10 min read
Recap of Tongcheng Travel’s 7th Big Data Technology Salon – Talks on StarRocks, Paimon, Iceberg, Data+AI, Vector Retrieval, Real‑Time Computing, and Hotel Ranking
DataFunTalk
DataFunTalk
Dec 27, 2023 · Big Data

Amoro Mixed Hive: A Unified Lakehouse Solution for Real‑Time and Batch Data Processing

This article describes how NetEase Youdao replaced its Doris‑based real‑time data warehouse with Amoro Mixed Hive, detailing the architectural challenges, the Mixed Hive design, implementation steps, performance optimizations, community contributions, and future roadmap to achieve a unified lakehouse with minute‑level freshness and reduced development and operational costs.

AmoroFlinkHive
0 likes · 12 min read
Amoro Mixed Hive: A Unified Lakehouse Solution for Real‑Time and Batch Data Processing
DataFunTalk
DataFunTalk
Dec 27, 2023 · Big Data

Apache Flink 2023: Core Technical Achievements and Future Directions

The article reviews Apache Flink's rapid development over the past decade, highlighting its 2023 community growth, SIGMOD award, major releases, streaming SQL enhancements, incremental checkpointing, batch maturity, cloud‑native scaling, and integration with the emerging Lakehouse architecture.

Apache FlinkSQLbig data
0 likes · 11 min read
Apache Flink 2023: Core Technical Achievements and Future Directions
DataFunSummit
DataFunSummit
Dec 20, 2023 · Cloud Native

Building a Cloud‑Native Lakehouse with Apache Iceberg and Amoro

This article introduces the background, challenges, and cloud‑native solutions of lakehouse architecture, explains Apache Iceberg’s open table format and its cloud‑native features, details Amoro’s management and self‑optimizing capabilities, showcases three real‑world cloud migration cases, and outlines future development plans.

AmoroApache IcebergData Management
0 likes · 12 min read
Building a Cloud‑Native Lakehouse with Apache Iceberg and Amoro
StarRocks
StarRocks
Dec 19, 2023 · Big Data

How WeChat Achieved Sub‑Second Real‑Time Analytics with StarRocks Lakehouse

WeChat transformed its data platform from Hadoop and ClickHouse to a StarRocks‑based lakehouse, tackling massive data volume, ultra‑low latency, and storage fragmentation by deploying lake‑on‑warehouse and warehouse‑lake fusion architectures, real‑time incremental materialized views, and unified SQL access, resulting in dramatic cost cuts and performance gains.

Materialized ViewStarRocksWeChat
0 likes · 15 min read
How WeChat Achieved Sub‑Second Real‑Time Analytics with StarRocks Lakehouse
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Dec 8, 2023 · Cloud Computing

How Alibaba Cloud EMR Powers Serverless StarRocks for Seamless Lakehouse Analytics

This article summarizes Li Yu's presentation on Alibaba Cloud EMR's deep collaboration with the StarRocks community, detailing major contributions across versions, the serverless StarRocks product’s core capabilities, and future plans to enhance OLAP‑lakehouse integration, performance, and cloud‑native elasticity.

Alibaba CloudEMROLAP
0 likes · 7 min read
How Alibaba Cloud EMR Powers Serverless StarRocks for Seamless Lakehouse Analytics
StarRocks
StarRocks
Nov 23, 2023 · Databases

How StarRocks Redefines Lakehouse Architecture with Compute‑Storage Separation

StarRocks, an open‑source MPP analytical database, consolidates BI, interactive, and real‑time analytics into a single engine by evolving from version 1.0 to 3.x, introducing compute‑storage separation, unified catalog, generated columns, operator spill, and advanced materialized views, while outlining its cloud‑native lakehouse roadmap.

MPP databaseStarRockscompute-storage separation
0 likes · 22 min read
How StarRocks Redefines Lakehouse Architecture with Compute‑Storage Separation
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Nov 23, 2023 · Big Data

Why Apache Paimon Is Revolutionizing Streaming Lakehouse Architecture with Flink

The article traces the shift from traditional Hive‑based warehouses to modern lakehouse architectures, explains the advantages of lake formats, introduces Apache Paimon as a streaming‑first data lake integrated with Flink, presents performance benchmarks showing its superiority over Hudi, and demonstrates a real‑time streaming lakehouse workflow.

Apache PaimonFlinkbig data
0 likes · 15 min read
Why Apache Paimon Is Revolutionizing Streaming Lakehouse Architecture with Flink
Big Data Technology Architecture
Big Data Technology Architecture
Nov 14, 2023 · Big Data

Open Source Big Data Platform 3.0: Streaming Lakehouse, Serverless Architecture, and AI Integration

The talk outlines the evolution of Alibaba Cloud's open‑source big data platform from Hadoop‑based EMR to a 3.0 architecture featuring a streaming lakehouse, full serverless compute and storage, AI‑driven operations, and upcoming vector search services, highlighting technical motivations, challenges, and product releases.

big datacloud-nativelakehouse
0 likes · 14 min read
Open Source Big Data Platform 3.0: Streaming Lakehouse, Serverless Architecture, and AI Integration
dbaplus Community
dbaplus Community
Nov 8, 2023 · Big Data

Choosing Between Data Warehouse, Data Lake, and Lakehouse: When to Use Each

This article compares traditional data warehouses, modern data lakes, and emerging lakehouse architectures, explaining their design patterns, advantages, disadvantages, and suitable use cases, while detailing implementation considerations such as schema design, ETL/ELT processes, file formats like Delta, Iceberg, and Hudi, and factors influencing platform selection.

Apache SparkDelta LakeETL
0 likes · 20 min read
Choosing Between Data Warehouse, Data Lake, and Lakehouse: When to Use Each
StarRocks
StarRocks
Oct 31, 2023 · Databases

How Ctrip Accelerated Report Queries 10× with StarRocks: A Real‑World Lakehouse Migration

Ctrip migrated its Artnova reporting platform from Hive‑based queries to StarRocks, first loading data into OLAP tables and then using StarRocks as a lakehouse with Hive catalog, Data Cache and materialized views, achieving average query latency reductions from 20 seconds to 1.5 seconds, over 7× speed‑up versus Trino and up to 40× acceleration for complex workloads.

Data CacheMaterialized ViewOLAP
0 likes · 15 min read
How Ctrip Accelerated Report Queries 10× with StarRocks: A Real‑World Lakehouse Migration
DataFunSummit
DataFunSummit
Oct 16, 2023 · Big Data

Bilibili's Iceberg‑Based Lakehouse Platform: Technical Practices for Sub‑Second Query Response

This article details Bilibili's implementation of an Iceberg‑based lakehouse platform that unifies storage and analytics, addressing Hive’s performance and latency issues through multidimensional sorting, various file‑level indexes, cube pre‑aggregation, star‑tree structures, and an automated Magnus service for intelligent optimization, achieving near‑second query responses.

OLAPQuery Accelerationbig data
0 likes · 14 min read
Bilibili's Iceberg‑Based Lakehouse Platform: Technical Practices for Sub‑Second Query Response
Sohu Tech Products
Sohu Tech Products
Oct 11, 2023 · Industry Insights

How StarRocks Materialized Views Power Real‑Time Lakehouse Analytics

The article provides a deep technical overview of StarRocks 3.0’s data‑lake analysis capabilities, its unified Lakehouse architecture, Catalog integration, Trino compatibility, extensive I/O optimizations, materialized view features, resource isolation techniques, real‑world use cases, and future development directions.

AnalyticsMaterialized ViewStarRocks
0 likes · 22 min read
How StarRocks Materialized Views Power Real‑Time Lakehouse Analytics
DataFunTalk
DataFunTalk
Oct 5, 2023 · Big Data

Building a Unified Streaming‑Batch Lakehouse with Amoro Mixed Iceberg

This article describes how Shanghai Steel Union leveraged Amoro Mixed Iceberg on top of Apache Iceberg to create a unified streaming‑batch lakehouse, addressing small‑file and upsert challenges, simplifying architecture, improving data freshness, and providing a scalable solution for real‑time and batch analytics.

AmoroApache IcebergData Management
0 likes · 13 min read
Building a Unified Streaming‑Batch Lakehouse with Amoro Mixed Iceberg
DataFunSummit
DataFunSummit
Sep 25, 2023 · Big Data

Trino in Bilibili Lakehouse: Compute Engine, Stability, and Containerization Practices

This article presents Bilibili's practical implementation of Trino within a lakehouse architecture, focusing on the compute engine placement, stability enhancements, and containerized deployment, while detailing indexing strategies, pre‑computation techniques, Iceberg metadata optimizations, and performance gains for large‑scale analytical queries.

ContainerizationIndexingPrecomputation
0 likes · 14 min read
Trino in Bilibili Lakehouse: Compute Engine, Stability, and Containerization Practices
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 18, 2023 · Big Data

Unified Real‑Time and Batch Data Warehouse Architecture with Hudi Lakehouse

The article explains the mainstream Lambda data‑warehouse architecture, its benefits and challenges, then introduces Hudi as a lake‑house solution that unifies real‑time and offline storage, describes the multi‑layer service design, and showcases three practical scenarios—stream processing, real‑time multidimensional analysis, and stream‑batch data reuse—demonstrating how the integrated architecture improves latency, cost, and operational complexity.

HudiLambda architecturebatch processing
0 likes · 13 min read
Unified Real‑Time and Batch Data Warehouse Architecture with Hudi Lakehouse
DataFunTalk
DataFunTalk
Sep 16, 2023 · Big Data

StarRocks Data Lake Analysis, Materialized Views, and Lakehouse Architecture

This article explains how StarRocks 3.0 extends real‑time data‑warehouse capabilities to support data‑lake analysis, external catalog integration, Trino compatibility, extensive I/O optimizations, and powerful materialized‑view features that together enable a unified, cloud‑native Lakehouse solution with high performance and flexible resource isolation.

SQLStarRocksbig data
0 likes · 20 min read
StarRocks Data Lake Analysis, Materialized Views, and Lakehouse Architecture
DataFunSummit
DataFunSummit
Sep 8, 2023 · Big Data

Tianqiong OLAP Real‑time Lakehouse Fusion Platform Architecture Practice

This article explains why lake‑warehouse fusion is needed, describes the challenges of integrating real‑time data warehouses with data lakes, introduces a new StarRocks‑based architecture that supports real‑time ingestion, cooling, offline loading, and adaptive hot‑cold query rewriting, and outlines future plans and Q&A.

Real-time OLAPStarRocksbig data
0 likes · 21 min read
Tianqiong OLAP Real‑time Lakehouse Fusion Platform Architecture Practice
StarRocks
StarRocks
Sep 6, 2023 · Big Data

How Paimon + StarRocks Revolutionize Lakehouse Analytics

This article reviews traditional Lambda and Kappa data‑warehouse architectures, then details four Paimon‑StarRocks lakehouse solutions—including a data‑lake center, accelerated query with materialized views, hot‑cold data separation, and the JNI connector—while also outlining StarRocks’ future roadmap for lakehouse analytics.

Materialized ViewPaimonStarRocks
0 likes · 11 min read
How Paimon + StarRocks Revolutionize Lakehouse Analytics
DataFunTalk
DataFunTalk
Sep 4, 2023 · Big Data

Unified Batch‑Stream Storage with Hudi and LAS: Architecture, Design, and Deployment

This article presents a comprehensive overview of a batch‑stream unified storage solution built on Hudi and the Lakehouse Analysis Service (LAS), covering background challenges, architectural design, data organization, read/write mechanisms, BTS architecture, real‑world deployment scenarios, and future development plans.

Batch-StreamHudiLAS
0 likes · 22 min read
Unified Batch‑Stream Storage with Hudi and LAS: Architecture, Design, and Deployment
Data Thinking Notes
Data Thinking Notes
Aug 27, 2023 · Big Data

How ByteDance’s LAS Team Unified Real‑Time and Offline Warehousing with a Lakehouse Solution

This article analyzes the shortcomings of mainstream Lambda‑style data warehouse architectures, introduces Hudi‑based lakehouse design principles, details the three‑layer unified storage architecture, data distribution, model and read/write mechanisms, and showcases real‑time streaming, multidimensional analysis, and stream‑batch reuse scenarios along with future roadmap plans.

Hudilakehousestreaming
0 likes · 14 min read
How ByteDance’s LAS Team Unified Real‑Time and Offline Warehousing with a Lakehouse Solution
Tencent Cloud Developer
Tencent Cloud Developer
Aug 23, 2023 · Big Data

WeChat Experiment Platform: Architecture Design and Iceberg Lakehouse Optimization

The WeChat Experiment Platform migrated its 60,000 metric, 200,000 core, 30 PB plus data pipeline to an Iceberg based lakehouse, leveraging three layer metadata, fine grained partitioning, MERGE into writes, time travel snapshots and skew handling UDFs, which cut core time by 69%, saved ~100 PB storage, and reduced latency by up to 70%.

Merge IntoMetric ComputationSpark
0 likes · 18 min read
WeChat Experiment Platform: Architecture Design and Iceberg Lakehouse Optimization