Tagged articles

11 articles

Page 1 of 1

Sep 25, 2023 · Big Data

Trino in Bilibili Lakehouse: Compute Engine, Stability, and Containerization Practices

This article presents Bilibili's practical implementation of Trino within a lakehouse architecture, focusing on the compute engine placement, stability enhancements, and containerized deployment, while detailing indexing strategies, pre‑computation techniques, Iceberg metadata optimizations, and performance gains for large‑scale analytical queries.

IcebergLakehousePrecomputation

0 likes · 14 min read

Trino in Bilibili Lakehouse: Compute Engine, Stability, and Containerization Practices

ITPUB

Jan 20, 2023 · Big Data

How Bilibili Supercharged OLAP Queries with Iceberg Lakehouse Optimizations

This article details Bilibili's practical deployment of an Iceberg lake‑warehouse architecture within its OLAP platform, covering the motivations for lake‑warehouse integration, core Iceberg optimizations such as data‑organization sorting, Z‑order and secondary indexes, the Magnus intelligent management platform, and future roadmap plans.

Precomputationbigdataindexing

0 likes · 16 min read

How Bilibili Supercharged OLAP Queries with Iceberg Lakehouse Optimizations

IT Architects Alliance

May 19, 2022 · Big Data

How Apache Kylin Enables Sub‑Second OLAP on Massive Data Sets

Apache Kylin leverages pre‑computed OLAP cubes on Hadoop/Spark/Flink to deliver sub‑second query responses for massive datasets, detailing its architecture, integration with BI platforms, user security, cube building, monitoring, and storage using HBase, illustrating how it overcomes big‑data analytical challenges.

Apache KylinBig DataData Warehouse

0 likes · 12 min read

How Apache Kylin Enables Sub‑Second OLAP on Massive Data Sets

Big Data Technology & Architecture

Jan 24, 2021 · Big Data

Design and Implementation of a Big Data OLAP Platform Based on Apache Kylin

This article explains the background, challenges, and architectural design of a big‑data OLAP platform that integrates Apache Kylin with a BI system, detailing pre‑computation strategies, cube construction, user authentication, storage engines, and query mechanisms to achieve sub‑second analytics on massive datasets.

Apache KylinData WarehouseHBase

0 likes · 11 min read

Design and Implementation of a Big Data OLAP Platform Based on Apache Kylin

DataFunTalk

Nov 23, 2020 · Big Data

Choosing OLAP Solutions for Large-Scale Data at Youku

The article examines the challenges big data brings to traditional technologies and surveys major OLAP solutions—MPP, batch processing, and pre‑computation—including Greenplum, Druid, Kylin, and Hadoop‑based engines, then outlines Youku’s specific use‑case selections for real‑time APIs, BI reporting, and ad‑hoc analysis.

MPPOLAPPrecomputation

0 likes · 13 min read

Choosing OLAP Solutions for Large-Scale Data at Youku

DataFunSummit

Nov 12, 2020 · Big Data

OLAP Engine Selection and Challenges in Large-Scale Data at Youku

This article explores the challenges big data brings to traditional data technologies and reviews various OLAP solutions—including MPP, batch processing, pre‑computation, and Hadoop‑based engines—while detailing Youku’s specific business scenarios and how different OLAP engines are selected to meet performance, scalability, and real‑time analysis requirements.

AnalyticsBig DataData Warehouse

0 likes · 14 min read

OLAP Engine Selection and Challenges in Large-Scale Data at Youku

Big Data Technology & Architecture

Aug 5, 2020 · Big Data

An Introduction to Apache Kylin: Architecture, Core Concepts, Installation, and Enterprise Use Cases

This article provides a comprehensive overview of Apache Kylin, covering its background, core OLAP concepts, technical architecture, installation steps, cube-building methods, real‑world enterprise deployments, and resources for further learning, illustrating how it enables sub‑second query performance on massive datasets.

Apache KylinBig DataCube

0 likes · 20 min read

An Introduction to Apache Kylin: Architecture, Core Concepts, Installation, and Enterprise Use Cases

Xueersi Online School Tech Team

Sep 27, 2019 · Big Data

Design Principles and Architecture of Apache Kylin for Sub‑Second OLAP Queries

This article explains how Apache Kylin, an open‑source distributed analytics engine built on Hadoop/Spark, achieves sub‑second OLAP query performance through pre‑computed cubes, a layered cuboid generation algorithm, bitmap‑based distinct counting, dimension optimization techniques, and tight integration with HBase for storage and fast SQL querying.

Apache KylinBig DataCube

0 likes · 15 min read

Design Principles and Architecture of Apache Kylin for Sub‑Second OLAP Queries

Meituan Technology Team

Feb 8, 2018 · Big Data

Operational Topic Data Productization: Architecture, Modeling, and Service Layers at Meituan-Dianping

To tackle Meituan‑Dianping’s massive POI and user data challenges, the team built a hybrid 3NF‑plus‑dimensional data warehouse with ODS, BAS, FACT, and optional TOPIC layers, added Kylin‑based pre‑computation cubes, a plug‑in middle‑platform service (metric dictionary, rule engine, compute engine), and interactive ECharts visualizations, delivering unified metrics, faster ad‑hoc queries, scalable analytics, and reduced development costs.

PrecomputationService Architecturevisualization

0 likes · 17 min read

Operational Topic Data Productization: Architecture, Modeling, and Service Layers at Meituan-Dianping

Meituan Technology Team

Dec 28, 2017 · Big Data

Design and Implementation of a Scalable Scenario Query System for Meituan

Meituan built a scalable scenario‑query platform that unifies traffic, activity and investment data by layering RPC services, a Storm‑driven pre‑computation tree stored in Redis/Tair, and a middle‑platform API with circuit‑breaker logic, cutting response times from seconds to under one second while dramatically reducing code coupling and simplifying future feature development.

Apache StormBig DataNoSQL

0 likes · 12 min read

Design and Implementation of a Scalable Scenario Query System for Meituan

ITPUB

Sep 22, 2017 · Big Data

How Baidu Waimai Scaled Traffic Analysis with Apache Kylin: A Deep Dive

This article presents a detailed case study of Baidu Waimai's traffic analysis platform, outlining the data challenges of high dimensionality and volume, the evaluation of OLAP engines, the adoption of Apache Kylin for pre‑computation, the end‑to‑end data modeling, cube construction, incremental builds, and integration with Saiku‑Mondrian reporting, while sharing practical lessons and performance gains.

Apache KylinBig DataOLAP

0 likes · 29 min read

How Baidu Waimai Scaled Traffic Analysis with Apache Kylin: A Deep Dive