Scaling Presto at ByteDance: Architecture, Performance & Stability

Introduction

Within ByteDance, Presto powers ad‑hoc queries, BI visual analysis, and near‑real‑time analytics, handling close to one million queries per day.

ByteDance OLAP Data Engine Platform Presto Deployment Usage

Over recent years, ByteDance’s OLAP data engine evolved from a diverse ecosystem to a more consolidated, domain‑specific operation. Offline data resides in HDFS, while business and log data are stored in MQ and Kafka. Different compute engines serve different workloads: Presto handles ad‑hoc and some BI queries, SparkSQL processes large‑scale batch analytics and ETL, and Flink manages streaming data ingestion.

Since 2020, Presto has been introduced to meet growing ad‑hoc query demand. The cluster now spans tens of thousands of cores, serving roughly one million daily queries across most ad‑hoc scenarios and some BI analysis.

Presto Cluster Stability and Performance Improvements

To balance load across independent Presto clusters, a unified Gateway performs request routing and rate‑limiting using static rules (e.g., allowed clusters, disaster‑recovery clusters) and dynamic load‑aware routing via Coordinator REST APIs.

A multi‑Coordinator design replaces the single‑Coordinator model, providing active‑active redundancy. Coordinators and Workers discover each other through Zookeeper; Workers randomly select a Coordinator for heartbeat reporting, while Coordinators expose load and query execution metrics via REST APIs for coordinated scheduling. This reduces cluster downtime from minutes to under three seconds.

Large‑scale queries can monopolize resources and degrade performance. To mitigate this, ByteDance introduced rule‑based and cost‑based query‑time prediction, followed by an Adaptive Cancel feature that periodically estimates remaining execution time and aborts queries exceeding a threshold.

The History Server persists query execution details beyond the in‑memory UI, offering the same analysis experience as Presto’s UI and enabling monitoring dashboards.

Optimizations in Different Scenarios

Ad‑hoc Query Analysis

Presto, a resident MPP SQL engine, eliminates Spark Context startup overhead, delivering lower end‑to‑end latency. Compatibility layers rewrite user SQL to Presto syntax and support Hive UDFs, many of which have been contributed back to the Presto community.

BI Visualization

To meet high QPS and low‑latency demands, materialized views were added. Automatic view discovery analyzes query history to recommend and create views, while lifecycle management handles partition‑level updates and deletions. Query rewrite leverages existing materialized views to simplify execution.

Near‑Real‑Time Analytics

ByteDance introduced Hudi for incremental data updates, reducing pipeline latency to minutes. A dedicated Hudi connector extracts Hudi‑specific partition scheduling and memory‑management optimizations, preventing OOM on Workers and simplifying version upgrades.

These enhancements are now offered externally through Volcano Engine’s Lakehouse Analytics Service (LAS), a serverless data processing and analysis platform compatible with Spark, Presto, and Flink.