Presto High‑Performance Engine Practice at Meitu: Technical Selection, HA Design, and Cross‑Cluster Scheduling

Meitu evaluated three ad‑hoc ROLAP solutions—Hive on Spark, Impala, and Presto—highlighting Presto's lightweight, near‑real‑time query capabilities, its strong community support, and its suitability for flexible, high‑performance analytics, while noting its memory‑intensive nature and lack of built‑in HA.

The team implemented two coordinator high‑availability (HA) designs: (1) a dual‑cluster deployment with session state persisted in a database, and (2) a master‑slave setup using KeepAlived to provide a virtual IP that seamlessly fails over to a standby coordinator, eliminating single‑point failures without excessive resource duplication.

To improve resource utilization, Meitu introduced cross‑cluster scheduling that forwards suitable offline tasks to the Presto cluster during its low‑traffic window (0–9 am), leveraging cloud‑based storage separation to allow both offline and online clusters to access the same data.

Future plans categorize workloads into large‑shuffle, medium‑large, and small tasks, recommending Hive on Spark for the largest jobs, Spark SQL for medium tasks, and Presto for small, latency‑sensitive queries, while addressing challenges such as UDF compatibility, syntax differences, and unified permission checks.

The article concludes with a Q&A discussing alternatives like Doris and ClickHouse, performance comparisons between Presto and Hive on MapReduce, and an overview of the intelligent engine that matches tasks to the most appropriate execution engine based on historical metrics.