Apache Flink OLAP Engine: Architecture, Optimizations, and Use Cases

The session, presented by Alibaba technical expert He Xiaoling, introduces Apache Flink's new scenario – an OLAP engine – and outlines its background, architecture, and future plans.

It explains the concept of OLAP and its three main categories: Multi‑dimensional OLAP (MOLAP), Relational OLAP (ROLAP), and Hybrid OLAP (HOLAP), describing their characteristics and typical implementations such as Kylin, Druid, Presto, and Impala.

The article details why Flink can serve as an OLAP engine, emphasizing its unified engine that combines stream, batch, and OLAP processing, unified APIs, multi‑layer APIs (SQL, Table API, DataStream API), high performance, rich connectors, flexible failover, and easy deployment.

Several performance optimizations are discussed: client serviceification to reduce query latency, a custom CollectionTableSink to limit result size and avoid OOM, scheduling improvements (eager mode, FIFO resource allocation, multithreaded resource manager), source push‑down for column‑store formats, aggregate push‑down using metadata, cross‑join elimination, and an adaptive local aggregate that disables low‑benefit aggregations based on runtime sampling.

Benchmark results on a 1 TB Star Schema Benchmark show Flink’s OLAP performance comparable to or better than Presto, and a data‑exploration use case demonstrates low‑latency, real‑time feedback for data lake queries.

Future work includes contributing the engine back to the Flink community, improving resource isolation for concurrent queries, and further performance enhancements tailored to OLAP workloads.