What’s New in Apache Flink 1.11? A Deep Dive into Features and Performance

Overview

Flink 1.11.0 was released on July 7 after four months of development, bringing enhancements to the ecosystem, usability, production readiness, and stability. The release was managed by Alibaba senior technical expert Wang Zhijiang and Ververica’s Piotr Nowojski.

Release Process

Each version selects 1‑2 release managers from volunteers; 1.11.0 had managers from China and Germany, reflecting the importance of Chinese contributions. The development cycle includes feature kickoff, a 2‑3 month development period, feature freeze, release candidates, and a final vote.

Statistics

236 contributors submitted 2,325 commits, resolved 1,474 JIRA issues, and addressed over 30 FLIPs. Chinese contributors accounted for 62% of the effort.

Ecosystem and Usability Improvements

Table & SQL CDC Support

Flink now supports Change Data Capture (CDC) in Table & SQL, enabling real‑time processing of changelog streams and integration with tools like Debezium and Canal. FLIP‑95 introduces new Table source and sink interfaces for CDC.

CREATE TABLE my_table ( ... ) WITH ( 'connector'='...', 'format'='debezium-json', 'debezium-json.schema-include'='true', 'debezium-json.ignore-parse-errors'='true' );

JDBC Catalog

FLIP‑93 adds a JDBC catalog and a Postgres implementation, allowing automatic schema discovery and compile‑time validation for relational databases.

Hive Real‑Time Warehouse

Flink 1.11.0 extends Hive integration with real‑time write support, partition handling, vectorized reads for ORC and Parquet, and a Hive dialect for SQL compatibility.

New Source API

FLIP‑27 redesigns the source architecture with Split Enumerator and Source Reader components, decoupling split discovery from processing and supporting unified batch‑stream connectors.

PyFlink Enhancements

Python UDFs can now be vectorized using Pandas (udf_type="pandas"), reducing serialization overhead via Apache Arrow. Additional features include seamless Table‑DataFrame conversion, Python UDTF support, Cython‑based performance boosts, and custom metrics.

Production Readiness and Stability

Application Mode and Kubernetes

Application mode (FLIP‑85) launches a cluster per application, moving job‑graph generation to the JobManager and reducing client bottlenecks. Native Kubernetes support adds node selectors, labels, tolerations, and automatic Hadoop configuration.

Checkpoint & Savepoint Optimizations

Savepoint metadata and state are now stored together with relative paths, simplifying migration. New checkpoint coordinator cancellation, buffer reductions, and the unaligned checkpoint mechanism (FLIP‑76) improve latency and resilience under backpressure.

Conclusion

The 1.11.0 release demonstrates growing Chinese contributions and sets the stage for the next major Flink version with anticipated heavyweight features.