What’s New in Apache Flink 1.10? Deep Dive into Major Features and Enhancements

Introduction

Apache Flink is a next‑generation open‑source big‑data compute engine supporting stream, batch and machine‑learning workloads. In January 2019 Alibaba’s real‑time computing team open‑sourced the Blink engine and contributed it to Flink, culminating in the Blink‑to‑Flink merge in Flink 1.10.

Flink 1.10.0, released on 12 February 2020, marks the first double‑digit version with major improvements in production readiness, functionality and performance.

Key Statistics

Version 1.10.0 involved 218 contributors, resolved 1,270 JIRA issues, and added over 1.02 million lines of code across 2,661 commits, with Alibaba’s team contributing more than 60 % of the code.

Memory Management Optimisation

Previous Flink versions had fragmented memory configuration for streaming and batch, and RocksDB state backend could exceed container limits. In 1.10.0 the TaskExecutor memory model and managed memory (FLIP‑49) were redesigned, making configuration clearer and allowing RocksDB memory to be managed.

Controlled memory usage shows a significant reduction compared with the uncontrolled scenario.

Batch Compatibility with Hive

Flink 1.10 enhances Hive integration to production‑grade, supporting full Hive catalog reading, data format compatibility, and direct use of Hive UDF/UDTF/UDAF. Batch execution gains include vectorised ORC reads, elastic memory allocation (FLIP‑53), shuffle compression, and scheduler optimisations.

Meta compatibility – direct Hive catalog access for Hive 1.x/2.x/3.x.

Data format compatibility – read/write Hive tables, including partitions.

UDF compatibility – invoke Hive functions from Flink SQL.

SQL DDL Enhancements

Flink 1.10.0 allows WATERMARK and computed columns in CREATE TABLE statements and distinguishes between temporary, permanent, system and catalog functions.

CREATE TABLE table_name (
  WATERMARK FOR columnName AS <watermark_strategy_expression>
) WITH (
  ...
)

CREATE [TEMPORARY|TEMPORARY SYSTEM] FUNCTION
[IF NOT EXISTS] [catalog_name.] [db_name.]function_name
AS identifier [LANGUAGE JAVA|SCALA]

Python UDF Support

While Flink 1.9 introduced PyFlink, only Java‑based UDFs were available. Flink 1.10 adds native Python UDFs (FLIP‑58), enabling custom functions to be registered and used in Table API/SQL.

pip install apache-flink

Native Kubernetes Integration

Flink 1.10 provides native Kubernetes support (FLINK‑9953). The resource manager communicates directly with K8s to request pods on demand, simplifying deployment and reducing resource overhead in multi‑tenant environments.

Conclusion

The Blink engine’s integration and the extensive feature set in Flink 1.10 make it a robust platform for real‑time and batch analytics, with contributions from major tech companies and a thriving open‑source community.