Flink Streaming Job Tuning Guide: Memory Model, Network Stack, RocksDB, and More

This guide combines the principles of Apache Flink with the background of JD's Real‑Time Computing platform (JRC) to provide a comprehensive tuning methodology for large‑scale Flink streaming jobs.

1. TaskManager Memory Model Tuning – The article explains the TaskManager memory hierarchy, maps Flink JVM parameters to TaskManager memory settings, and demonstrates a manual calculation example for an 8C/16G TaskManager, showing how to allocate heap, off‑heap, network, and managed memory partitions.

t.m.process.size = 16384
t.m.flink.size = t.m.process.size * apus.memory.incontainer.available.ratio
= 16384 * 0.9 = 14745.6
t.m.jvm-metaspace.size = [t.m.process.size - t.m.flink.size] * apus.metaspace.incutoff.ratio
= [16384 - 14745.6] * 0.25 = 409.6
$overhead = MIN{t.m.process.size * t.m.jvm-overhead-fraction, t.m.jvm-overhead.max}
= MIN{16384 * 0.1, 1024} = 1024
$network = MIN{t.m.flink.size * t.m.network.fraction, t.m.network.max}
= MIN{14745.6 * 0.3, 5120} = 4423.68
$managed = t.m.flink.size * t.m.managed.fraction
= 14745.6 * 0.25 = 3686.4
t.m.task.off-heap.size = t.m.flink.size * apus.taskmanager.memory.task.off-heap.fraction
= 14745.6 * 0.01 = 147.4
t.m.task.heap.size = t.m.flink.size - $network - $managed - t.m.task.off-heap.size - t.m.framework.heap.size - t.m.framework.off-heap.size
= 14745.6 - 4423.68 - 3686.4 - 147.4 - 128 - 128 = 6232.12

2. Network Stack Tuning – Describes Flink’s Netty‑based network stack, buffer allocation rules, and how network buffer usage depends on parallelism and topology rather than actual traffic. It provides a tuning example that adjusts t.m.network.fraction and t.m.network.max to avoid Insufficient network buffers errors and improve CPU usage by increasing buffer timeout.

3. RocksDB and State Tuning – Introduces FRocksDB, the managed memory mechanism, and key parameters such as s.b.r.memory.write-buffer-ratio and s.b.r.memory.high-prio-pool-ratio . It advises enabling the fully managed mode, selecting appropriate predefined options (e.g., SPINNING_DISK_OPTIMIZED ), and monitoring metrics like block‑cache usage, flushes, and compactions.

4. Checkpoint and Miscellaneous Optimizations – Covers checkpoint timeout, pause intervals, state TTL via compaction filters, state scaling with key groups, local recovery settings, and object reuse for operator chains. It also highlights JobManager resource sizing and several practical tips (e.g., watermark interval, timer deduplication, custom serializers).

The article concludes with reference links to the Flink documentation, source code, and FRocksDB repository.