This article explains how to monitor and interpret Full GC in Flink TaskManagers, detect and address data skew through proper data distribution and parallelism settings, and recommends aligning consumer parallelism with Kafka partitions, while also providing practical tips for using tools like Prometheus and Arthas.
This article details the major memory model changes in Flink 1.11 for JobManager and TaskManager, compares them with Flink 1.9, provides concrete JVM command examples, explains the relationship between memory settings and parallelism, and introduces fine‑grained resource management for streaming workloads.
This article presents a detailed guide for optimizing large‑scale Apache Flink streaming jobs on the JD Real‑Time Computing platform, covering TaskManager memory model tuning, network stack configuration, RocksDB state management, checkpoint strategies, and additional performance tips with practical examples and calculations.
This article explains the official Flink memory model diagram, shows real‑world TaskManager memory parameters, and breaks down the five major memory components—including process, Flink, JVM heap, off‑heap, Metaspace, and overhead—providing configuration guidance for optimal resource allocation.
This article explains how Flink on YARN allocates TaskManager memory, breaks down the JVM heap, network buffers, and Flink Managed Memory, and shows how to calculate each component using configuration parameters and source‑code analysis.
This article explains the new unified TaskManager memory model introduced in Flink 1.10, detailing each memory component, its configuration parameters, how they map to JVM settings, and practical guidance for both standalone and containerized deployments, including a concrete YARN example.
This article explains the architecture of a Flink cluster, detailing the startup procedures for JobManager and TaskManager, the three deployment modes, and the end‑to‑end flow of a Flink job from client code through StreamGraph, JobGraph, ExecutionGraph to the physical execution on TaskManagers.
This tutorial explains how to run Apache Flink jobs on Kubernetes by building Docker images, deploying JobManager and TaskManager components with Kubernetes manifests, configuring high‑availability with ZooKeeper and HDFS, and using SavePoints and scaling techniques to manage and extend Flink streaming applications.
This article explains Flink’s architecture, detailing the roles of Client, JobManager and TaskManager, walks through a SocketTextStreamWordCount example, and clarifies the four‑layer graph model—StreamGraph, JobGraph, ExecutionGraph, and the physical execution graph—highlighting why each layer exists.
This article explains Flink’s cluster architecture, the roles of Client, JobManager and TaskManager, demonstrates a SocketTextStreamWordCount example, and details the four-layer execution graph (StreamGraph, JobGraph, ExecutionGraph, Physical Execution Graph) to illustrate how Flink schedules and runs streaming jobs.
