Understanding Flink Task Slots, Resource Allocation, and Slot Sharing Mechanisms

Task Slot

In Flink, each TaskManager runs as a JVM process and can host one or more subtasks; a task slot represents a subset of the TaskManager's resources, typically memory, allowing isolation of subtasks. Adjusting the number of slots per TaskManager controls the degree of isolation, with each slot sharing the same JVM when multiple slots are configured.

By default, Flink permits subtasks belonging to the same job to share a slot, which simplifies parallelism calculation and improves resource utilization, especially for lightweight map tasks.

Flink also provides APIs to control resource sharing, such as startNewChain, disableChaining, and slotSharingGroup, allowing users to avoid unintended sharing.

Resource Allocation Logic

The JobManager requests resources from the ResourceManager via the SlotProvider (Scheduler). The ResourceManager coordinates TaskManagers to satisfy these requests.

A detailed call flow diagram follows:

During execution, the SlotSharingManager decides whether multiple Executions can share the same slot, first checking the SlotPool for available resources before requesting new ones from the ResourceManager.

Scheduler

Requests slots from the SlotPool via allocateSlot.

Releases slots back to the SlotPool via returnLogicSlot.

SlotPool

Interaction with other components:

Scheduler → SlotPool: Scheduler requests resources.

SlotPool → ResourceManager: If SlotPool cannot satisfy the request, it asks the ResourceManager.

JobMaster → SlotPool: Resources obtained from TaskManagers are handed to SlotPool via JobMaster.

LogicalSlot

SingleLogicalSlot contains a SlotContext interface; PhysicalSlot extends SlotContext.

PhysicalSlot implements the Payload interface; PhysicalSlot.tryAssignPayload assigns a PhysicalSlot to a LogicalSlot.

PhysicalSlot

AllocatedSlot represents a resource segment allocated from a TaskExecutor.

Code example (illustrative) shows how a PhysicalSlot is assigned to a LogicalSlot.

Slot Resource Sharing

Flink supports two sharing mechanisms:

CoLocationGroup: Guarantees that the n‑th parallel instance of JobVertices in the same group run on the same slot.

SlotSharingGroup: Allows different JobVertices to share a slot with a looser constraint.

SlotSharingManager

Each sharing group is managed by a SlotSharingManager. For normal slot sharing, the manager looks up existing slots by JobVertex ID; for co‑location groups, it matches based on CoLocationConstraint.

Other Related Classes

CoLocationConstraint

SingleLogicalSlot

MultiTaskSlot