Deep Dive into Kubernetes Pod Creation: Exploring the kubelet Source Code

The author, Hua Zai, shares his experience of studying Kubernetes internals after four years of using the platform, focusing on the source code that handles pod creation in version 1.21.3.

Pod Creation Flow in kubelet

1. The CRUD operations for pods are triggered in

pkg/kubelet/kubelet.go

within the

syncLoop()

function.

<code>// syncLoop is the main loop for processing changes. It watches for changes from three channels (file, apiserver, and http) and creates a union of them. For any new change seen, it will run a sync against desired state and running state. If no changes are seen, it synchronizes the last known desired state every sync-frequency seconds. Never returns.
func (kl *Kubelet) syncLoop(updates <-chan kubetypes.PodUpdate, handler SyncHandler) {
    klog.InfoS("Starting kubelet main sync loop")
    // ...
}</code>

Within

syncLoop()

, the function

kl.syncLoopIteration()

performs the specific pod operations.

<code>kl.syncLoopMonitor.Store(kl.clock.Now())
if !kl.syncLoopIteration(updates, handler, syncTicker.C, housekeepingTicker.C, plegCh) {
    break
}
</code>

2.

syncLoopIteration

receives several important channels:

<code>// Arguments:
// 1. configCh:   a channel to read config events from
// 2. handler:    the SyncHandler to dispatch pods to
// 3. syncCh:     a channel to read periodic sync events from
// 4. housekeepingCh: a channel to read housekeeping events from
// 5. plegCh:    a channel to read PLEG updates from

func (kl *Kubelet) syncLoopIteration(configCh <-chan kubetypes.PodUpdate, handler SyncHandler,
    syncCh <-chan time.Time, housekeepingCh <-chan time.Time, plegCh <-chan *pleg.PodLifecycleEvent) bool {
    select {
    case u, open := <-configCh:
        if !open {
            klog.ErrorS(nil, "Update channel is closed, exiting the sync loop")
            return false
        }
        // ...
    }
    // ...
}
</code>

3.

SyncHandler

is an interface implemented by kubelet for testability, defining methods for common pod operations:

<code>type SyncHandler interface {
    HandlePodAdditions(pods []*v1.Pod)
    HandlePodUpdates(pods []*v1.Pod)
    HandlePodRemoves(pods []*v1.Pod)
    HandlePodReconcile(pods []*v1.Pod)
    HandlePodSyncs(pods []*v1.Pod)
    HandlePodCleanups() error
}
</code>

4. Pod operations are represented by constants such as

ADD

,

DELETE

,

UPDATE

, etc., each invoking the corresponding handler method. For example, an

ADD

triggers

HandlePodAdditions

:

<code>switch u.Op {
case kubetypes.ADD:
    klog.V(2).InfoS("SyncLoop ADD", "source", u.Source, "pods", format.Pods(u.Pods))
    handler.HandlePodAdditions(u.Pods)
}
</code>

5.

HandlePodAdditions

performs several steps:

<code>// 1. Sort pods by creation time
sort.Sort(sliceutils.PodsByCreationTime(pods))

// 2. Add pod to podManager (source of truth for desired state)
kl.podManager.AddPod(pod)

// 3. Check if the pod is a static pod
mirrorPod, _ := kl.podManager.GetMirrorPodByPod(pod)

// 4. Dispatch work to the pod worker
kl.dispatchWork(pod, kubetypes.SyncPodCreate, mirrorPod, start)

// 5. Register pod with the probe manager for health checks
kl.probeManager.AddPod(pod)
</code>

6.

dispatchWork

launches an asynchronous worker that calls

UpdatePod

:

<code>kl.podWorkers.UpdatePod(&UpdatePodOptions{
    Pod:        pod,
    MirrorPod:  mirrorPod,
    UpdateType: syncType,
    OnCompleteFunc: func(err error) {
        if err != nil {
            metrics.PodWorkerDuration.WithLabelValues(syncType.String()).Observe(metrics.SinceInSeconds(start))
        }
    },
})
</code>

7.

UpdatePod

creates a new pod worker (or restarts one) and invokes

managePodLoop

to process updates:

<code>go func() {
    defer runtime.HandleCrash()
    p.managePodLoop(podUpdates)
}()
</code>

8. Inside

managePodLoop

, each pod update is synchronized via

syncPodFn

, which eventually calls

SyncPod

in

pkg/kubelet/kuberuntime/kuberuntime_manager.go

to align the running pod with the desired state.

<code>// SyncPod syncs the running pod into the desired pod by executing the following steps:
// 1. Compute sandbox and container changes.
// 2. Kill pod sandbox if necessary.
// 3. Kill any containers that should not be running.
// 4. Create sandbox if necessary.
// 5. Create ephemeral containers.
// 6. Create init containers.
// 7. Create normal containers.
func (m *kubeGenericRuntimeManager) SyncPod() {
    podContainerChanges := m.computePodActions(pod, podStatus)
    // ... steps 2‑7 implementation ...
}
</code>

9. Additional responsibilities of the pod worker include creating pod data directories, handling volumes, retrieving image pull secrets, and invoking the container runtime's

SyncPod

method.

The article concludes with a brief invitation to follow the author's public account for more technical content.