How Kubernetes Schedules Pods and Secures API Access: A Hands‑On Deep Dive

Overview

The article demonstrates a step‑by‑step experiment to understand how a containerized application interacts with a Kubernetes cluster, how the API Server authenticates clients using certificates, and how the scheduler assigns the pod to an appropriate node.

1. Sample Go Web Application

A minimal Go program app.go listens on port 2580 and returns a static string.

package main
import (
    "github.com/gorilla/mux"
    "log"
    "net/http"
)
func about(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("This is a small app for kubernetes...
"))
}
func main() {
    r := mux.NewRouter()
    r.HandleFunc("/", about)
    log.Fatal(http.ListenAndServe("0.0.0.0:2580", r))
}

Compiling with go build produces an executable that depends on the host OS libraries (shown by ldd app).

2. Building a Container Image

A Dockerfile creates an isolated environment for the binary:

FROM centos
ADD app /usr/local/bin

3. Pushing the Image to a Registry

The built image is pushed to an Alibaba Cloud container registry. The full image reference is:

registry.cn-hangzhou.aliyuncs.com/kube-easy/app:latest

The address consists of registry domain, namespace, image name, and tag.

4. Kubernetes API Server Entry Point

The API Server runs as a web service on the master node, exposing an HTTPS endpoint (e.g., https://xx.xxx.xxx.xxx:6443) that serves as the cluster’s gateway.

5. Mutual TLS Authentication

Kubernetes uses CA‑signed certificates for two‑way TLS authentication between the client and the API Server. The CA acts as a trusted third party, issuing a cluster CA certificate and a client certificate. The client trusts the cluster CA, and the API Server trusts the client CA, establishing a secure channel.

Certificate details (subject, issuer, validity) are shown using openssl output for both client and server certificates, illustrating that the same CA signs both sides.

6. KubeConfig File

The KubeConfig file contains the client certificate, client key, and the cluster CA certificate, all base64‑encoded. Decoding reveals the same fields displayed in the previous certificate sections.

7. Pod Scheduling Basics

Kubernetes treats the cluster as an operating system with master and worker nodes. A pod is the smallest deployable unit, encapsulating one or more tightly coupled containers.

8. Scheduling Algorithm – Pre‑filter

The scheduler first eliminates nodes that do not satisfy predefined predicates (e.g., PodFitsResourcesPred, PodFitsHostPortsPred). Nodes failing these checks are filtered out.

9. Scheduling Algorithm – Scoring

Remaining nodes are scored. Two built‑in scoring functions are highlighted:

LeastResourceAllocation : higher remaining CPU/memory ratio yields a higher score.

BalancedResourceAllocation : smaller difference between CPU and memory usage yields a higher score.

Additional factors such as pod‑node affinity and inter‑pod spread are also considered. Each score is multiplied by a weight (default weight = 1) and summed to produce the final node score.

10. Creating and Running the Pod

A pod manifest in JSON specifies the image, command, and container port:

{
  "apiVersion": "v1",
  "kind": "Pod",
  "metadata": {"name": "app"},
  "spec": {
    "containers": [{
      "name": "app",
      "image": "registry.cn-hangzhou.aliyuncs.com/kube-easy/app:latest",
      "command": ["app"],
      "ports": [{"containerPort": 2580}]
    }]
  }
}

The pod is created via a curl POST request to the API Server, using the client certificate and key.

11. Observing Scheduler Logs

Running the scheduler with --v=10 reveals detailed logs for the pre‑filter and scoring phases. Example log excerpts show the execution of CheckVolumeBindingPred (which does not filter the test pod) and the calculation of scores using the two strategies described earlier.

Log analysis shows that the default scheduler assigns equal weight to each scoring component, resulting in node scores of 29, 28, and 29 for a three‑node cluster. An undocumented high‑weight policy ( NodePreferAvoidPodsPriority with weight 10000) explains a discrepancy between observed and calculated scores.

Conclusion

By building a simple containerized web service, pushing it to a registry, and manually interacting with the Kubernetes API Server and scheduler, the article clarifies how mutual TLS authentication works and how the scheduler’s two‑step algorithm selects the most suitable node for a pod.