How to Deploy a Kubernetes v1.28.8 Cluster with KubeKey on Ubuntu

Server Configuration

Three Ubuntu 20.04 LTS VMs are prepared with the following specifications:

Hostname: vm-16-11-ubuntu, IP: 192.168.9.131, CPU: 128, Memory: 256Gi, System Disk: 500GB, Data Disk: 1TB, Role: k8s-master

Hostname: vm-16-16-ubuntu, IP: 192.168.9.132, CPU: 128, Memory: 256Gi, System Disk: 500GB, Data Disk: 1TB, Role: k8s-master

Hostname: vm-16-7-ubuntu, IP: 192.168.9.133, CPU: 128, Memory: 256Gi, System Disk: 500GB, Data Disk: 1TB, Role: k8s-master

Software versions used in the environment:

Operating System: Ubuntu 20.04 LTS

K8s: v1.28.8

Containerd: 1.7.13

KubeKey: v3.1.1

Using KubeKey to Deploy K8s

1. OS Base Configuration

Hostname, DNS resolution, time synchronization, firewall, and system dependencies are omitted for brevity.

apt-get install -y curl socat conntrack ebtables ipset ipvsadm

2. Install K8s

2.1 Download KubeKey

URL: https://github.com/kubesphere/kubekey

root@VM-16-7-ubuntu:~# mkdir ~/kubekey
root@VM-16-7-ubuntu:~# cd ~/kubekey
root@VM-16-7-ubuntu:~/kubekey# curl -sfL https://get-kk.kubesphere.io | sh -
Downloading kubekey v3.1.1 from https://github.com/kubesphere/kubekey/releases/download/v3.1.1/kubekey-v3.1.1-linux-amd64.tar.gz ...
Kubekey v3.1.1 Download Complete!
root@VM-16-7-ubuntu:~/kubekey# ll
total 116376
drwxr-xr-x 2 root root 4096 May 7 17:50 .
drwx------ 7 root root 4096 May 7 17:49 ..
-rwxr-xr-x 1 root root 81950729 Apr 16 12:30 kk*
-rw-r--r-- 1 root root 37206726 May 7 17:50 kubekey-v3.1.1-linux-amd64.tar.gz

Check supported Kubernetes versions:

root@VM-16-7-ubuntu:~/kubekey# ./kk version --show-supported-k8s
v1.28.0
v1.28.1
v1.28.2
v1.28.3
v1.28.4
v1.28.5
v1.28.6
v1.28.7
v1.28.8
v1.29.0
v1.29.1
v1.29.2
v1.29.3

Create the cluster with the desired version:

root@VM-16-7-ubuntu:~/kubekey# ./kk create config -f k8s-v1288.yaml --with-kubernetes v1.28.8
Generate KubeKey config file successfully

2.2 Create K8s Cluster Config File

The chosen Kubernetes version is v1.28.8, so the configuration file is named k8s-v1288.yaml . The file’s kind: Cluster section is edited to specify hosts, roleGroups, internalLoadbalancer, domain, clusterName, autoRenewCerts, and containerManager.

hosts: define node IP, SSH user, password, and port

roleGroups: assign three nodes as etcd, control‑plane, and worker

internalLoadbalancer: enable built‑in HAProxy

domain: custom domain name

clusterName: default cluster.local

autoRenewCerts: true to auto‑renew certificates

containerManager: containerd

root@VM-16-7-ubuntu:~/kubekey# ./kk create cluster -f k8s-v1288.yaml
... (installation logs) ...
Installation is complete.

Please check the result using the command:

kubectl get pod -A

3. Verify K8s Cluster

3.1 Verify Cluster Status

root@VM-16-7-ubuntu:~# kubectl get nodes -owide
NAME               STATUS   ROLES                  AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE          KERNEL-VERSION   CONTAINER-RUNTIME
vm-16-11-ubuntu    Ready    control-plane,worker   9m46s v1.28.8   172.19.16.11  <none>        Ubuntu 20.04 LTS  5.4.0-174-generic containerd://1.7.13
vm-16-16-ubuntu    Ready    control-plane,worker   9m43s v1.28.8   172.19.16.16  <none>        Ubuntu 20.04 LTS  5.4.0-174-generic containerd://1.7.13
vm-16-7-ubuntu     Ready    control-plane,worker   10m   v1.28.8   172.19.16.7   <none>        Ubuntu 20.04 LTS  5.4.0-174-generic containerd://1.7.13

4. Deploy Test Resources

root@VM-16-7-ubuntu:~# kubectl create deployment nginx --image=nginx:alpine --replicas=2
deployment.apps/nginx created
root@VM-16-7-ubuntu:~# kubectl create service nodeport nginx --tcp=80:80
service/nginx created
root@VM-16-7-ubuntu:~# kubectl get deployment -o wide
NAME   READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES        SELECTOR
nginx  2/2     2            2           21s   nginx        nginx:alpine  app=nginx
root@VM-16-7-ubuntu:~# kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP           NODE               NOMINATED NODE   READINESS GATES
nginx-b4ccb96c6-q5c52   1/1     Running   0          21s   10.233.123.1 vm-16-7-ubuntu   <none>            <none>
nginx-b4ccb96c6-wgjh4   1/1     Running   0          21s   10.233.96.1  vm-16-7-ubuntu   <none>            <none>

5. Verify Service

Access Pod Directly

root@VM-16-7-ubuntu:~# curl -I http://10.233.96.1
HTTP/1.1 200 OK
Server: nginx/1.25.5
Date: Tue, 07 May 2024 10:24:00 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 16 Apr 2024 15:47:06 GMT
Connection: keep-alive
ETag: "661e9d7a-267"
Accept-Ranges: bytes

root@VM-16-7-ubuntu:~# curl -I http://10.233.123.1
HTTP/1.1 200 OK
Server: nginx/1.25.5
Date: Tue, 07 May 2024 10:24:10 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 16 Apr 2024 15:47:06 GMT
Connection: keep-alive
ETag: "661e9d7a-267"
Accept-Ranges: bytes

Access via NodePort

root@VM-16-7-ubuntu:~# curl http://172.19.16.7:31377
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p>
<p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>