How to Build a Fully HA Kubernetes Cluster with Nginx, Keepalived, and Harbor

Prerequisites

Prepare three nodes (1 master, 2 workers) with hostnames, /etc/hosts entries, SSH trust, time sync, disabled swap, SELinux, firewalld, and enable br_netfilter.

Install Docker and Containerd

Install Docker CE from Alibaba mirrors, start and enable it, then configure the daemon.json registry mirrors. Install containerd (included with Docker) and set SystemdCgroup=true.

# Docker installation commands
yum install -y yum-utils
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install -y docker-ce
systemctl start docker && systemctl enable docker

# Containerd configuration
sed -i 's#SystemdCgroup = false#SystemdCgroup = true#' /etc/containerd/config.toml
systemctl restart containerd

Kubernetes Packages

Install kubeadm, kubelet and kubectl version 1.24.1, enable and start kubelet.

yum install -y kubelet-1.24.1 kubeadm-1.24.1 kubectl-1.24.1 --disableexcludes=kubernetes
systemctl enable --now kubelet

Initialize the Control Plane

Pull required images from Alibaba registry, then run kubeadm init with custom image repository and pod network CIDR for Flannel.

kubeadm init \
  --apiserver-advertise-address=192.168.0.113 \
  --image-repository registry.aliyuncs.com/google_containers \
  --control-plane-endpoint=cluster-endpoint \
  --kubernetes-version v1.24.1 \
  --service-cidr=10.1.0.0/16 \
  --pod-network-cidr=10.244.0.0/16 \
  --v=5

Install Flannel CNI

Deploy Flannel as the pod network.

docker pull quay.io/coreos/flannel:v0.14.0
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Join Worker Nodes

On each worker run the join command generated by kubeadm.

kubeadm join 192.168.0.113:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

IPVS Mode for kube-proxy

Load ip_vs modules, install ipvsadm, edit the kube-proxy ConfigMap to set mode=ipvs, then restart the kube-proxy pods.

High Availability Control Plane (Stacked)

Add a second master node (192.168.0.116), repeat the installation steps, but skip kubeadm init. Use a virtual IP managed by Keepalived and Nginx as a load balancer.

Install Nginx and Keepalived

Configure Nginx stream block to proxy the API server traffic, and Keepalived with VRRP to provide a floating IP (192.168.0.120).

# Keepalived config (excerpt)
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication { auth_type PASS auth_pass 1111 }
    virtual_ipaddress { 192.168.0.120/24 }
}

Kubernetes Dashboard

Deploy the dashboard, expose it via a NodePort service (port 31443), create an admin ServiceAccount and retrieve its token for login.

Harbor Private Registry

Install Helm, add the Harbor chart, generate a self‑signed certificate, create a Namespace and secret, then deploy Harbor with NFS storage provisioned by the nfs‑subdir‑external‑provisioner.

helm repo add harbor https://helm.goharbor.io
helm install myharbor harbor/harbor \
  --set expose.ingress.hosts.core=myharbor.com \
  --set externalURL=https://myharbor.com \
  --set persistence.persistentVolumeClaim.registry.storageClass=nfs-client \
  --set harborAdminPassword=Harbor12345

Configure Containerd to Trust Harbor

Add the Harbor CA to /etc/containerd/myharbor.com/ca.crt and update /etc/containerd/config.toml with the registry entry.

After completing these steps you have a production‑ready, highly available Kubernetes cluster with a private image registry and a web UI.