Step‑by‑Step Guide to Deploy a High‑Availability Kubernetes Cluster on CentOS

1. Architecture Overview

Deploy five virtual machines (LB/DNS, etcd, master, Harbor, and worker nodes) with CentOS 7.5, static IPs, and sufficient CPU/RAM.

1.1 Hardware Environment

LB/DNS: 2C/2G, IP 10.211.55.11

etcd nodes: 4C/8G, IPs 10.211.55.21, 10.211.55.22

Harbor/NFS: 2C/2G, IP 10.211.55.200

2. Preparation Steps

2.1 Create Virtual Machines

<code># vim /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE="Ethernet"
BOOTPROTO="static"
IPADDR=10.211.55.11
PREFIX=24
GATEWAY=10.211.55.1
DNS1=10.211.55.1
</code>

2.2 System Initialization

<code># hostnamectl set-hostname zdd211-11.host.com
# systemctl stop firewalld && systemctl enable firewalld
# setenforce 0 && sed -i 's/enforcing/disabled/' /etc/selinux/config
# swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab
# ntpdate time.windows.com
# yum -y install epel-release wget net-tools telnet tree nmap sysstat lrzsz dos2unix bind-utils
</code>

2.3 DNS Service Initialization

<code># yum install bind -y
# vim /etc/named.conf
options {
  listen-on port 53 { 10.211.55.11; };
  allow-query { any; };
  forwarders { 10.211.55.1; };
  recursion yes;
  dnssec-enable no;
  dnssec-validation no;
}
# named-checkconf
# systemctl start named && systemctl enable named
# dig -t A zdd211-200.host.com @10.211.55.11 +short
</code>

Configure Client DNS

<code># vim /etc/sysconfig/network-scripts/ifcfg-eth0
... (same as above, add DNS1=10.211.55.11)
# cat /etc/resolv.conf
search host.com
nameserver 10.211.55.11
</code>

2.4 Certificate Authority (cfssl)

<code># wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -O /usr/bin/cfssl
# wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -O /usr/bin/cfssl-json
# chmod +x /usr/bin/cfssl*
# mkdir -p /opt/certs && cd /opt/certs
# vim ca-csr.json
{ "CN":"Goldwind", "key":{"algo":"rsa","size":2048}, "names":[{"C":"CN","ST":"beijing","L":"beijing","O":"od","OU":"ops"}], "ca":{"expiry":"175200h"} }
# cfssl gencert -initca ca-csr.json | cfssl-json -bare ca
</code>

2.5 Docker Installation

<code># curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun
# mkdir -p /data/docker /etc/docker
# cat > /etc/docker/daemon.json <<EOF
{
  "graph":"/data/docker",
  "storage-driver":"overlay2",
  "insecure-registries":["registry.access.redhat.com","quay.io","harbor.od.com"],
  "registry-mirrors":["https://q2gr04ke.mirror.aliyuncs.com"],
  "bip":"172.7.200.1/24",
  "exec-opts":["native.cgroupdriver=systemd"],
  "live-restore":true
}
EOF
# systemctl start docker && systemctl enable docker
</code>

2.6 Deploy Harbor Private Registry

<code># wget https://storage.googleapis.com/harbor-releases/release-1.8.0/harbor-offline-installer-v1.8.3.tgz
# tar -xf harbor-offline-installer-v1.8.3.tgz -C /opt
# mv /opt/harbor /opt/harbor-v1.8.3 && ln -sf /opt/harbor-v1.8.3 /opt/harbor
# vim /opt/harbor/harbor.yml
hostname: harbor.od.com
http:
  port: 180
harbor_admin_password: Harbor12345
data_volume: /data/harbor
log:
  level: info
  rotate_count: 50
  rotate_size: 200M
  location: /data/harbor/logs
# yum -y install docker-compose
# ./install.sh
# docker-compose ps   # verify all containers are up
</code>

3. Deploy Master Node Services

3.1 etcd Cluster

<code># cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer etcd-peer-csr.json | cfssl-json -bare etcd-peer
# useradd -s /sbin/nologin -M etcd
# wget https://github.com/etcd-io/etcd/releases/download/v3.1.20/etcd-v3.1.20-linux-amd64.tar.gz
# tar -xf etcd-v3.1.20-linux-amd64.tar.gz -C /opt && ln -s /opt/etcd-v3.1.20 /opt/etcd
# mkdir -p /opt/etcd/certs /data/etcd /data/logs/etcd-server
# scp ca.pem etcd-peer.pem etcd-peer-key.pem /opt/etcd/certs/
# chown -R etcd:etcd /opt/etcd/certs /data/etcd /data/logs/etcd-server
# cat > /opt/etcd/etcd-server-startup.sh <<'EOS'
#!/bin/sh
./etcd \
  --name etcd-server-55-12 \
  --data-dir /data/etcd/etcd-server \
  --listen-peer-urls https://10.211.55.12:2380 \
  --listen-client-urls https://10.211.55.12:2379,http://127.0.0.1:2379 \
  --initial-advertise-peer-urls https://10.211.55.12:2380 \
  --advertise-client-urls https://10.211.55.12:2379,http://127.0.0.1:2379 \
  --initial-cluster etcd-server-55-12=https://10.211.55.12:2380,etcd-server-55-21=https://10.211.55.21:2380,etcd-server-55-22=https://10.211.55.22:2380 \
  --ca-file ./certs/ca.pem \
  --cert-file ./certs/etcd-peer.pem \
  --key-file ./certs/etcd-peer-key.pem \
  --client-cert-auth \
  --trusted-ca-file ./certs/ca.pem \
  --log-output stdout
EOS
# chmod +x /opt/etcd/etcd-server-startup.sh
# yum -y install supervisor && systemctl enable --now supervisord
# cat > /etc/supervisord.d/etcd-server.ini <<'EOS'
[program:etcd-server-211-12]
command=/opt/etcd/etcd-server-startup.sh
directory=/opt/etcd
autostart=true
autorestart=true
stdout_logfile=/data/logs/etcd-server/etcd.stdout.log
EOS
# supervisorctl update && supervisorctl status
</code>

3.2 kube‑apiserver Cluster

<code># tar -xf kubernetes-server-linux-amd64-v1.15.4.tar.gz -C /opt && ln -s /opt/kubernetes-v1.15.4 /opt/kubernetes
# mkdir -p /opt/kubernetes/server/bin/certs /opt/kubernetes/server/bin/conf
# Generate client, apiserver certificates with cfssl (similar to etcd step)
# scp ca.pem client.pem client-key.pem apiserver.pem apiserver-key.pem to each master node
# cat > /opt/kubernetes/server/bin/kube-apiserver.sh <<'EOS'
#!/bin/bash
./kube-apiserver \
  --apiserver-count 2 \
  --audit-log-path /data/logs/kubernetes/kube-apiserver/audit-log \
  --audit-policy-file ./conf/audit.yaml \
  --authorization-mode RBAC \
  --client-ca-file ./certs/ca.pem \
  --etcd-cafile ./certs/ca.pem \
  --etcd-certfile ./certs/client.pem \
  --etcd-keyfile ./certs/client-key.pem \
  --etcd-servers https://10.211.55.12:2379,https://10.211.55.21:2379,https://10.211.55.22:2379 \
  --service-cluster-ip-range 192.168.0.0/16 \
  --tls-cert-file ./certs/apiserver.pem \
  --tls-private-key-file ./certs/apiserver-key.pem \
  --v 2
EOS
# chmod +x /opt/kubernetes/server/bin/kube-apiserver.sh
# cat > /etc/supervisord.d/kube-apiserver.ini <<'EOS'
[program:kube-apiserver-211-21]
command=/opt/kubernetes/server/bin/kube-apiserver.sh
directory=/opt/kubernetes/server/bin
autostart=true
autorestart=true
stdout_logfile=/data/logs/kubernetes/kube-apiserver/apiserver.stdout.log
EOS
</code>

3.3 controller‑manager

<code># cat > /opt/kubernetes/server/bin/kube-controller-manager.sh <<'EOS'
#!/bin/sh
./kube-controller-manager \
  --cluster-cidr 172.7.0.0/16 \
  --leader-elect true \
  --master http://127.0.0.1:8080 \
  --service-cluster-ip-range 192.168.0.0/16 \
  --root-ca-file ./certs/ca.pem \
  --v 2
EOS
# chmod +x ...
# cat > /etc/supervisord.d/kube-controller-manager.ini <<'EOS'
[program:kube-controller-manager-211-21]
command=/opt/kubernetes/server/bin/kube-controller-manager.sh
directory=/opt/kubernetes/server/bin
autostart=true
autorestart=true
stdout_logfile=/data/logs/kubernetes/kube-controller-manager/controller.stdout.log
EOS
</code>

3.4 kube‑scheduler

<code># cat > /opt/kubernetes/server/bin/kube-scheduler.sh <<'EOS'
#!/bin/sh
./kube-scheduler \
  --leader-elect \
  --master http://127.0.0.1:8080 \
  --v 2
EOS
# chmod +x ...
# cat > /etc/supervisord.d/kube-scheduler.ini <<'EOS'
[program:kube-scheduler-211-21]
command=/opt/kubernetes/server/bin/kube-scheduler.sh
directory=/opt/kubernetes/server/bin
autostart=true
autorestart=true
stdout_logfile=/data/logs/kubernetes/kube-scheduler/scheduler.stdout.log
EOS
</code>

3.5 4‑Layer Load Balancing (nginx + keepalived)

<code># vim /etc/nginx/nginx.conf (add stream block)
stream {
  upstream kube-apiserver {
    server 10.211.55.21:6443 max_fails=3 fail_timeout=30s;
    server 10.211.55.22:6443 max_fails=3 fail_timeout=30s;
  }
  server {
    listen 7443;
    proxy_pass kube-apiserver;
  }
}
# systemctl start nginx && systemctl enable nginx
# yum -y install keepalived
# vim /etc/keepalived/check_port.sh (script to monitor 7443)
# vim /etc/keepalived/keepalived.conf (configure virtual IP 10.211.55.10)
# systemctl start keepalived && systemctl enable keepalived
</code>

4. Deploy Node Services

4.1 kubelet

<code># Generate kubelet certificate with cfssl (hosts include all node IPs)
# scp kubelet.pem and kubelet-key.pem to each node
# kubelet config (kubelet.kubeconfig) created with kubectl config set-cluster/credentials/context
# cat > /opt/kubernetes/server/bin/kubelet.sh <<'EOS'
#!/bin/sh
./kubelet \
  --anonymous-auth=false \
  --cgroup-driver systemd \
  --cluster-dns 192.168.0.2 \
  --cluster-domain cluster.local \
  --runtime-cgroups=/systemd/system.slice \
  --kubelet-cgroups=/systemd/system.slice \
  --fail-swap-on=false \
  --client-ca-file ./certs/ca.pem \
  --tls-cert-file ./certs/kubelet.pem \
  --tls-private-key-file ./certs/kubelet-key.pem \
  --hostname-override $(hostname) \
  --pod-infra-container-image harbor.od.com/public/pause:latest \
  --kubeconfig ./conf/kubelet.kubeconfig \
  --log-dir /data/logs/kubernetes/kube-kubelet \
  --root-dir /data/kubelet
EOS
# chmod +x ...
# cat > /etc/supervisord.d/kube-kubelet.ini <<'EOS'
[program:kube-kubelet-211-21]
command=/opt/kubernetes/server/bin/kubelet.sh
directory=/opt/kubernetes/server/bin
autostart=true
autorestart=true
stdout_logfile=/data/logs/kubernetes/kube-kubelet/kubelet.stdout.log
EOS
</code>

4.2 kube‑proxy (IPVS mode)

<code># Generate kube‑proxy client certificate (system:kube-proxy)
# scp kube-proxy-client.pem and kube-proxy-client-key.pem to each node
# kube-proxy.kubeconfig created similarly to kubelet
# Load IPVS kernel modules (ipvs.sh script)
# cat > /opt/kubernetes/server/bin/kube-proxy.sh <<'EOS'
#!/bin/sh
./kube-proxy \
  --cluster-cidr 172.7.0.0/16 \
  --hostname-override $(hostname) \
  --proxy-mode=ipvs \
  --ipvs-scheduler=nq \
  --kubeconfig ./conf/kube-proxy.kubeconfig
EOS
# chmod +x ...
# cat > /etc/supervisord.d/kube-proxy.ini <<'EOS'
[program:kube-proxy-211-21]
command=/opt/kubernetes/server/bin/kube-proxy.sh
directory=/opt/kubernetes/server/bin
autostart=true
autorestart=true
stdout_logfile=/data/logs/kubernetes/kube-proxy/proxy.stdout.log
EOS
</code>

5. Verify the Cluster

<code># kubectl get cs
# kubectl get node
# kubectl create -f nginx-ds.yaml   # DaemonSet example
# kubectl get pods -o wide
</code>

All core components report Healthy , nodes show Ready , and the sample DaemonSet runs on each worker node.