Building a Kubernetes Cluster from Scratch: kubeadm vs Binary Deployment Guide

1. Cluster Architecture Planning

Two cluster topologies: single-master and multi-master (high availability). Diagrams illustrate component distribution for each approach.

2. Hardware Requirements

OS: CentOS 7.x 64-bit

Minimum: 2GB RAM, 2 CPU cores, 30GB disk

Network connectivity between all nodes

Internet access for image pulling

Swap must be disabled

3. Deployment Methods Comparison

Two primary production deployment methods are compared:

kubeadm : Official tool using kubeadm init and kubeadm join for rapid deployment. Simpler but abstracts away details, making troubleshooting harder.

Binary packages : Manual component deployment from GitHub releases. More labor-intensive but provides deeper understanding of internal workings and easier long-term maintenance.

The author recommends binary deployment for learning and controllability.

4. kubeadm Deployment Walkthrough

4.1 Environment Setup

Three nodes defined:

k8s-master: 192.168.217.100

k8s-node1: 192.168.217.101

k8s-node2: 192.168.217.102

4.2 System Initialization (all nodes)

Disable firewall:

systemctl stop firewalld && systemctl disable firewalld

Disable SELinux permanently: sed -i 's/enforcing/disabled/' /etc/selinux/config then reboot; temporarily: setenforce 0 Disable swap permanently: sed -ri 's/.*swap.*/#&/' /etc/fstab then reboot; temporarily: swapoff -a Set hostnames: hostnamectl set-hostname <hostname> (k8s-master, k8s-node1, k8s-node2)

Configure /etc/hosts on all nodes with the three IPs

Enable bridge-nf-call-iptables and IP forwarding via /etc/sysctl.d/k8s.conf and load br_netfilter module

Time synchronization: yum install ntpdate -y && ntpdate time.windows.com Enable IPVS: install ipset ipvsadm, create /etc/sysconfig/modules/ipvs.modules with required modprobe commands, make executable and run

4.3 Install Docker, kubeadm, kubelet, kubectl (all nodes)

Docker 18.06.3 from Alibaba Cloud mirror with systemd cgroup driver and registry mirror

Add Kubernetes YUM repo from Alibaba Cloud

Install specific versions: yum install -y kubelet-1.18.0 kubeadm-1.18.0 kubectl-1.18.0 Set kubelet cgroup driver to systemd in /etc/sysconfig/kubelet: KUBELET_EXTRA_ARGS="--cgroup-driver=systemd" Enable kubelet:

systemctl enable kubelet

4.4 Master Node Initialization

kubeadm init \
  --apiserver-advertise-address=192.168.217.100 \
  --image-repository registry.aliyuncs.com/google_containers \
  --kubernetes-version v1.18.0 \
  --service-cidr=10.96.0.0/12 \
  --pod-network-cidr=10.244.0.0/16

Post-init: copy admin.conf to $HOME/.kube/config and chown.

4.5 Join Worker Nodes

kubeadm join 192.168.217.100:6443 --token 4016im.eg4e10yamcbxjm59 \
  --discovery-token-ca-cert-hash sha256:ce2111ce594e5189255144a72268250e5eedda87470cc3a1f69f8c973927699e

Token management: default 24h TTL; create new with kubeadm token create --print-join-command; permanent token: kubeadm token create --ttl 0.

4.6 Deploy CNI (Flannel)

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Verify with kubectl get pods -n kube-system, then kubectl get nodes should show Ready, kubectl get cs shows Healthy, kubectl cluster-info confirms control plane.

5. Binary Package Deployment (High-Availability)

5.1 Node Roles and Components

k8s-master (192.168.217.100): kube-apiserver, kube-controller-manager, kube-scheduler, docker, etcd

k8s-node1 (192.168.217.101): kubelet, kube-proxy, docker, etcd

k8s-node2 (192.168.217.102): kubelet, kube-proxy, docker, etcd

5.2 System Initialization

Same as section 4.2.

5.3 Deploy etcd Cluster (3 nodes)

etcd provides distributed key-value storage for Kubernetes. A 3-node cluster tolerates one failure.

5.3.1 Certificate Generation with cfssl

Download cfssl, cfssljson, cfssl-certinfo v1.2 from https://pkg.cfssl.org/R1.2/ Create CA: ca-config.json (87600h expiry, kubernetes profile with signing, key encipherment, server/client auth), ca-csr.json (CN=kubernetes, RSA 2048, CN/Beijing/Beijing/k8s/System)

Generate CA: cfssl gencert -initca ca-csr.json | cfssljson -bare ca - Generate server cert for etcd: server-csr.json with hosts = three node IPs, same names; sign with CA:

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server

5.3.2 etcd Binary Deployment

Download etcd v3.4.9 from GitHub:

https://github.com/etcd-io/etcd/releases/download/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz

Extract etcd and etcdctl to /opt/etcd/bin/ Create /opt/etcd/cfg/etcd.conf with member and clustering config (ETCD_NAME, ETCD_DATA_DIR, LISTEN_PEER_URLS, LISTEN_CLIENT_URLS, INITIAL_ADVERTISE_PEER_URLS, ADVERTISE_CLIENT_URLS, INITIAL_CLUSTER, INITIAL_CLUSTER_TOKEN, INITIAL_CLUSTER_STATE=new)

Create systemd unit /usr/lib/systemd/system/etcd.service with cert/key/CA paths, zap logger, restart on-failure, LimitNOFILE=65536

Copy certs to /opt/etcd/ssl/, scp entire /opt/etcd and service file to node1/node2

On each node, adjust ETCD_NAME and IP addresses in etcd.conf

Start and enable etcd on all nodes:

systemctl daemon-reload && systemctl start etcd && systemctl enable etcd

Verify:

systemctl status etcd.service

5.4 Generate API Server Certificates

Similar cfssl process in ~/TLS/k8s/: create CA, then server cert with extensive host list (10.0.0.1, 127.0.0.1, kubernetes DNS names, all node IPs, plus 192.168.217.1, 192.168.217.2, 192.168.31.198), and kube-proxy client cert (CN=system:kube-proxy, no hosts).

5.5 Deploy Master Components

5.5.1 Download Kubernetes Binaries

mkdir -pv /opt/kubernetes/{bin,cfg,ssl,logs}
wget "https://dl.k8s.io/v1.18.10/kubernetes-server-linux-amd64.tar.gz"
tar -zxvf kubernetes-server-linux-amd64.tar.gz
cd kubernetes/server/bin
cp kube-apiserver kube-scheduler kube-controller-manager /opt/kubernetes/bin
cp kubectl /usr/bin/

5.5.2 kube-apiserver Configuration

KUBE_APISERVER_OPTS="--logtostderr=false \
--v=2 \
--log-dir=/opt/kubernetes/logs \
--etcd-servers=https://192.168.217.100:2379,https://192.168.217.101:2379,https://192.168.217.102:2379 \
--bind-address=192.168.217.100 \
--secure-port=6443 \
--advertise-address=192.168.217.100 \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--enable-bootstrap-token-auth=true \
--token-auth-file=/opt/kubernetes/cfg/token.csv \
--service-node-port-range=30000-32767 \
--kubelet-client-certificate=/opt/kubernetes/ssl/server.pem \
--kubelet-client-key=/opt/kubernetes/ssl/server-key.pem \
--tls-cert-file=/opt/kubernetes/ssl/server.pem \
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/etcd/ssl/ca.pem \
--etcd-certfile=/opt/etcd/ssl/server.pem \
--etcd-keyfile=/opt/etcd/ssl/server-key.pem \
--audit-log-maxage=30 \
--audit-log-maxbackup=3 \
--audit-log-maxsize=100 \
--audit-log-path=/opt/kubernetes/logs/k8s-audit.log"

Copy generated certs to /opt/kubernetes/ssl/.

5.5.3 TLS Bootstrapping Setup

Generate token: head -c 16 /dev/urandom | od -An -t x | tr -d ' ' → 6cd622a46af13091337d98f0ac9da4d0. Create /opt/kubernetes/cfg/token.csv:

6cd622a46af13091337d98f0ac9da4d0,kubelet-bootstrap,10001,"system:nodebootstrapper"

. Create clusterrolebinding:

kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap

.

5.5.4 systemd Units and Startup

Create service files for apiserver, controller-manager, scheduler with EnvironmentFile pointing to respective configs, ExecStart with $KUBE_XXX_OPTS, Restart=on-failure. Reload, start, enable each.

5.5.5 Verify Master Components

kubectl get cs

shows Healthy for scheduler, controller-manager, etcd.

5.6 Docker Installation

Same as section 4.3 on all nodes.

5.7 Deploy Node Components

5.7.1 Prepare Node Directories and Binaries

Create /opt/kubernetes/{bin,cfg,ssl,logs} on nodes. SCP kubelet, kube-proxy binaries and ssl directory from master.

5.7.2 Generate Bootstrap and kube-proxy Kubeconfigs

On master, run a script ( configure.sh) that uses kubectl config commands to create bootstrap.kubeconfig (with token auth) and kube-proxy.kubeconfig (with client cert/key). SCP both to /opt/kubernetes/cfg/ on each node.

5.7.3 kubelet Configuration (per node)

KUBELET_OPTS="--logtostderr=false \
--v=2 \
--log-dir=/opt/kubernetes/logs \
--hostname-override=k8s-node1 \
--network-plugin=cni \
--kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \
--bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \
--cert-dir=/opt/kubernetes/ssl \
--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0"

Node2 uses --hostname-override=k8s-node2.

5.7.4 kubelet systemd and Startup

Service file with After=docker.service, EnvironmentFile, ExecStart, Restart=on-failure, LimitNOFILE=65536. Enable and start.

5.7.5 kube-proxy Configuration (per node)

KUBE_PROXY_OPTS="--logtostderr=false \
--v=2 \
--log-dir=/opt/kubernetes/logs \
--kubeconfig=/opt/kubernetes/cfg/kube-proxy.kubeconfig \
--hostname-override=k8s-node1"

Node2 uses k8s-node2.

5.7.6 kube-proxy systemd and Startup

Service file with After=network.target, similar structure. Enable and start.

5.8 Deploy CNI (Flannel)

Same as section 4.6.

5.9 Approve kubelet CSRs

On master: kubectl get csr shows pending requests; approve each: kubectl certificate approve <NAME>.