Backing Up and Restoring etcd in a Kubernetes Cluster

This article demonstrates how to back up the etcd cluster of a Kubernetes control plane and restore it on a cluster with one master and one worker node, using etcdctl and standard Linux commands.

Step 1 Install etcd client

Install the etcd client on Ubuntu.

apt install etcd-client

Step 2 Create Nginx deployment

Create an Nginx deployment with multiple replicas to verify etcd data restoration.

kubectl create deployment nginx — image nginx --replicas=5

Check that the new Pods are running.

controlplane $ kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-77b4fdf86c-6m8gl   1/1     Running   0          50s
nginx-77b4fdf86c-bfcsr   1/1     Running   0          50s
nginx-77b4fdf86c-bqmqk   1/1     Running   0          50s
nginx-77b4fdf86c-nkh7j   1/1     Running   0          50s
nginx-77b4fdf86c-x946x   1/1     Running   0          50s

Step 3 Backup etcd cluster

Create a backup directory and run the etcd snapshot command.

mkdir etcd-backup
ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
    --cacert=/etc/kubernetes/pki/etcd/ca.crt \
    --cert=/etc/kubernetes/pki/etcd/server.crt \
    --key=/etc/kubernetes/pki/etcd/server.key \
    snapshot save ./etcd-backup/etcdbackup.db

You can obtain the certificate paths from the etcd pod in the kube-system namespace if you do not remember them.

Retrieve the etcd pod definition to see the exact certificate locations:

kubectl get pods -n kube-system

kubectl get pods etcd-controlplane -o yaml -n kube-system

Step 4 Verify backup data

Check the snapshot status to ensure the backup succeeded.

ETCDCTL_API=3 etcdctl --write-out=table snapshot status ./etcd-backup/etcdbackup.db

Step 5 Restore backup to the cluster

Delete the existing Nginx deployment, then restore the etcd snapshot.

kubectl delete deploy nginx

ETCDCTL_API=3 etcdctl snapshot restore etcd-backup/etcdbackup.db

If a hash mismatch error occurs, add the --skip-hash-check=true flag to the restore command.

ETCDCTL_API=3 etcdctl snapshot restore etcd-backup/etcdbackup.db --skip-hash-check=true

Move the restored data to the etcd data directory and replace the existing member folder.

cd default.etcd
mv /var/lib/etcd/member/ /var/lib/etcd/member.bak
mv member/ /var/lib/etcd/

Temporarily move the static pod manifests from /etc/kubernetes/manifests/ to stop the control‑plane components, then stop kubelet.

mkdir temp_yaml_files
mv /etc/kubernetes/manifests/* temp_yaml_files/
systemctl stop kubelet

After the data is in place, move the manifests back and restart kubelet.

mv temp_yaml_files/* /etc/kubernetes/manifests/
systemctl start kubelet

Finally, verify that the Nginx deployment is restored.

kubectl get pods

Congratulations! The etcd data has been successfully restored.