Step-by-Step Guide to Deploy Flannel CNI with Host‑GW, VXLAN, and iptables Optimization in Kubernetes

1. Introduction

1.1 Preface

Kubernetes defines a network model but delegates its implementation to CNI plugins. Common CNI plugins include Flannel, Calico, Canal, Contiv, OpenContrail, NSX‑T, and Kube‑router.

1.2 Flannel three network models

host‑gw model: all nodes must share the same physical gateway; it adds a static route on each host.

VXLAN model: works when hosts are in different subnets; creates a virtual network device and tunnel for communication.

Direct routing model: uses host‑gw when nodes share a gateway, otherwise falls back to VXLAN.

2. Cluster planning

Example deployment on 10.4.7.21 (repeat similarly for 10.4.7.22).

Hostname   Role    IP
hdss7-21   Flannel 10.4.7.21
hdss7-22   Flannel 10.4.7.22

3. Download software, extract, create symlink

Download URL: https://github.com/flannel-io/flannel/

# cd /opt/src/
# wget https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz
# mkdir /opt/flannel-v0.11.0
# tar -zxvf flannel-v0.11.0-linux-amd64.tar.gz -C /opt/flannel-v0.11.0
# ln -s /opt/flannel-v0.11.0 /opt/flannel

4. Final directory structure

# cd /opt/flannel
# ls
flanneld  mk-docker-opts.sh  README.md

5. Copy client certificates (etcd client)

# mkdir cert
# cd cert/
# scp hdss7-200:/opt/certs/ca.pem .
# scp hdss7-200:/opt/certs/client.pem .
# scp hdss7-200:/opt/certs/client-key.pem .
# ll

6. Create configuration

Each host may have different values; for node 10.4.7.22 set FLANNEL_SUBNET=172.7.22.1/24.

# cd ..
# vim subnet.env
FLANNEL_NETWORK=172.7.0.0/16
FLANNEL_SUBNET=172.7.21.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false

7. Create startup script

# vim flanneld.sh
#!/bin/sh
./flanneld \
  --public-ip=10.4.7.21 \
  --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 \
  --etcd-keyfile=./cert/client-key.pem \
  --etcd-certfile=./cert/client.pem \
  --etcd-cafile=./cert/ca.pem \
  --iface=ens33 \
  --subnet-file=./subnet.env \
  --healthz-port=2401

8. Set permissions and create log directory

# chmod +x flanneld.sh
# mkdir -p /data/logs/flanneld

9. Operate etcd to add host‑gw network

# cd /opt/etcd
# ./etcdctl set /coreos.com/network/config '{"Network":"172.7.0.0/16","Backend":{"Type":"host-gw"}}'
# ./etcdctl get /coreos.com/network/config
# ./etcdctl member list

10. Create supervisor configuration

# vim /etc/supervisord.d/flannel.ini
[program:flanneld-7-21]
command=/opt/flannel/flanneld.sh
numprocs=1
directory=/opt/flannel
autostart=true
autorestart=true
startsecs=30
startretries=3
exitcodes=0,2
stopsignal=QUIT
stopwaitsecs=10
user=root
redirect_stderr=true
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log
stdout_logfile_maxbytes=64MB
stdout_logfile_backups=4
stdout_capture_maxbytes=1MB
stdout_events_enabled=false

11. Start service and check

# supervisorctl update
# tail -100f /data/logs/flanneld/flanneld.stdout.log

12. Deploy other nodes and verify services

Repeat steps 3‑11 on 10.4.7.22. After flannel starts, routing tables show routes to the other node’s pod network, confirming proper CNI operation.

13. Verify pod network connectivity

From node 21 run curl 172.7.22.2 and from node 22 run curl 172.7.21.2; both return the default nginx welcome page, proving pod‑to‑pod communication.

14. Optimize iptables rules on each compute node

Default SNAT hides the real pod source IP. Remove the MASQUERADE rule for pod‑to‑pod traffic and adjust POSTROUTING to keep the original source.

# yum install iptables-services -y
# systemctl start iptables && systemctl enable iptables
# iptables -t nat -D POSTROUTING -s 172.7.21.0/24 ! -o docker0 -j MASQUERADE
# iptables -t nat -I POSTROUTING -s 172.7.21.0/24 ! -d 172.7.0.0/16 ! -o docker0 -j MASQUERADE
# iptables -t filter -D INPUT -j REJECT --reject-with icmp-host-prohibited
# iptables -t filter -D FORWARD -j REJECT --reject-with icmp-host-prohibited
# iptables-save > /etc/sysconfig/iptables
# service iptables save

15. Persist iptables rules and restart Docker

# systemctl restart docker
# iptables-save | grep -i postrouting | grep docker0

16. Troubleshoot DNS and external connectivity

If a pod cannot reach the internet, compare /etc/resolv.conf of a Docker‑run container and a pod created via kubectl. Adjust the pod’s nameserver to a reachable DNS (e.g., 10.4.7.11) and reinstall required tools such as curl to restore external access.