How to Deploy a Multi‑Node Kafka Cluster with Zookeeper from Scratch

Kafka Cluster Deployment and Startup

In this article we demonstrate how to build a Kafka cluster and briefly introduce key concepts. It assumes the reader already knows basic Kafka fundamentals.

Kafka natively supports clustering; even a single node runs in cluster mode.

The cluster relies on Zookeeper for coordination, and early Kafka versions stored much data in Zookeeper.

Any Kafka node that registers with the same Zookeeper belongs to the same cluster.

Each node is identified by a unique brokerId.

Kafka cluster topology diagram:

Roles in a Kafka cluster:

Broker – the Kafka server node.

Leader – handles produce and consume requests; producers push messages to the leader and consumers poll from it.

Follower – replicates the leader’s data; each leader has multiple followers.

Four virtual machines are used for the demonstration:

Machine IP 192.168.99.1 – broker server (brokerId 0)

Machine IP 192.168.99.2 – broker server (brokerId 1)

Machine IP 192.168.99.3 – broker server (brokerId 2)

Machine IP 192.168.99.4 – Zookeeper node (cluster coordinator)

Zookeeper Installation

Kafka uses Zookeeper for distributed coordination, so Zookeeper must be installed first. Both require a JDK, which is assumed to be installed:

[[email protected] ~]# java --version
java 11.0.5 2019-10-15 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.5+10-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.5+10-LTS, mixed mode)

Download Zookeeper from the official site:

https://zookeeper.apache.org/releases.html#download

Download with wget:

[[email protected] ~]# cd /usr/local/src
[[email protected] /usr/local/src]# wget https://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.6.1/apache-zookeeper-3.6.1-bin.tar.gz

Extract and rename the directory:

[[email protected] /usr/local/src]# tar -zxvf apache-zookeeper-3.6.1-bin.tar.gz
[[email protected] /usr/local/src]# mv apache-zookeeper-3.6.1-bin ../zookeeper

Copy the sample configuration and edit the data directory:

[[email protected] /usr/local/zookeeper/conf]# cp zoo_sample.cfg zoo.cfg
# Set a large data directory
[dataDir=/data/zookeeper]
[[email protected] /usr/local/zookeeper/conf]# mkdir -p /data/zookeeper

Start Zookeeper:

[[email protected] /usr/local/zookeeper/bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

Verify it is listening on port 2181:

[[email protected] ~]# netstat -lntp | grep 2181
tcp6       0      0 :::2181                 :::*                 LISTEN      7825/java

If a firewall is enabled, open port 2181:

[[email protected] ~]# firewall-cmd --zone=public --add-port=2181/tcp --permanent
[[email protected] ~]# firewall-cmd --reload

Kafka Installation

After Zookeeper, download Kafka:

https://zookeeper.apache.org/releases.html#download

Download with wget:

[[email protected] ~]# cd /usr/local/src
[[email protected] /usr/local/src]# wget https://mirror.bit.edu.cn/apache/kafka/2.5.0/kafka_2.13-2.5.0.tgz

Extract and rename:

[[email protected] /usr/local/src]# tar -xvf kafka_2.13-2.5.0.tgz
[[email protected] /usr/local/src]# mv kafka_2.13-2.5.0 ../kafka

Edit server.properties to configure the broker:

[[email protected] /usr/local/kafka/config]# vim server.properties
# Unique broker ID
broker.id=0
# Internal listener address
listeners=PLAINTEXT://192.168.99.1:9092
# External address (if needed)
advertised.listeners=PLAINTEXT://192.168.99.1:9092
# Log directory
log.dirs=/usr/local/kafka/kafka-logs
# Zookeeper connection
zookeeper.connect=192.168.99.4:2181
[[email protected] /usr/local/kafka/config]# mkdir /usr/local/kafka/kafka-logs

Add Kafka binaries to the PATH:

[[email protected] ~]# vim /etc/profile
export KAFKA_HOME=/usr/local/kafka
export PATH=$PATH:$KAFKA_HOME/bin
[[email protected] ~]# source /etc/profile

Start Kafka:

[[email protected] ~]# kafka-server-start.sh /usr/local/kafka/config/server.properties &

Verify it is listening on port 9092:

[[email protected] ~]# netstat -lntp | grep 9092
tcp6       0      0 192.168.99.1:9092       :::*                 LISTEN      31943/java

If a firewall is enabled, open port 9092:

[[email protected] ~]# firewall-cmd --zone=public --add-port=9092/tcp --permanent
[[email protected] ~]# firewall-cmd --reload

Copy the Kafka directory to the other two machines and adjust their configurations:

[[email protected] ~]# rsync -av /usr/local/kafka 192.168.99.2:/usr/local/kafka
[[email protected] ~]# rsync -av /usr/local/kafka 192.168.99.3:/usr/local/kafka

On each node edit server.properties to set a unique broker.id and listeners (e.g., 192.168.99.2 and 192.168.99.3).

After starting all brokers, check Zookeeper for registered broker IDs:

[[email protected] ~]# /usr/local/zookeeper/bin/zkCli.sh
[zk: localhost:2181(CONNECTED) 4] ls /brokers/ids
[0, 1, 2]

Kafka Replication

Replication creates multiple copies of log partitions for redundancy. The replication factor determines how many copies each partition has, and it can be configured per topic.

Replication algorithm:

Sort all N brokers and the i‑th partition.

Assign partition i to broker (i mod N).

Assign replica j of partition i to broker ((i + j) mod N).

Kafka Broker Failure Handling

Two failure scenarios:

Loss of heartbeat between broker and Zookeeper.

Follower lagging far behind the leader.

Kafka removes the failed broker, avoids data loss, and rebalances load across the remaining nodes.

Kafka Leader Election Overview

Kafka does not use a voting algorithm. It maintains an In‑Sync Replica (ISR) set and selects the fastest replica as the leader.

If all ISR replicas are down, Kafka may perform an unclean leader election. Two options are:

Wait for an ISR replica to recover and become leader (higher safety, lower availability).

Immediately promote the first recovered replica, even if not in ISR (higher availability, possible data loss).

Recommendation: disable unclean leader election and configure a minimum ISR.

More details on ISR can be found at:

https://www.jianshu.com/p/ff296d51385a

https://blog.csdn.net/qq_37502106/article/details/80271800