Build Military‑Grade Kafka Security in Four Easy Steps

As the core hub of distributed messaging systems, securing Kafka is essential for operations teams. This article guides you through four stages from basic to financial‑grade protection, building a robust message fortress.

Stage 1: Bare‑metal Mode – Basic Configuration (Use with Caution)

Reference: author's blog for installing Kafka. This is the original configuration where data is transmitted in plaintext, suitable only for isolated test environments; production must be upgraded.

Risk Warning: No encryption or authentication; attackers can easily eavesdrop or tamper with data.

Stage 2: SSL Armor – Transport Encryption

Creating a Digital Certificate Authority

# 1. Generate CA private key and self‑signed certificate
openssl req -new -x509 -nodes \
 -keyout ca.key -out ca.crt \
 -days 3650 -subj "/C=CN/ST=GuangDong/L=GuangZhou/CN=Kafka Root CA"

# 2. Import CA certificate into PKCS#12 truststore (shared by client and server)
keytool -keystore kafka.server.truststore.p12 -storetype PKCS12 \
 -alias CARoot -import -file ca.crt \
 -storepass truststore_password -noprompt

# Clean temporary files
rm -f ca.srl

Server Certificate

# 1. Generate server PKCS#12 keystore (private key + unsigned cert)
keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 \
 -alias localhost -validity 3650 \
 -genkey -keyalg RSA -keysize 2048 \
 -storepass keystore_password \
 -dname "C=CN/ST=GuangDong/L=GuangZhou/CN=kafka-server"

# 2. Generate CSR
keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 \
 -alias localhost -certreq -file server.csr \
 -storepass keystore_password

# 3. Sign server certificate with CA (add SAN)
openssl x509 -req -CA ca.crt -CAkey ca.key \
 -in server.csr -out server-signed.crt \
 -days 3650 -CAcreateserial \
 -extfile <(printf "subjectAltName=IP:172.139.20.17,IP:172.139.20.81,IP:172.139.20.177")

# 4. Import CA and signed certificate into server keystore
keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 \
 -alias CARoot -import -file ca.crt \
 -storepass keystore_password -noprompt

keytool -keystore kafka.server.keystore.p12 -storetype PKCS12 \
 -alias localhost -import -file server-signed.crt \
 -storepass keystore_password -noprompt

# 5. Verify SAN extension
keytool -list -v -keystore kafka.server.keystore.p12 -alias localhost -storepass keystore_password | grep -A4 SubjectAlternativeName

# Clean temporary files
rm -f server.csr

Key Configuration

listeners=PLAINTEXT://:9092,SSL://:9093
advertised.listeners=PLAINTEXT://172.139.20.17:9092,SSL://172.139.20.17:9093

# SSL settings
ssl.keystore.location=/app/kafka/pki/kafka.server.keystore.p12
ssl.keystore.password=keystore_password
ssl.keystore.type=PKCS12
ssl.truststore.location=/app/kafka/pki/kafka.server.truststore.p12
ssl.truststore.password=truststore_password
ssl.truststore.type=PKCS12
ssl.client.auth=none

Validate SSL

Client configuration (ssl-client.properties)

security.protocol=SSL
ssl.truststore.location=/app/kafka/pki/kafka.server.truststore.p12
ssl.truststore.password=truststore_password
ssl.truststore.type=PKCS12

Consume messages (SSL)

bin/kafka-console-consumer.sh --bootstrap-server 172.139.20.17:9093 --topic test --consumer.config ~/ssl-client.properties --from-beginning

Protection Effect: TLS encrypted transport defends against man‑in‑the‑middle attacks.

Stage 3: SCRAM Shield – Authentication

Configure SCRAM‑SHA‑512

bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config 'SCRAM-SHA-512=[password=admin-password]' --entity-type users --entity-name admin

JAAS Configuration

KafkaServer {
  org.apache.kafka.common.security.scram.ScramLoginModule required
  username="admin"
  password="admin-secret";
};

Key Configuration

listeners=PLAINTEXT://:9092,SSL://:9093,SASL_PLAINTEXT://:9094
advertised.listeners=PLAINTEXT://172.139.20.17:9092,SSL://172.139.20.17:9093,SASL_PLAINTEXT://172.139.20.17:9094

# SASL settings
sasl.enabled.mechanisms=SCRAM-SHA-512

Validate SASL_PLAINTEXT

Client configuration (sasl-plain-client.properties)

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin-password";

Consume messages (SSL)

bin/kafka-console-consumer.sh --bootstrap-server 172.139.20.17:9094 --topic test --consumer.config ~/sasl-plain-client.properties --from-beginning

Security Upgrade: Username/password authentication with dynamic salt encryption blocks illegal access.

Stage 4: Dual Sword – SSL + SASL

Key Configuration

listeners=PLAINTEXT://:9092,SSL://:9093,SASL_PLAINTEXT://:9094,SASL_SSL://:9095
advertised.listeners=PLAINTEXT://172.139.20.17:9092,SSL://172.139.20.17:9093,SASL_PLAINTEXT://172.139.20.17:9094,SASL_SSL://172.139.20.17:9095

Validate SASL_SSL

Client configuration (sasl-ssl-client.properties)

security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="admin" password="admin-password";
ssl.truststore.location=/app/kafka/pki/kafka.server.truststore.p12
ssl.truststore.password=truststore_password

Consume messages (SASL_SSL)

bin/kafka-console-consumer.sh --bootstrap-server 172.139.20.17:9095 --topic test --consumer.config ~/sasl-ssl-client.properties --from-beginning

Military‑grade Protection: Transport encryption + mutual authentication + dynamic credentials meet financial‑level security requirements.

Avoiding Pitfalls (Hard‑Earned Lessons)

Certificate trap: SAN must include all broker IPs.

Protocol isolation: keep management commands on PLAINTEXT channel.

Password management: use different passwords for keystore and truststore.

Version compatibility: JDK 11+ needs PKCS12 compatibility considerations.