Step-by-Step Guide to Building a 3-Node Apache Storm Cluster on CentOS

Experiment Environment

CentOS 6.9

apache-storm-1.1.0

Three virtual machines: 192.168.1.129, 192.168.1.214, 192.168.1.241

Preparation

1. Modify the hostname and /etc/hosts on each server so they can be accessed by hostname. #hostname xuliugen129 Or edit /etc/hostname directly.

2. Update /etc/hosts to enable ping between the three nodes.

3. Stop the firewall on all three servers.

service firewalld stop

Zookeeper Installation and Configuration

Refer to the previous article for Zookeeper cluster setup; no additional steps are repeated here.

Storm Installation and Configuration

1. Download Apache Storm 1.1.0, for example from the Tsinghua mirror.

wget http://mirrors.tuna.tsinghua.edu.cn/apache/storm/apache-storm-1.1.0/apache-storm-1.1.0.tar.gz

Extract the archive to /home/xuliugen/server on each node. /home/xuliugen/server 2. Edit storm.yaml located in /home/xuliugen/server/apache-storm-1.1.0/conf. Only a few sections need to be changed (see image).

Apply the same configuration on all three servers.

Storm Startup and Usage

1. Storm commands are located in /home/xuliugen/server/apache-storm-1.1.0/bin.

2. Start the Nimbus daemon on the host xuliugen129.

Verify that Nimbus started correctly with jps.

3. Start the Storm UI on the same machine.

After startup, jps shows the core process. Access the UI at http://192.168.1.129:8080.

4. Start supervisor daemons on the other two hosts ( xuliugen214 and xuliugen241).

./storm supervisor

The Storm UI now displays the three‑node cluster.

5. Finally, check Zookeeper nodes on any Zookeeper server.

Project Code Usage

Source code can be downloaded from CSDN or Baidu Cloud.

The core jar is storm-core-1.1.0.jar. A topology consists of Spouts and Bolts linked by Stream Groupings (see diagram).

Key code snippets (Spout, UpperBolt, AppendBolt, TopologyMain) are shown as images.

To submit the topology:

Export the project as a jar and upload it to /home/xuliugen/temp on the Nimbus host.

Create required directories on the supervisor hosts, e.g., /home/xuliugen/server/stormdata/.

Run the submission command:

./storm jar /home/xuliugen/temp/stormdemo.jar com.xuliugen.demo.topology.TopologyMain

After submission, the topology appears in the Storm UI.

Running results are shown below.