How to Build a Scalable Netty‑Based Push System for Millions of IoT Devices

Introduction

In recent IoT development work, the need arose for a system that can handle massive device connections, support bidirectional communication, and push messages to devices. The solution must also be applicable to web chat, server‑push scenarios, and SDK‑based messaging platforms.

Technical Selection

To meet high connection counts and full‑duplex communication, traditional Java IO was discarded in favor of NIO, and Netty was chosen for its community support and performance.

Below is the overall architecture diagram:

Protocol Parsing

A custom lightweight protocol is defined instead of HTTP to reduce unnecessary data transfer and to support full‑duplex interaction. Security fields are reserved in the protocol.

Simple Implementation

4.1 Registration & Authentication

Clients first register via an HTTP request, receive a token, and store the token locally. The token is validated against Redis or a database, and the client then establishes a long TCP connection to the push‑server.

4.2 Channel Mapping

After a client connects, its unique identifier (e.g., phone number) is mapped to the Netty Channel and stored in a Map. The mapping is also saved as an attribute on the channel so that the server can retrieve the identifier later.

Retrieving the phone number from the channel:

Note: The map that stores client‑channel relationships should be pre‑sized to avoid frequent resizing, as it is the most memory‑intensive object.

4.3 Message Upstream

Incoming messages are first classified (text, image, video, etc.) using a header field or a simple JSON field. After parsing, the message is handed off to business logic, which should be decoupled via an interface and reflection, similar to the lightweight cicada framework.

Pseudo‑code for the interface‑based processing is shown in the following images:

4.4 Message Downstream

For downstream delivery, the server looks up the target client’s Channel from the map and forwards the message. Broadcast or system notifications iterate over the map and send to each channel.

Distributed Solution

5.1 Architecture Overview

The single‑node design is extended to a cluster to support millions of connections. Nginx load‑balances the registration/auth service, which returns a token and selects an appropriate push‑server. The management platform monitors online counts and pushes messages.

5.2 Service Discovery

Each push‑server registers its address in Zookeeper at startup. The registration/auth module subscribes to Zookeeper to obtain the latest list of push‑servers.

5.3 Routing Strategies

Round‑robin allocation

Hash‑mod (similar to HashMap)

Consistent hashing with optional rebalance

Weight‑based dynamic load adjustment

5.4 Stateful Connections

Client‑channel relationships are stored in Redis so that any node can locate the correct push‑server for a given client. When a client disconnects, the entry is removed.

5.5 Push Routing

For mass notifications (e.g., 100,000 clients), the platform distributes the target IDs via Nginx to a push‑router, which then queries Redis for the corresponding push‑servers and forwards the messages using HTTP or Netty.

5.6 Message Flow

High‑volume upstream data can be offloaded to Kafka for decoupling; downstream consumers read from Kafka and persist to a database.

Distributed Issues

6.1 Monitoring

Health of each push‑server node (memory, GC, OS usage) is tracked, along with online client counts from both the node and Redis to ensure consistency.

6.2 Logging

Every request includes a trace‑ID for end‑to‑end log correlation. Tools like ELK are used for log aggregation and analysis.

Conclusion

Building a reliable push system involves many considerations—registration, authentication, channel management, routing, state handling, monitoring, and logging. Practical experience is essential to avoid pitfalls and achieve a stable, scalable solution.