Building a Scalable Netty Push System for Millions of IoT Devices

Preface

In recent IoT development work I needed a system that supports massive device connections, device registration, and bidirectional message push, which can also serve web chat, server‑push scenarios, and SDK‑based push platforms.

Technical Selection

To handle a large number of connections with full‑duplex communication and guaranteed performance, the traditional Java IO is unsuitable, so NIO is chosen, and Netty is selected for its community support and documentation.

The overall architecture is shown below:

Protocol Analysis

A custom lightweight protocol is preferred over HTTP to meet full‑duplex requirements and reduce unnecessary data transmission. Security considerations are also embedded in the protocol.

Simple Implementation

Registration & Authentication

Clients must first register and obtain a token via an HTTP request. The token is stored in Redis or a database and attached to subsequent TCP long‑connection requests to the push‑server.

After authentication, the client establishes a TCP long‑connection to the push‑server, which handles upstream and downstream messages.

Channel Relationship Management

When a client connects, its unique identifier (e.g., phone number) is mapped to the Netty Channel. The mapping is stored in a local Map (similar to SpringBoot long‑connection heartbeat integration).

public static void putClientId(Channel channel, String clientId) {
    channel.attr(CLIENT_ID).set(clientId);
}

public static String getClientId(Channel channel) {
    return (String) getAttribute(channel, CLIENT_ID);
}

On client disconnect, the mapping is removed and a log entry is recorded:

String telNo = NettyAttrUtil.getClientId(ctx.channel());
NettySocketHolder.remove(telNo);
log.info("Client offline, TelNo=" + telNo);

It is advisable to pre‑size the Map to avoid frequent resizing, as it is a memory‑intensive structure.

Message Upstream

Incoming messages are first classified by type (text, image, video, etc.) using a header field or a simple JSON field, then processed accordingly.

Message Parsing & Business Decoupling

Parsing occurs in channelRead(). To keep business logic separate, an interface is defined for handling specific message types, and implementations are instantiated via reflection after parsing.

Pseudo‑code illustration:

Message Downstream

For point‑to‑point chat, the server forwards the message to the target client’s Channel. System notifications are broadcast by iterating the channel map.

Pseudo‑code:

Distributed Solution

The single‑node implementation is extended to support millions of connections via horizontal scaling.

Architecture Overview

Key components include Nginx for load balancing, a registration/auth module, a push‑server cluster, a management platform, and middleware such as Redis, Zookeeper, Kafka, and MySQL.

Service Registration & Discovery

Each push‑server registers its address in Zookeeper at startup. The registration/auth module subscribes to Zookeeper to obtain the latest service list.

Routing Strategy

Various algorithms are discussed to balance client connections across nodes: round‑robin, hash modulo, consistent hashing, and weighted routing. Heartbeat mechanisms detect node failures, prompting clients to re‑register and obtain a new node.

When a node restarts, pending messages are stored locally and resent after a new node is acquired.

Stateful Connections

Because connections are stateful, the mapping between client identifiers and their serving node must be stored centrally (Redis). On client disconnect, the mapping is removed.

Pseudo‑code:

Push Routing

For mass notifications (e.g., 100,000 clients), the platform distributes client IDs via Nginx to push‑routes, which then retrieve the corresponding push‑server from Redis and send messages via HTTP or Netty.

Message Flow

High‑volume upstream data can be decoupled using Kafka: messages are published to Kafka, and downstream consumers persist them to a database.

Reference: Disruptor memory overflow article

Distributed Issues

Application Monitoring

Monitoring includes node health, memory usage, GC, off‑heap memory, online client counts, and Redis metrics to ensure consistency.

Log Handling

Each request should carry a trace ID for end‑to‑end tracing. Tools like ELK are recommended for log aggregation and analysis.

Conclusion

Building a stable push system involves many aspects—protocol design, authentication, channel management, routing, state handling, distributed deployment, monitoring, and logging. Practical experience reveals many pitfalls that are hard to anticipate without hands‑on implementation.