How to Build a Million‑Connection Netty Server: Tips, Code, and Tuning

What is Netty

Netty (http://netty.io/) is an asynchronous event‑driven network application framework for rapid development of high‑performance protocol servers and clients.

High‑performance, scalable architecture. Zero‑Copy to reduce memory copying.

Native socket support on Linux.

Compatible with Java 1.7 NIO2 and earlier NIO. Pooled Buffers reduce buffer allocation pressure.

Bottlenecks Overview

The two primary goals of a long‑connection service are:

Support as many concurrent connections as possible.

Achieve the highest possible queries per second (QPS).

More Connections

Non‑Blocking I/O

Both Java NIO and Netty use non‑blocking I/O, so a thread per connection is not required.

Java NIO Sample for a Million Connections

ServerSocketChannel ssc = ServerSocketChannel.open();
Selector sel = Selector.open();
ssc.configureBlocking(false);
ssc.socket().bind(new InetSocketAddress(8080));
SelectionKey key = ssc.register(sel, SelectionKey.OP_ACCEPT);
while (true) {
    sel.select();
    Iterator<SelectionKey> it = sel.selectedKeys().iterator();
    while (it.hasNext()) {
        SelectionKey skey = it.next();
        it.remove();
        if (skey.isAcceptable()) {
            ssc.accept(); // accept only, no business logic
        }
    }
}

Netty Implementation for a Million Connections

NioEventLoopGroup bossGroup = new NioEventLoopGroup();
NioEventLoopGroup workerGroup = new NioEventLoopGroup();
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup);
bootstrap.channel(NioServerSocketChannel.class);
bootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
    @Override
    protected void initChannel(SocketChannel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        // add handlers here
    }
});
bootstrap.bind(8080).sync();

Kernel Limits

The real bottleneck is often the Linux kernel configuration that limits open files and sockets. Adjust ulimit -n, fs.file-max, and net.core.somaxconn to increase the limit.

Validating Capacity

A Netty test client can open ~60 000 connections on a single machine (limited by available ports). To reach a million connections, multiple machines or virtual machines with bridged network interfaces are required. Disabling keep-alive on the server prevents automatic disconnection of idle sockets.

Netty Test Client for Connection Validation

NioEventLoopGroup workerGroup = new NioEventLoopGroup();
Bootstrap b = new Bootstrap();
b.group(workerGroup);
b.channel(NioSocketChannel.class);
b.handler(new ChannelInitializer<SocketChannel>() {
    @Override
    protected void initChannel(SocketChannel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        // add handlers
    }
});
for (int i = 0; i < 60000; i++) {
    b.connect("127.0.0.1", 8080);
}

Higher QPS

With non‑blocking I/O, QPS does not degrade as the number of connections grows, provided sufficient memory.

Optimizing Data Structures

Avoid costly operations such as frequent size() on ConcurrentLinkedQueue. Replace with an AtomicInteger counter to track size, which reduces traversal overhead.

CPU Bottleneck Diagnosis

Run a stress test and use VisualVM (or VisualGC) to identify hot methods. Sort by self‑time to locate the most CPU‑intensive code paths.

GC Bottleneck Mitigation

Excessive old‑generation GC can be reduced by increasing the young generation size, e.g., -XX:NewRatio=3 (young:old = 1:3). In production, allocate more old‑gen space when connections persist for long periods.

Final Results

On a 16‑core, 120 GB RAM machine (JVM heap limited to 8 GB) running Java 1.6, the team achieved 600 k concurrent connections and 200 k QPS with low system load, indicating the remaining bottleneck was I/O rather than CPU or memory. Increasing heap size and further kernel tuning can raise the connection count beyond these figures.