Deep Dive into ZooKeeper ServerCnxn: Request Reception, Session Management, and Transaction Processing

ZooKeeper uses ServerCnxn as the main class for network communication, with two concrete implementations: NIOServerCnxn and NettyServerCnxn. The article focuses on the NIO version.

The server creates a NIOServerCnxn instance for each client to handle its requests. The reception code (shown in the image) reads data from the client socket using Java NIO and SelectionKey as the channel handle.

When a request arrives, readPayload distinguishes between an already‑initialized connection and one that is still initializing. Uninitialized connections are treated as session‑creation requests.

Session creation involves several steps: deserializing ConnectRequest, checking read‑only mode, validating the client’s zxid, determining sessionTimeout, and either reconnecting with an existing sessionId or generating a new one.

The SessionTracker (implemented by SessionTrackerImpl) registers and activates sessions. It maintains three collections: sessionsById, sessionsWithTimeout, and sessionSets (buckets of sessions that will expire around the same time). Activation updates the expiration bucket using roundToInterval.

public class SessionTrackerImpl extends Thread implements SessionTracker

Session expiration is performed by locating the appropriate bucket and marking all contained sessions as expired.

After a session is created, the client’s next request will be processed by readRequest:

if (!initialized) {
    readConnectRequest();
} else {
    readRequest();
}

Transaction requests (e.g., a node update) are handled by ZookeeperServer.processPacket, which deserializes a RequestHeader, determines the request type, and forwards the request to the processing chain.

Before forwarding, the server increments the outstanding request counter and may throttle incoming traffic if the limit is exceeded:

protected void incrOutstandingRequests(RequestHeader h) {
    if (h.getXid() >= 0) {
        synchronized (this) { outstandingRequests++; }
        synchronized (this.factory) {
            if (zkServer.getInProcess() > outstandingLimit) {
                disableRecv();
            }
        }
    }
}

The request‑processor chain for a Leader server is built as follows:

@Override
protected void setupRequestProcessors() {
    RequestProcessor finalProcessor = new FinalRequestProcessor(this);
    RequestProcessor toBeAppliedProcessor = new Leader.ToBeAppliedRequestProcessor(finalProcessor, getLeader().toBeApplied);
    commitProcessor = new CommitProcessor(toBeAppliedProcessor, Long.toString(getServerId()), false);
    commitProcessor.start();
    ProposalRequestProcessor proposalProcessor = new ProposalRequestProcessor(this, commitProcessor);
    proposalProcessor.initialize();
    firstProcessor = new PrepRequestProcessor(this, proposalProcessor);
    ((PrepRequestProcessor)firstProcessor).start();
}

The chain consists of:

PrepRequestProcessor : queues the request and performs basic validation.

ProposalRequestProcessor : forwards the request, creates a proposal, sends it to followers, and invokes SyncRequestProcessor to log the transaction.

SyncRequestProcessor : writes the transaction to the FileTxnSnapLog, triggers snapshots when thresholds are met, and flushes logs.

CommitProcessor : matches proposals from the leader with acknowledgments from followers and passes committed requests downstream.

Leader.ToBeAppliedRequestProcessor : ensures the request order before final application.

FinalRequestProcessor : applies the transaction to the in‑memory DataTree, updates the zxid, and records the proposal in the in‑memory committedLog.

Data persistence in ZooKeeper works via two mechanisms: a write‑ahead log (multiple log files named by the first zxid in the file) and periodic snapshots of the entire in‑memory database. Snapshots are triggered after a configurable number of log entries, with a random offset to avoid simultaneous snapshots across servers.

Session activation and expiration are also logged. The addCommittedProposal method stores proposals in an in‑memory list with a size limit, removing the oldest entry when the limit is exceeded.

public void addCommittedProposal(Request request) {
    WriteLock wl = logLock.writeLock();
    try {
        wl.lock();
        if (committedLog.size() > commitLogCount) {
            committedLog.removeFirst();
            minCommittedLog = committedLog.getFirst().packet.getZxid();
        }
        // serialize request header and txn ...
        committedLog.add(p);
        maxCommittedLog = p.packet.getZxid();
    } finally {
        wl.unlock();
    }
}

Finally, the server builds a ReplyHeader and sends the response back to the client via ServerCnxn.sendResponse, optionally closing the session if required.

ReplyHeader hdr = new ReplyHeader(request.cxid, lastZxid, err.intValue());
cnxn.sendResponse(hdr, rsp, "response");
if (closeSession) { cnxn.sendCloseSession(); }

Overall, the article provides a comprehensive walkthrough of how ZooKeeper receives client connections, creates and tracks sessions, processes transaction requests, and manages persistence through its layered request‑processor architecture.