How to Build a Youzan‑Custom NSQ Client with Partitioning, Lookup, and Message Tracing

Overview

Youzan's self‑developed NSQ adds data partitioning and replication for high availability and load balancing, introduces partition‑based message production/consumption, ordered consumption, and message tracing. Building a client for this version requires extending the standard NSQ client while preserving compatibility.

Workflow and Configuration

The client workflow includes service discovery, nsqd connection, message send/receive, and ordered consumption. Configuration follows the original NSQ spec, but the custom version mandates using nsqlookupd for service discovery because nsqd nodes can change roles dynamically.

nsqd Discovery Service

Clients query the custom nsqlookupd via HTTP GET

/lookup?topic={topic_name}&access={r|w}&metainfo=true

. The access parameter distinguishes producer (w) and consumer (r) requests, while metainfo returns partition metadata (total partitions and replica count). Example queries:

Producer: /lookup?topic=test&access=w&metainfo=true Consumer: /lookup?topic=test&access=r The lookup response is JSON, e.g.:

{
    "status_code":200,
    "status_txt":"OK",
    "data":{
        "channels":["BaseConsumer"],
        "meta":{"partition_num":1,"replica":1},
        "partitions":{"0":{...}},
        "producers":[{...}]
    }
}

nsqd Connection

Clients establish a TCP connection to each partition returned in the partitions field. The connection sequence follows the standard NSQ protocol:

Send magic flag

Send IDENTIFY command and process response

Send

SUB <topic> <channel_name> <topic_partition>

(or SUB_ORDER for ordered topics) and handle response

Send RDY command

SUB command syntax:

SUB <topic> <channel_name> <topic_partition>

Possible responses:

OK

E_INVALID
E_BAD_TOPIC
E_BAD_CHANNEL
E_SUB_ORDER_IS_MUST  // topic only supports ordered consumption

Publishing commands:

PUB <topic_name> <topic_partition>
[4-byte size][N-byte binary data]

OK

E_INVALID
E_BAD_TOPIC
E_BAD_MESSAGE
E_PUB_FAILED
E_FAILED_ON_NOT_LEADER
E_FAILED_ON_NOT_WRITABLE
E_TOPIC_NOT_EXIST

Message ACK and Flow Control

Consumers acknowledge successful processing with FIN. On failure they send REQ <delay_ms> to requeue the message. If processing takes longer, TOUCH can reset the timeout. Duplicate deliveries may occur after timeout; clients should ensure idempotent handling.

Ordered Consumption

Ordered topics require producers to specify a shardingID that maps messages to a fixed partition. Consumers connect using the new SUB_ORDER command:

SUB_ORDER <topic_name> <channel_name> <topic_partition>

If a client attempts a regular SUB on an ordered topic, the server returns E_SUB_ORDER_IS_MUST and closes the connection. During ordered consumption the nsqd will not deliver the next message until the current one is finished (FIN).

Message Tracing

The custom NSQ adds a 16‑byte TraceID to the 16‑byte internal message ID. Producers can use the PUB_TRACE command to include a TraceID:

PUB_TRACE <topic_name> <topic_partition>
[4-byte size][8-byte trace id][N-byte binary data]

The response includes both internal ID and TraceID:

OK(2-bytes)+[8-byte internal id]+[8-byte trace id]+[8-byte disk queue offset]+[4-byte data size]

Clients may toggle tracing via configuration or runtime switches, printing trace information on both producer and consumer sides.

Conclusion

The article consolidates Youzan's NSQ extensions—service discovery, ordered consumption, and message tracing—and provides practical guidance for implementing a compatible client library.