Master WebSocket: From Handshake to Real‑Time Data Streams

Tip: Reading this article takes about 10 minutes.

Basic Introduction

Wikipedia Definition

WebSocket is a network transport protocol that enables full‑duplex communication over a single TCP connection, residing at the application layer of the OSI model. It was standardized by the IETF in 2011 as RFC 6455 and later supplemented by RFC 7936. The WebSocket API is standardized by the W3C.

WebSocket simplifies data exchange between client and server and allows the server to push data proactively. In the WebSocket API, the browser and server perform a single handshake, after which a persistent bidirectional connection is established.

Glossary :

IETF : Internet Engineering Task Force, an open standards organization that develops and promotes voluntary Internet standards, especially the TCP/IP protocol suite.

Web IDL : Interface Description Language used to describe APIs implemented in web browsers.

Browser Support

Use Cases

Any scenario that requires timely message push, such as customer service chat or email notifications, can use WebSocket.

History

Before browsers implemented WebSocket, server push was typically achieved with polling or long‑polling techniques.

Polling

Periodically sending requests to check for new messages.

Long Polling

Process :

Browser sends a request to the server.

Server keeps the connection open until a message arrives.

When a new message arrives or a timeout occurs, the server responds.

Browser immediately sends a new request.

Below is a screenshot of a long‑polling request in QQ Mail web client:

WebSocket Advantages

Lower control overhead.

Stronger real‑time performance.

Maintains connection state without cookie‑based session mechanisms.

Better binary support.

Improved compression.

Knowledge Popularization

Binary

Basic Concepts

bit : the smallest binary unit, representing

0

or

1

.

byte : 8 bits; in Node.js represented by

Uint8Array

.

KB : 1024 bytes.

MB : 1024 KB.

Signed : first bit is a sign bit; e.g.,

Int8Array

ranges from –128 to 127.

Unsigned : all bits represent magnitude; e.g.,

Uint8Array

.

Operations

Examples

<code>   1 1 0 0 1 1 0 0
 & 1 0 0 1 1 0 1 1
-----------------
   1 0 0 0 1 0 0 0
</code>

<code>   1 1 0 0 1 1 0 0
 | 1 0 0 1 1 0 1 1
-----------------
   1 1 0 1 1 1 1 1
</code>

<code>   1 1 0 0 1 1 0 0
 ^ 1 0 0 1 1 0 1 1
-----------------
   0 1 0 1 0 1 1 1
</code>

Note : XORing a number with the same value twice yields the original number, e.g.,

0b1010 == (0b1010 ^ 0b1100 ^ 0b1100) == 10

.

<code> ~ 1 1 0 0 1 1 0 0
-----------------
   0 0 1 1 0 0 1 1
</code>

<code>   1 1 0 0 1 1 0 0 << 1
-----------------------
   1 0 0 1 1 0 0 0
</code>

Note : Left shift discards the high bit and pads with 0.

<code>   1 1 0 0 1 1 0 0 >> 1
-----------------------
   0 1 1 0 0 1 1 0
</code>

Endianness

Big‑endian and little‑endian are different byte orderings, collectively called byte order .

Big‑endian : most significant byte first, matching human reading order.

Little‑endian : least significant byte first.

Stream

Streams provide two main benefits:

Memory efficiency : process data without loading the entire payload into memory.

Time efficiency : start processing as soon as data arrives, without waiting for the full payload.

Memory usage comparison with Buffer

Processing a ~61 MB file to compute an MD5 hash using Buffer versus Stream demonstrates memory differences.

Buffer

<code>const crypto = require('crypto');
const fs = require('fs');
const path = require('path');
const { performance } = require('perf_hooks');
const v8 = require('v8');

function bufferImpl() {
  v8.writeHeapSnapshot('buffer.1.heapsnapshot');

  const start = performance.now();
  const filename = path.join(__dirname, 'test.zip');
  const buffer = fs.readFileSync(filename);
  const hash = crypto.createHash('md5').update(buffer).digest('hex');

  v8.writeHeapSnapshot('buffer.2.heapsnapshot');

  console.log('Buffer Impl:');
  console.log('  Hash:', hash);
  console.log('  Cost:', performance.now() - start);
}
</code>

Stream

<code>const crypto = require('crypto');
const fs = require('fs');
const path = require('path');
const { performance } = require('perf_hooks');
const v8 = require('v8');

function streamImpl() {
  v8.writeHeapSnapshot('stream.1.heapsnapshot');

  const start = performance.now();
  const filename = path.join(__dirname, 'test.zip');
  const stream = fs.createReadStream(filename);
  const md5 = crypto.createHash('md5').setEncoding('hex');
  const hash = stream.pipe(md5);

  v8.writeHeapSnapshot('stream.2.heapsnapshot');

  console.log('Stream Impl:');
  process.stdout.write('  Hash: ');
  hash.pipe(process.stdout);
  hash.on('end', () => {
    console.log('\n  Cost:', performance.now() - start);
  });
}
</code>

Node Stream API

stream.Writable

Common methods :

.write(chunk)

– write a data chunk.

stream.Readable

Common events :

data

– data chunk received.

error

– error.

end

– read completed.

Common methods :

.pipe(writable)

– connect a readable pipe to a writable pipe, allowing data to flow directly.

Maximum Transmission Unit (MTU)

MTU refers to the largest packet size that a network protocol can transmit at the data link layer. Ethernet commonly uses an MTU of 1500 bytes. The

ifconfig

command can display the MTU of each network interface.

Maximum Segment Size (MSS)

MSS is the maximum TCP payload size negotiated after the TCP connection is established. If the underlying MTU is 1500 bytes, MSS = 1500 – 20 (IP header) – 20 (TCP header) = 1460 bytes.

Nagle Algorithm

Purpose : reduce network congestion.

Rules :

If the packet length reaches MSS, it may be sent.

If the packet contains FIN, it may be sent.

If TCP_NODELAY is set, it may be sent.

If TCP_CORK is not set and all small packets have been acknowledged, it may be sent.

If none of the above conditions are met, the packet is sent after a timeout (typically 200 ms).

In short, packets wait up to 200 ms to allow small packets to be coalesced before transmission.

TCP Sticky Packets and Fragmentation

Due to MSS and the Nagle algorithm, TCP may encounter sticky packets or fragmentation.

Protocol Analysis

1. HTTP handshake and protocol upgrade (rfc6455#section-1.2)

Client sends the following headers:

<code>GET /chat HTTP/1.1
Host: server.example.com
Origin: http://example.com

Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13
</code>

Server responds with:

<code>HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Protocol: chat
</code>

The

Sec-WebSocket-Accept

value is generated by concatenating the client key with the GUID

"258EAFA5-E914-47DA-95CA-C5AB0DC85B11"

, applying SHA‑1, and then Base64‑encoding the result.

2. Data Frame Parsing (rfc6455#section-5.2)

<code>
  0               1               2               3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-------+-+-------------+-------------------------------+
  |F|R|R|R| opcode|M| Payload len |    Extended payload length    |
  |I|S|S|S|  (4)  |A|   (7)       |   (if payload len==126/127)   |
  |N|V|V|V|       |S|             |                               |
  | |1|2|3|       |K|             |                               |
  +-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +
  |     Extended payload length continued, if payload len == 127 |
  + - - - - - - - - - - - - - - +-------------------------------+
  |               |Masking-key, if MASK set to 1                |
  +-------------------------------+-------------------------------+
  | Masking-key (continued)       |          Payload Data          |
  +-------------------------------- - - - - - - - - - - - - - - - +
  :                     Payload Data continued ...               :
  + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
  |                     Payload Data continued ...               |
  +---------------------------------------------------------------+
</code>

FIN : 1‑bit indicating final fragment of a message.

RSV1‑RSV3 : 1‑bit each for extensions.

opcode : 4‑bit operation code (e.g., 0x1 text, 0x2 binary, 0x8 close, 0x9 ping, 0xA pong).

MASK : 1‑bit indicating whether a masking key is present.

Payload len : 7‑bit length; values 0‑125 represent actual length, 126 indicates a following 16‑bit length, 127 indicates a following 64‑bit length.

Masking-key : 32‑bit key present when MASK=1.

Payload Data : optional extension data followed by application data.

3. Some Details

3.1 Masking (rfc6455#section-5.3)

Masking encrypts or decrypts payload data. The algorithm (Node.js) is:

<code>/**
 * Process control frame
 * @param data data
 * @param maskingKey mask key (4 bytes)
 */
function mask(data, maskingKey) {
  for (let i = 0; i < data.length; i++) {
    data[i] = data[i] ^ maskingKey[i % 4];
  }
}
</code>

Protocol requires clients to mask data sent to the server, while servers must not mask data sent to clients.

3.2 Ping and Pong (rfc6455#section-5.5.2)

Ping/Pong frames are used for heartbeat detection. When one side sends a ping, the other must reply with a pong.

4. Demo Implementation

See: https://github.com/peakchen90/tiny-websocket

Some Thoughts

Why does the HTTP handshake use \r\n as line terminator?

Reference: https://www.rfc-editor.org/rfc/rfc2616.html#section-2.2

How to carry sender information when sending binary files?

A custom binary protocol can reserve the first 8 bits for the length of the sender’s nickname, followed by the nickname bytes (UTF‑8), then the actual binary payload.

<code>// 0 0 0 0 0 0 0 0 : nickname length
// ...           : nickname bytes
// ...           : binary data
const nickBuffer = Buffer.from(sender['nickname']);
const headerBuffer = Buffer.allocUnsafe(1 + nickBuffer.length);
headerBuffer.writeUInt8(nickBuffer.length, 0);
headerBuffer.set(nickBuffer, 1);

wss.broadcast([headerBuffer, message], true);
</code>

How does WebSocket handle large files?

Large files can exhaust server memory. Clients should use Blob, and the server should only forward messages without reassembling fragments.

What are the responsibilities of the IP and TCP protocols?

IP locates hosts; TCP locates host processes (via ports).

Reference Links

Bilingual version : RFC 6455 Chinese translation.

English version : RFC 6455 The WebSocket Protocol.

WebSocket: 5 minutes from beginner to mastery : https://juejin.cn/post/6844903544978407431

ws: a Node.js WebSocket library : https://github.com/websockets/ws

Demo code : https://github.com/peakchen90/tiny-websocket