Choosing the Right Protocol for AI Streaming: SSE, Chunked, or WebSocket (Part 1)

Streaming options for AI output

In AI services the server often needs to send incremental data to the client. Three HTTP‑based mechanisms are compared: Server‑Sent Events (SSE), HTTP Chunked Transfer Encoding, and WebSocket.

1. Server‑Sent Events (SSE)

What it is

SSE is a lightweight server‑push technology defined by the WHATWG HTML Living Standard. It streams plain‑text events over an HTTP connection with Content‑Type: text/event-stream.

Mechanics

Connection keep‑alive – relies on the HTTP keep‑alive header.

Automatic reconnection – browsers reconnect after the interval specified by the retry field.

Resume support – if the server sends an id field the browser stores it and resends Last‑Event‑ID on reconnect.

HTTP/2 multiplexing – multiple SSE streams can share a single TCP connection (limited by SETTINGS_MAX_CONCURRENT_STREAMS = 100).

Limitations of the native EventSource

The built‑in EventSource API only supports GET, cannot set custom headers (e.g., Authorization) and cannot send a JSON body. OpenAI works around this by using fetch + ReadableStream to parse the SSE format, enabling POST with a JSON payload.

Reference specifications

WHATWG HTML Living Standard – https://html.spec.whatwg.org/multipage/server-sent-events.html<br/>MDN Server‑sent events – https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events

Python FastAPI example

from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
import asyncio, json, time

app = FastAPI()

@app.get("/stream")
async def sse_stream(request: Request):
    async def event_generator():
        count = 0
        while True:
            if await request.is_disconnected():
                break
            yield f"id: {count}
"
            payload = {"time": time.time(), "count": count}
            yield f"data: {json.dumps(payload)}

"
            count += 1
            await asyncio.sleep(1)
    return StreamingResponse(event_generator(),
        media_type="text/event-stream",
        headers={"Cache-Control":"no-cache","Connection":"keep-alive","X-Accel-Buffering":"no"})

JavaScript client example

const source = new EventSource('/stream');
source.onmessage = e => {
    const data = JSON.parse(e.data);
    console.log('Received:', data);
};
source.onerror = e => {
    if (e.target.readyState === EventSource.CLOSED) {
        console.error('Connection permanently closed');
    } else {
        console.log('Reconnecting...');
    }
};

Key pitfalls

Only unidirectional (server → client).

Native EventSource limited to GET; POST requires manual parsing or a library.

Binary data must be Base64‑encoded.

HTTP/1.1 browsers limit to six concurrent connections per host.

Typical use cases

AI large‑model token streaming (OpenAI, Claude, Gemini).

Real‑time price feeds, social updates.

Log dashboards.

Call sequence:

2. HTTP Chunked Transfer Encoding

What it is

Chunked Transfer Encoding is defined in RFC 9112 §7.1 (HTTP/1.1) and RFC 9113 §6.1 (HTTP/2). It allows a server to send a response body in a series of length‑prefixed chunks when the total size is unknown. It is a transport‑layer feature, not a separate protocol.

Mechanics

Each chunk starts with its length in hexadecimal followed by \r\n, the data, and another \r\n. A zero‑length chunk terminates the response.

Can be combined with Content‑Encoding: gzip (compress then chunk).

Designed for dynamic content where the size cannot be predetermined (e.g., large queries, live logs).

Reference specifications

RFC 9112 – https://datatracker.ietf.org/doc/html/rfc9112#section-7.1<br/>RFC 9113 – https://datatracker.ietf.org/doc/html/rfc9113#section-6.1

Python Flask/FastAPI example

from flask import Flask, Response
import time

app = Flask(__name__)

@app.route('/chunked')
def chunked_download():
    def generate_chunks():
        yield "First chunk data
"
        time.sleep(1)
        yield "Second chunk data
"
        time.sleep(1)
        yield "Third chunk data
"
        time.sleep(1)
        yield "End
"
    return Response(generate_chunks(), content_type='text/plain')

JavaScript fetch client

async function fetchChunked() {
    const response = await fetch('/chunked');
    const reader = response.body.getReader();
    const decoder = new TextDecoder();
    let buffer = '';
    while (true) {
        const {done, value} = await reader.read();
        if (done) break;
        buffer += decoder.decode(value, {stream:true});
        const lines = buffer.split('
');
        buffer = lines.pop() || '';
        for (const line of lines) {
            console.log('Current accumulated data:', line);
        }
    }
}

Key pitfalls

The read() method may return data that does not align with chunk boundaries; browsers strip the length header and CRLF, delivering only the raw payload.

No built‑in event semantics – the stream is just a sequence of bytes.

No automatic reconnection; resumable downloads require Range requests.

Typical use cases

Large file download with progressive write to disk.

Progressive image rendering (blur‑to‑sharp).

Streaming JSON where each chunk is a separate JSON object.

Video streaming in progressive chunks.

Call sequence:

3. WebSocket

What it is

WebSocket (RFC 6455) provides a full‑duplex communication channel over a single TCP connection. After an HTTP Upgrade handshake the protocol switches to a lightweight frame format.

Mechanics

Handshake – client sends GET /… with Upgrade: websocket and Connection: Upgrade; server replies 101 Switching Protocols and Sec‑WebSocket‑Accept.

Frame structure – each frame contains FIN, Opcode (text, binary, close, ping, pong), MASK (client‑to‑server must be set), payload length and optional masking key. Header size ranges from 2 to 14 bytes.

Ping/Pong – automatic heartbeat handled by browsers and server libraries.

Reference specifications

RFC 6455 – https://datatracker.ietf.org/doc/html/rfc6455<br/>W3C WebSocket API – https://websockets.spec.whatwg.org/

Python websockets 13+ example

import asyncio, json, websockets

async def chat_handler(websocket):
    print(f"Client connected from {websocket.remote_address}")
    try:
        async for message in websocket:
            data = json.loads(message)
            print(f"Received JSON: {data}")
            await websocket.send(json.dumps({"echo": data}))
    except websockets.ConnectionClosed:
        print("Client disconnected")

async def main():
    async with websockets.serve(chat_handler, "0.0.0.0", 8080):
        print("WebSocket server running on ws://0.0.0.0:8080")
        await asyncio.Future()

if __name__ == "__main__":
    asyncio.run(main())

JavaScript browser client

const ws = new WebSocket('wss://example.com/chat');
ws.onopen = () => console.log('Connected');
ws.send(JSON.stringify({type:'join', room:'general'}));
ws.onmessage = event => {
    try {
        const data = JSON.parse(event.data);
        console.log('Received:', data);
    } catch {
        console.log('Received binary:', event.data);
    }
};
ws.onclose = e => console.log(`Closed: code=${e.code}`);
ws.onerror = err => console.error('Error:', err);
// simple manual reconnection
setTimeout(() => new WebSocket('wss://example.com/chat'), 3000);

Key pitfalls

Requires an HTTP Upgrade step; some legacy proxies or firewalls may block it.

Stateful connection – scaling often needs sticky sessions or shared state.

No built‑in automatic reconnection; the client must implement it.

Browser limits concurrent WebSocket connections (~255 in Chrome).

Typical use cases

Real‑time chat (Discord, web‑based WeChat).

Multiplayer online games (position sync, state broadcast).

Collaborative editing (Google Docs, Figma).

High‑frequency trading data feeds.

Call sequence:

Takeaways

One‑way push → SSE; raw byte streaming → Chunked Transfer Encoding; true bidirectional communication → WebSocket.

SSE is ideal for AI token streams and notification‑style updates. Chunked Transfer Encoding is a generic HTTP mechanism for streaming arbitrary binary data when the total size is unknown. WebSocket provides genuine full‑duplex messaging with minimal overhead.