How Whistle Implements a Powerful Node.js HTTP Proxy for Real‑Time Debugging

Introduction This article helps you understand Whistle's implementation principles and learn how to build a simple packet‑capture debugging tool.

Whistle is a cross‑platform web packet‑capture debugging (HTTP) proxy built on Node.js. Its main features are:

Real‑time capture: supports HTTP, HTTPS, HTTP2, WebSocket, TCP and other common web requests.

Modify request/response: instead of breakpoint debugging, Whistle uses rule‑based configuration similar to system hosts.

Extensibility: supports plugins written in Node or inclusion as independent NPM packages.

The article covers the following topics:

What is an HTTP proxy

Implement a simple HTTP proxy

Full HTTP proxy architecture (Whistle)

Specific implementation details

Reference materials

1. What is an HTTP Proxy

An HTTP proxy is an intermediary service between client and server. Without a proxy, the client connects directly to the server. With a proxy, the client first connects to the proxy, sends the target server address, and the proxy then establishes the connection on behalf of the client.

2. Implement a Simple HTTP Proxy

Below is a minimal Node.js HTTP proxy implementation:

<code>const http = require('http');
const { connect } = require('net');

// Utility to parse host and port
const getHostPort = (host, defaultPort) => {
  let port = defaultPort || 80;
  const index = host.indexOf(':');
  if (index !== -1) {
    port = host.substring(index + 1);
    host = host.substring(0, index);
  }
  return { host, port };
};

const getOptions = (req, defaultPort) => {
  const { host, port } = getHostPort(req.headers.host, defaultPort);
  return {
    hostname: host,
    port,
    path: req.url || '/',
    method: req.method,
    headers: req.headers,
    rejectUnauthorized: false,
  };
};

const handleClose = (req, res) => {
  const destroy = err => {
    req.destroy();
    res && res.destroy();
  };
  res && res.on('error', destroy);
  req.on('error', destroy);
  req.once('close', destroy);
};

const server = http.createServer();
// Handle normal HTTP requests
server.on('request', (req, res) => {
  const client = http.request(getOptions(req), svrRes => {
    res.writeHead(svrRes.statusCode, svrRes.headers);
    svrRes.pipe(res);
  });
  req.pipe(client);
  handleClose(req, client);
});

// Handle tunnel (HTTPS, HTTP2, WebSocket, TCP) requests
server.on('connect', (req, socket) => {
  const client = connect(getHostPort(req.url), () => {
    socket.write('HTTP/1.1 200 Connection Established\r\n\r\n');
    socket.pipe(client).pipe(socket);
  });
  handleClose(socket, client);
});

server.listen(8080);
</code>

The code creates an HTTP server that forwards client requests to the target server (handling both

request

and

connect

events). The

connect

event enables tunnel proxying for HTTPS, HTTP2, WebSocket, and TCP.

3. Full HTTP Proxy Architecture (Whistle)

The architecture consists of five modules:

Request entry module

Tunnel proxy module

HTTP request handling module

Rule management module

Plugin management module

4. Detailed Implementation

4.1 Request Entry Module

Whistle supports four entry methods:

Direct HTTP/HTTPS requests (similar to hosts/DNS mapping)

HTTP proxy (default entry, configured via system proxy or browser extension)

HTTPS proxy (encrypted HTTP proxy built on top of the HTTP proxy)

Socks5 proxy (uses the

socksv5

npm package to convert TCP to tunnel requests)

All requests are ultimately converted to either a tunnel proxy request or a normal HTTP request for further processing.

4.2 Tunnel Proxy Module

Key points:

Global rules determine whether a tunnel request should be parsed; otherwise it is treated as a raw TCP stream.

If parsing is needed, the first data frame is read via

socket.once('data', handler)

.

The frame is converted to a string and matched against

/^(\w+)\s+(\S+)\s+HTTP\/1.\d$/mi

to detect an HTTP request. If it is a

CONNECT

request, it is processed as a tunnel; otherwise it is handed to the HTTP request module.

Non‑HTTP frames are treated as HTTPS and processed using a man‑in‑the‑middle approach with a generated or custom certificate.

Certificate acquisition order: plugin‑provided via

sniCallback://plugin

, command‑line

-z certDir

or

~/.WhistleAppData/custom_certs

, then auto‑generated default certificate.

After obtaining a certificate, an HTTPS server is started to convert the HTTPS request into an HTTP request for the HTTP request module.

4.3 HTTP Request Handling Module

The processing consists of two phases:

Request phase

Match global rules.

If a rule references a plugin (e.g.,

whistle.xxx

), invoke the plugin hook to obtain additional rules and merge them.

Execute the final rule set and forward the request to the target service.

Response phase

Invoke matching plugin hooks to obtain response‑side rules.

Apply the merged rule set and send the modified response back to the client.

4.4 Rule Management

Whistle modifies requests/responses via configuration rules rather than breakpoint editing. Rules are divided into:

Global rules : applied to all requests; include UI‑configured rules, plugin

rules.txt

, remote rules imported with

@url

, etc.

Plugin rules : private to a plugin; provided by plugin hooks (

reqRulesServer

) or static files like

_rules.txt

.

4.5 Plugin Management

Plugins extend Whistle's capabilities, offering features such as authentication, UI interaction, request/response handling, traffic statistics, rule setting, packet capture, stream encoding/decoding, context‑menu extensions, data synchronization, and custom HTTPS certificates. Examples include

whistle.script

,

whistle.vase

,

whistle.inspect

, and

whistle.sni-callback

.

Design principles:

Completeness : every functional point (auth, cert generation, capture, rule setting, request handling) is extensible.

Stability : plugin failures do not affect core functionality; each plugin runs in an isolated process communicating via HTTP.

Usability : simple npm‑based development and scaffolding tools (e.g.,

lack

).

5. References

GitHub repository: https://github.com/avwo/whistle

Official plugin repository: https://github.com/whistle-plugins

Documentation: https://wproxy.org/whistle/