Capturing Node.js HTTPS Requests with Charles: A Step‑by‑Step Guide

Background

Developing a CLI tool "Kup" with Node.js.

Kup uses Got to send HTTPS requests.

Goal: use Charles to monitor communication for debugging.

The method also applies to Fiddler, Whistle, and other proxy‑based debugging tools.

Initial Attempts

Preparation

Install Charles CA certificate on the machine and set it to “Always Trust”.

Start Charles HTTP proxy at http://127.0.0.1:8888 and enable “SSL Proxying”.

(These steps are the same as for capturing browser HTTPS traffic.)

Attempt 1

Set Charles as the macOS global proxy.

Run Kup from the command line. Charles does not capture the request – failure.

Attempt 2

Remove the global proxy and set a precise terminal proxy: export https_proxy=http://127.0.0.1:8888 Still no capture – failure.

Investigation

Both attempts show that Kup does not route traffic through Charles. Two general solutions exist:

Standard approach: make the Node.js application use Charles’s proxy directly, the same principle as browser traffic.

Work‑around: run Charles as a reverse proxy, wrap the real endpoint, and point the Node.js app to the wrapped address.

The reverse‑proxy method avoids installing the CA certificate and only requires changing the endpoint address, but it introduces address‑switching complexity and poor portability, so it is discarded.

The standard approach is more cumbersome because Node.js does not automatically honor the https_proxy environment variable. Got does not have built‑in proxy support; it expects an http.Agent instance, which must be created with a separate library.

Despite the extra steps, all work stays in the Node.js code and the author is more familiar with this capture principle, so this approach is chosen.

Final Implementation

Implementation steps:

Install https-proxy-agent and create a proxy agent:

const HttpsProxyAgent = require('https-proxy-agent')
const agent = new HttpsProxyAgent('http://127.0.0.1:8888')

Pass the agent to Got:

got(myApi, {
  agent: {
    https: agent, // specify HTTPS proxy
  },
})

Because Charles’s CA is self‑signed, disable certificate verification in Got:

got(myApi, {
  agent: {
    https: agent,
  },
  rejectUnauthorized: false, // disable cert check
})

After these changes, every request made by Kup appears in Charles – success.

The minimal implementation is sufficient for debugging; in production the proxy logic should be limited to debug mode, which is omitted here.

Conclusion

The author shares the exploration and decision process, inviting readers to suggest better solutions.