Why Node.js Log Decryption Fails and How Multi‑Process/Thread Fixes It

Background

In many projects, user‑side logs are recorded, stored locally, and periodically uploaded to aid rapid problem location when users report issues.

To ensure secure transmission and reduce log file size, logs are encrypted and compressed before being uploaded as a compressed package of encrypted files.

To view logs clearly, a user log management system is needed, but uploaded logs are encrypted and compressed, requiring real‑time decryption and decompression after upload.

Decryption and decompression are time‑consuming operations that require heavy computation, making it impossible to complete all decryption immediately when handling massive logs from many users, so uploaded logs have statuses such as "decrypting", "decryption completed", or "decryption failed".

A typical log system architecture is shown below:

According to decryption status changes, the process is divided into three stages:

The user terminal uploads the log to COS and notifies the backend log service, which records the log and sets its status to Undecrypted .

The log service notifies the decryption service to decrypt the newly uploaded log; after receiving a response, the status changes to Decrypting .

The decryption service completes decryption, uploads the plaintext log, and notifies the log service; the status becomes Decryption Completed . If an error occurs, the status changes to Decryption Failed .

In practice, several problems were observed:

Some logs remain in the "decrypting" state for a long time.

Many logs fail decryption (any step error sets the status to failed).

These issues are investigated below.

Problem Analysis

The first issue is logs staying in the "decrypting" state long after upload, indicating an exception in the decryption service during stage 3.

The decryption service is implemented with Node.js. Its architecture includes a master process for scheduling and load balancing, multiple worker processes for handling CGI requests, and a dedicated decryption process.

Node.js Multi‑Process Implementation

Benefits of Using Multiple Processes

Processes are the smallest unit of resource allocation; they are isolated with separate memory, reducing code complexity and coupling. A crash in a child process does not affect the main process, improving system robustness.

Drawbacks of Using Multiple Processes

Each process requires its own memory space and incurs high context‑switch overhead. Data is not shared, so inter‑process data transfer adds extra cost.

Node.js Modules for Multi‑Process

Node.js provides child_process and cluster . The child_process module offers four functions: spawn, execFile, exec, and fork. The fork method is commonly used to start another Node.js script.

Example demo:

// demo/parent.js
const ChildProcess = require('child_process');
console.log(`parent pid: ${process.pid}`);
const childProcess = ChildProcess.fork('./child.js');
childProcess.on('message', (msg) => {
    console.log("parent received:", msg);
});
// demo/child.js
console.log(`child pid: ${process.pid}`);
setInterval(() => {
    process.send(new Date());
}, 2000);
$ cd demo && node parent.js // run parent.js

Task manager shows the corresponding Node.js processes with their PIDs.

Node.js Inter‑Process Communication

Common Communication Methods

Two independent processes can exchange information via signals, sockets, shared memory, or named pipes.

Signals

Signals can be sent with the kill command, e.g., kill -USR2 3000. In Node.js, process.on('SIGUSR2', () => { ... }) receives the signal.

// Receiver
console.log("PID", process.pid);
setInterval(() => {
    console.log("PROCESS 1 is alive");
}, 5000);
process.on('SIGUSR2', () => {
    console.log("Received USR2 signal");
});
// Sender
const ChildProcess = require('child_process');
const result = ChildProcess.execSync('kill -USR2 58241');

Sockets

// Server
const net = require('net');
let server = net.createServer((client) => {
    client.on('data', (msg) => {
        console.log('ONCE', String(msg));
        client.write('server send message');
    })
});
server.listen(8087);
// Client
const net = require('net');
const client = new net.Socket();
client.connect('8087', '127.0.0.1');
client.on('data', (data) => console.log(String(data)));
client.write('client send message');

Shared Memory

Node.js does not provide native shared memory; it can be achieved via C++ extensions, which is complex and omitted here.

Named Pipes

// Create pipe
$ mkfifo /tmp/nfifo
// Server
const fs = require('fs');
fs.writeFile('/tmp/tmpipe', 'info to send', (data, err) => console.log(data, err));
// Client
const fs = require('fs');
fs.readFile('/tmp/tmpipe', (err, data) => {
    console.log(err, String(data));
});

Native Node.js Communication

// Master process (process.js)
const fork = require('child_process').fork;
const worker = fork('./child_process.js');
worker.send('start');
worker.on('message', (msg) => {
    console.log(`SERVER RECEIVED: ${msg}`);
});
// Child process (child_process.js)
process.on('message', (msg) => {
    console.log('CLIENT RECEIVED', msg);
    process.send('done');
});

Brother‑Process Communication

Brother processes communicate via a common parent: a child sends a message to the parent, which forwards it to the other child. This adds latency, so native named‑pipe (Windows) or Unix‑domain socket (*nix) is preferred.

// Server (Unix socket)
const net = require('net');
let server = net.createServer((client) => {
    client.on('data', (msg) => {
        console.log(String(msg));
        client.write('server send message');
    })
});
server.listen('/tmp/unix.sock');
// Client
const net = require('net');
const client = new net.Socket();
client.connect('/tmp/unix.sock');
client.on('data', (data) => console.log(String(data)));
client.write('client send message');

Sticky‑Packet Problem and Solution

When TCP combines multiple messages, JSON parsing fails with "Unexpected token {". The root cause is unclear data boundaries. Solutions include prefixing messages with length or using start/end delimiters.

Exception Handling

Decryption failures were often caused by inter‑process communication errors, especially when the decryption process exited unexpectedly. Common Node.js exit reasons include:

Event loop has no pending work.

Explicit process.exit() call.

Uncaught exceptions.

Unrejected promises (Node v15+).

Unlistened error events.

Unhandled signals.

Mitigations:

Keep the event loop busy (e.g., setInterval).

Avoid unnecessary process.exit().

Wrap code in try…catch and use process.setUncaughtExceptionCaptureCallback.

Handle promise rejections with .catch or process.on('unhandledRejection').

Check for listeners before emitting 'error'.

Register signal handlers with process.on('SIG...').

Node.js Multi‑Threading

The decryption process creates multiple threads using the built‑in worker_threads module, which provides isMainThread, Worker, and parentPort for communication.

// Worker thread
const { parentPort } = require('worker_threads');
parentPort.postMessage('msg');
// Main thread
const { Worker } = require('worker_threads');
const worker = new Worker('filepath');
worker.on('message', (msg) => console.log(msg));

Benchmarks on an 8‑core CPU showed:

Multi‑thread < Multi‑process < Single‑thread.

Multi‑thread is fastest because it fully utilizes CPU cores.

Multi‑process is slightly slower due to higher process‑creation overhead.

Thread‑Pool Issue

The thread pool removes idle threads by listening to the 'exit' event. However, after worker.terminate() the thread’s threadId becomes –1, causing the removal logic to delete the wrong entry and later assign tasks to a terminated thread, leading to communication timeouts.

Memory Leak Handling

Long‑running services can suffer memory leaks when objects (e.g., caches) grow unchecked. Using heapdump to capture heap snapshots helps locate leaks. In this case, tasks stuck in the thread pool due to communication timeouts kept memory occupied; adding a timeout and fixing the pool logic resolved the leak.

NPM Package Publishing Process

Register an npm account.

Initialize a local package with npm init.

Login via npm login.

Publish with npm publish (rename if name conflict).

Verify with npm view <package-name> and install via npm i <package-name>.

Results

Before fixing: many logs remained in the "decrypting" state and decryption failures were frequent.

After fixing: all logs decrypted successfully with no exceptions.