Understanding Node.js Stream API: Types, Back‑Pressure, and Practical Usage

When constructing complex systems, it is common to split them into independent functional components whose interfaces follow a certain contract and are connected to accomplish a larger task. For example, a shell connects commands with a pipe (|) where the input and output follow a text‑stream format.

In Node.js, the built‑in Stream module provides similar functionality, allowing components to be linked via .pipe().

This article series introduces Node.js streams from several perspectives:

The basic stream types and fundamental usage of the Stream module.

The concept of stream processing and the mechanism of back pressure .

How to develop stream‑based programs, including analyses of Gulp and Browserify , plus a hands‑on example.

Four Stream Types

Node.js defines four stream categories: Readable , Writable , Duplex , and Transform .

Readable

Creates a readable stream. For instance, a readable stream can consume data from an iterator: new ToReadable(iterable) returns a readable stream that downstream code can consume.

Key points for implementing a custom readable stream:

The _read method contains the logic to produce data from the underlying source.

Inside _read , call push(data) to place data into the stream.

push may be called synchronously or asynchronously.

When all data has been produced, invoke push(null) to signal the end of the stream.

After the stream ends, further push calls are invalid.

Consumers typically listen to the data event; the first data event fires on the next tick, allowing preparatory logic to run beforehand. When the stream is exhausted, an end event is emitted.

Writable

Creates a writable stream. Custom writable streams must implement _write(data, enc, next) instead of _read.

In simple cases, a plain writable object can be used: writable.write(data) forwards data to _write. After successful processing, _write must call next(err) (synchronously or asynchronously) to signal readiness for the next chunk. The upstream signals completion with writable.end([data]); after end, no further write calls are allowed. When all underlying writes finish, a finish event is emitted.

Duplex

A duplex stream combines readable and writable capabilities by inheriting from both Readable and Writable. It implements both _read and _write, allowing it to be used as a source (listening to data) and as a sink (calling write).

Transform

A transform stream is a specialized duplex where the writable side receives data, transforms it, and automatically pushes the result to the readable side. Users implement a _transform(chunk, encoding, callback) method while the base class already provides _read and _write.

objectMode

By default, streams operate on Buffer or String data. The objectMode option, when set to true, allows streams to handle arbitrary JavaScript objects without automatic conversion to Buffer. This enables patterns such as “you reap what you sow” – the exact objects pushed are emitted downstream.

Examples illustrate the difference between the default mode and objectMode using push(data) and the resulting output.

Series Preview

The stream series consists of three articles:

Part 1 – Basics: Introduction to the Stream API.

Part 2 – Advanced: In‑depth analysis of how streams support flow‑controlled data processing and the back pressure mechanism.

Part 3 – Practical: Designing programs with streams, extracting patterns from Browserify and Gulp , and building a Git‑repository changelog generator.

References

GitHub – substack/browserify-handbook

GitHub – zoubin/streamify-your-node-program

For more technical articles, visit the Meituan Dianping technology blog.