Mastering Node.js Streams: From Basics to Real-World Applications

Recently the term "流" (stream) has become popular, and in Unix a stream is a fundamental concept. The Unix philosophy—"a program does one thing well and works with other programs via text streams"—highlights its importance.

What Is a Stream?

In technical terms, a stream abstracts an input or output device; it is directional data. In Unix the pipe symbol | connects streams. Node.js provides a stream module that defines an abstract class implemented by objects such as request and response.

Node.js defines four stream classes: Readable , Writable , Duplex , and Transform . Duplex and Transform combine readable and writable capabilities, so understanding readable and writable streams is the priority.

Creating a Readable Stream

You can instantiate a readable stream directly:

var Readable = require('stream').Readable;
var rs = new Readable();
rs.push('beep ');
rs.push('boop ');
rs.push('null
');
rs.push(null);
rs.pipe(process.stdout);

Running the code outputs the data and ends when null is pushed.

Readable Stream Interface

A readable stream must implement the _read(size) method, which the runtime calls to fetch data. Example implementation:

var Readable = require('stream').Readable;
var rs = Readable();
var c = 97 - 1;
rs._read = function () {
    if (c >= 'z'.charCodeAt(0)) return rs.push(null);
    setTimeout(function () {
        rs.push(String.fromCharCode(++c));
    }, 200);
};
rs.pipe(process.stdout);
process.on('exit', function () {
    console.error('
_read() called ' + (c - 97) + ' times');
});
process.stdout.on('error', process.exit);

The stream stays in a flowing state only after its internal buffer reaches a certain level.

Stream Modes

Readable streams have two modes: flowing and paused . You can switch to flowing mode by:

Adding a data event handler.

Calling resume().

Calling pipe() to a writable stream.

And you can pause a stream by:

Calling pause() when there are no pipe destinations.

Removing all data handlers or calling unpipe() when pipes exist.

Common Applications

Instead of reading an entire file into memory with fs.readFile, you can stream it directly to the client:

var http = require('http');
var fs = require('fs');
var server = http.createServer(function (req, res) {
    var stream = fs.createReadStream(__dirname + '/kuaibo.txt');
    stream.pipe(res);
});
server.listen(8000);

This approach reduces memory usage and improves scalability for large files or many concurrent requests.

Streams can also be chained with pipe() for transformations, e.g., compressing data:

var r = fs.createReadStream('file.txt');
var z = zlib.createGzip();
var w = fs.createWriteStream('file.txt.gz');
r.pipe(z).pipe(w);

During piping, data flows as Buffer objects, and the pipe mechanism automatically manages back‑pressure by pausing and resuming the streams.

Building an HTTP Proxy with Streams

var http = require('http');
var net = require('net');
var url = require('url');
function request(cReq, cRes) {
    var u = url.parse(cReq.url);
    var options = {
        hostname: u.hostname,
        port: u.port || 80,
        path: u.path,
        method: cReq.method,
        headers: cReq.headers
    };
    var pReq = http.request(options, function (pRes) {
        cRes.writeHead(pRes.statusCode, pRes.headers);
        pRes.pipe(cRes);
    }).on('error', function () { cRes.end(); });
    cReq.pipe(pReq);
}
http.createServer().on('request', request).listen(8888, '0.0.0.0');

Conclusion

Node.js wraps the stream concept with a rich API; streams are essentially event emitters. By leveraging streams you can build efficient, low‑memory services such as file servers, proxies, and data pipelines, making it a practical choice for prototyping and gradually replacing heavier back‑end systems.