How cnpm rapid Accelerates npm Installations by Up to 10× with FUSE and Overlay

Background

The speaker, a Node.js engineer from Ant Group, shares the implementation details of the cnpm rapid mode, which was previously presented at SEE Conf as a method for second‑level npm installation speed.

Table of Contents

The talk is divided into three parts: analysis of cnpm rapid mode, how enterprises can integrate rapid mode to accelerate CI/CD, and how to contribute to the open‑source project.

cnpm rapid Mode Analysis

Performance comparison shows that cnpm rapid is up to 10× faster than the slowest npm, and 3× faster than the fastest pnpm and regular cnpm.

The benchmark environment includes a set of internal tests.

npm installation consists of four main steps: dependency‑tree generation, package download, extraction to node_modules, and script execution/permission changes.

Dependency tree generation

Package download

Package extraction to node_modules Other scripts and permission changes

Key bottlenecks are identified:

2211 registry requests to build the dependency tree.

2211 package downloads.

Extraction of tar.gz packages into many small files.

Network request optimization is achieved by moving dependency‑tree generation to a server‑side service that caches metadata in memory and distributed caches, reducing the process to a single HTTP request.

Disk I/O is optimized by writing tar.gz files directly to disk, avoiding the overhead of many small file writes. Since npm packages are typically ~16 KB tarballs that expand into many small files, writing a single large file is much faster.

To keep the filesystem read‑only for the tar buckets while allowing writes, a FUSE‑based user‑space filesystem (Nydus) is used together with an overlay filesystem (lower read‑only layer, upper writable layer, merged view).

Integrating rapid mode to accelerate CI/CD

The CI/CD integration consists of three steps: server‑side dependency‑tree generation, high‑speed package download and tar writing, and constructing a read‑write node_modules using Nydus and overlay.

Dependency‑tree generation uses @npmcli/arborist on the server with caching to speed up metadata retrieval.

Installation can be done by integrating npminstall with npmfs, following three steps: send package.json to the server, download and write packages with the optimized downloader, then construct node_modules via Nydus and overlay.

Send package.json to generate the dependency tree.

Download packages with the optimized writer.

Build node_modules using Nydus and overlay.

Pre‑install scripts must run in the upper writable layer before the overlay is merged; install and post‑install scripts run after the merge.

In Docker containers, enabling the FUSE device allows the overlay filesystem to work.

How to contribute to cnpm rapid

The project is open‑source on GitHub (cnpm and npminstall repositories). Contributors can help improve macOS stability, enhance the FUSE/overlay stack, and add features.

Project URLs:

https://github.com/cnpm/cnpm

https://github.com/cnpm/npminstall

Future Plans

Future work focuses on macOS process keep‑alive, stable upper‑layer filesystem, and aligning the developer experience with Linux CI/CD pipelines.

The ecosystem now has multiple npm installers (npm, pnpm, yarn, cnpm). Standardization is needed; corepack already bundles npm, pnpm, and yarn, and a TC39‑style governance model could further improve consistency, security, and user experience.