Why Yarn Beats npm: Deep Dive into Its Architecture and Workflow

Why Yarn?

Yarn is a JavaScript package manager similar to npm, cnpm, pnpm. It uses multi‑threaded downloading, local caching, and flat node_modules layout to improve speed and reduce duplication.

Differences with npm

Yarn downloads dependencies in parallel threads while npm is single‑threaded.

Yarn caches packages locally and prefers the cache before remote requests; npm always fetches.

Yarn flattens dependencies to a single level, reducing node_modules size compared with npm’s nested tree.

Differences with cnpm

cnpm uses a faster domestic mirror.

cnpm stores downloaded packages in its own cache and links them into node_modules via symlinks.

Differences with pnpm

Both Yarn and pnpm keep a single store of packages.

pnpm preserves the original npm dependency tree but creates symlinks for each package in node_modules.

Building a Simple Yarn Implementation

Step 1 – Download

Yarn reads the package.json to obtain the dependencies field and downloads each listed package.

{
  "dependencies": {
    "lodash": "4.17.20"
  }
}

Example code that fetches a package from the Yarn registry:

import fetch from 'node-fetch';
async function fetchPackage(packageJson) {
  const entries = Object.entries(packageJson.dependencies);
  for (const [key, version] of entries) {
    const url = `https://registry.yarnpkg.com/${key}/-/${key}-${version}.tgz`;
    const response = await fetch(url);
    if (!response.ok) {
      throw new Error(`Couldn't fetch package "${key}"`);
    }
    return await response.buffer();
  }
}

When a dependency is a local path (e.g., "../../customer-package") the code falls back to reading the file directly.

import fetch from 'node-fetch';
import fs from 'fs-extra';
async function fetchPackage(packageJson) {
  const entries = Object.entries(packageJson.dependencies);
  for (const [key, version] of entries) {
    if (['/', './', '../'].some(prefix => version.startsWith(prefix))) {
      return await fs.readFile(version);
    }
    // ...old code
  }
}

Step 2 – Flexible Version Matching

Yarn supports semver ranges such as "^15.6.0". The following helper resolves the highest satisfying version from the registry.

import semver from 'semver';
async function getPinnedReference(name, version) {
  if (semver.validRange(version) && !semver.valid(version)) {
    const response = await fetch(`https://registry.yarnpkg.com/${name}`);
    const info = await response.json();
    const versions = Object.keys(info.versions);
    const maxSatisfying = semver.maxSatisfying(versions, version);
    if (maxSatisfying === null) {
      throw new Error(`Couldn't find a version matching "${version}" for package "${name}"`);
    }
    version = maxSatisfying;
  }
  return { name, reference: version };
}

The main install routine uses this helper to replace caret/tilde ranges with a concrete version before downloading.

async function fetchPackage(packageJson) {
  const entries = Object.entries(packageJson.dependencies);
  for (const [name, version] of entries) {
    let realVersion = version;
    if (version.startsWith('~') || version.startsWith('^')) {
      const { reference } = await getPinnedReference(name, version);
      realVersion = reference;
    }
    // ...old code
  }
}

Step 3 – Recursive Dependencies

Yarn walks the dependency tree, fetching each package's own package.json and adding its dependencies to the queue.

async function getPackageDependencies(packageJson) {
  const packageBuffer = await fetchPackage(packageJson);
  const packageJson = await readPackageJsonFromArchive(packageBuffer);
  const dependencies = packageJson.dependencies || {};
  return Object.keys(dependencies).map(name => ({
    name,
    version: dependencies[name],
  }));
}

Step 4 – Linking Files

After all packages are cached, Yarn copies them into the project’s node_modules directory.

async function linkPackages({ name, reference, dependencies }, cwd) {
  const dependencyTree = await getPackageDependencyTree({ name, reference, dependencies });
  await Promise.all(
    dependencyTree.map(async dependency => {
      await linkPackages(dependency, `${cwd}/node_modules/${dependency.name}`);
    })
  );
}

Step 5 – Optimizing the Tree

Yarn flattens the tree and removes duplicate packages to keep node_modules small.

function optimizePackageTree({ name, reference, dependencies = [] }) {
  dependencies = dependencies.map(optimizePackageTree);
  for (const hardDep of dependencies) {
    for (const subDep of hardDep.dependencies) {
      const existing = dependencies.find(d => d.name === subDep.name);
      if (!existing) {
        dependencies.push(subDep);
      }
      if (!existing || existing.reference === subDep.reference) {
        hardDep.dependencies.splice(
          hardDep.dependencies.findIndex(d => d.name === subDep.name),
          1
        );
      }
    }
  }
  return { name, reference, dependencies };
}

Yarn Architecture Overview

Config : Yarn configuration files and defaults.

CLI : All Yarn commands.

Registries : Information about npm/Yarn registries, lockfiles, and package locations.

Lockfile : yarn.lock object describing exact versions.

Integrity Checker : Verifies that downloaded packages match their integrity hashes.

Package Resolver : Resolves package.json references, handling version ranges.

Package Fetcher : Downloads packages from the appropriate source.

Package Linker : Places packages into node_modules.

Package Hoister : Flattens the dependency tree.

Yarn Work Flow

Process Overview (using yarn add lodash )

checking : Validate .yarnrc, CLI arguments, package.json, and environment compatibility.

resolveStep : Resolve the full dependency tree and exact versions.

fetchStep : Download any missing packages into the cache.

linkStep : Flatten and copy cached packages into node_modules.

buildStep : Compile binary packages if needed.

Detailed Steps

Initialization – locating .yarnrc

const rc = getRcConfigForCwd(process.cwd(), process.argv.slice(2));
function getRcPaths(name, cwd) {
  // ...search parent directories, /etc, home, etc.
}

Parsing user command

const doubleDashIndex = process.argv.findIndex(e => e === '--');
const args = process.argv.slice(2, doubleDashIndex === -1 ? process.argv.length : doubleDashIndex);

Creating shared instances (config, reporter)

Yarn walks up the directory tree to find a package.json that defines a workspace, then determines the lockfile location.

Executing add command

Read yarn.lock for existing entries.

Run lifecycle scripts from package.json (preinstall, install, postinstall, etc.).

Fetching Packages

Yarn checks the cache, creates a destination folder, resolves the reference URL, and streams the .tgz file into the cache before extracting it.

Linking Packages

After fetching, Yarn resolves peer dependencies, flattens the tree, sorts destinations, and copies each package from the cache to the final node_modules location.

Q&A

How to increase network concurrency?

Use --network-concurrency <number>.

How to set a total network timeout?

Use --network-timeout <milliseconds>.

Why editing yarn.lock version fields alone does not work?

Yarn also validates the integrity hash; mismatched integrity causes the lockfile to be considered invalid.

How to know which version of a duplicated dependency is actually used?

Yarn’s hoisting algorithm places the first matching version higher in the tree; you can inspect the final layout with yarn list.

References

[1] Yarn official site: https://www.yarnpkg.cn/

[2] Forked Yarn source with Chinese comments: https://github.com/supergaojian/`yarn`

[3] Source‑level analysis of Yarn installation process: https://jishuin.proginn.com/p/763bfbd29d7e