Rethinking the Design Philosophy Behind Modern Frontend Bundlers

Build system concepts

A build system automates repeatable tasks. The paper Build Systems à la Carte: Theory and Practice abstracts common concepts:

Task : unit of work defined by a descriptor (e.g., Makefile, Excel formula).

Input : data consumed by a task.

Output : result produced by a task, possibly feeding other tasks.

Info : cross‑build metadata used in the next build (e.g., file modification times, bundler cache).

Store : location where inputs, outputs and info are persisted (e.g., the file system).

Build : given a set of tasks and an existing store, add new inputs and produce a new store.

Two orthogonal strategies differentiate build systems:

Whether a task is re‑executed ( Rebuilder ).

The order in which tasks are executed ( Scheduler ).

Scheduler variants

Topological : execute tasks in a static topological order derived from declared dependencies.

Restarting : pick a task; if its dependencies are not satisfied, pick another task, repeating until all are done.

Suspending : pick a task; if a dependency is missing, execute the dependency first, then resume the original task (easily expressed with async/await).

Rebuilder variants

Dirty‑bit : each task records a clean/dirty flag; after a build all are clean, and changed inputs mark the corresponding tasks dirty.

Verifying traces : store hashes or timestamps; on the next build compare them to decide whether a task can be reused.

Constructive traces : like verifying traces but also store the actual content, enabling cloud caching and remote execution.

Key properties of build systems

Dynamic dependencies : dependencies may be static or computed at build time.

Minimality : execute the smallest possible set of tasks.

Early cutoff : stop downstream tasks when an upstream task’s output does not change.

Concrete build‑system examples

make = topological modTimeRebuilder

Make uses static dependencies from a Makefile and a topological scheduler. It treats file modification time as Info and implements a dirty‑bit rebuilder. Minimality is achieved, but early cutoff is absent because a rebuilt file’s timestamp always changes. excel = restarting dirtyBitRebuilder Excel describes tasks via cell formulas, mixing static and dynamic dependencies, thus using a restarting scheduler. It records dirty cells and re‑executes dependent tasks. Static dependencies enjoy minimality; dynamic ones do not. bazel = restarting ctRebuilder Bazel also uses a restarting scheduler and a constructive‑trace rebuilder, providing cloud caching and remote execution. shake = suspending vtRebuilder Shake tracks dependencies at execution time (verifying traces). It achieves both minimality and early cutoff, and supports dynamic dependencies because dependencies are discovered dynamically. cloudShake = suspending ctRebuilder Cloud Shake extends Shake with constructive traces, adding cloud caching. buck2 = suspending ctRebuilder Buck2, co‑authored by a Shake author, mirrors Cloud Shake, supports dynamic dependencies, minimality, early cutoff, cloud caching, and remote execution via the DICE incremental computation engine.

Bundlers as build systems

A bundler can be seen as a build system plus a partial task descriptor. Early task runners such as gulp and grunt are essentially build systems. Modern bundlers differ in two ways:

Dependencies are highly dynamic—module generation, optimization, and plugins can alter task logic.

Cycles are common—bundlers must handle circular module dependencies, which many classic build systems cannot.

Bundler builds fall into two categories: watch‑mode rebuilds that keep the compiler alive, and one‑off builds that terminate the compiler.

Pass‑based bundlers

passBasedBundler = foreach ctRebuilder

Webpack, Parcel, Rollup and esbuild execute a series of passes. Different passes use different scheduler/rebuilder combos:

SideEffectsFlagPlugin uses a suspending scheduler (needs parent module information) with an always‑true rebuilder.

FlagProvidedExportsPlugin uses a restarting scheduler and a verifying‑trace rebuilder (vtRebuilder) to recompute exports when re‑exports change.

Most other passes use a simple foreach scheduler with a constructive‑trace rebuilder (ctRebuilder) and persistent caching.

Because passes are isolated, early cutoff cannot be achieved across passes, leading to extra cache‑verification work and slower builds.

Turbopack

turbopack = suspending ctRebuilder

Turbopack adopts a query‑based model: tasks are defined and retrieved on demand. In development it runs only the tasks needed for the queried result, avoiding unrelated work. Its incremental engine “turbo tasks” implements the scheduler and rebuilder concepts, achieving both minimality and early cutoff.

Vite

vite = suspending vtRebuilder

Vite does not bundle in production; in development it compiles individual modules on demand using the browser’s native ESM loader. Concurrency is limited by browser request limits, and caches cannot be shared across sessions, preventing an upgrade from vtRebuilder to ctRebuilder.

Rspack

incrementalRspack = foreach dirtyBitAndCtRebuilder

Rspack is a pass‑based bundler that introduces “affected‑based incremental” computation. Each stage collects changes; subsequent stages compute the set of affected tasks and re‑execute only those, adding early cutoff and improving minimality. This approach aligns with self‑adjusting computation:

The fundamental idea is to track the control and data dependencies in a computation in such a way that changes to data can be propagated through the computation by identifying the affected pieces that depend on the changes and re‑doing the affected pieces. — Self‑Adjusting Computation

Summary of useful build‑system features for bundlers

Minimality – reduces rebuild work.

Early cutoff – enhances minimality by stopping downstream work when upstream output is unchanged.

Parallelism – explicit dependencies enable concurrent execution (suspending scheduler often uses async/await workers).

Remote cache – cloud‑based storage of inputs/outputs.

Remote execution – distributed task execution for higher throughput.

References

[1] Build Systems à la Carte: Theory and Practice – https://www.microsoft.com/en-us/research/uploads/prod/2020/04/build-systems-jfp.pdf

[2] Make – https://www.gnu.org/software/make/

[3] Shake – https://shakebuild.com/

[4] Bazel – https://bazel.build/

[5] Excel – https://en.wikipedia.org/wiki/Microsoft_Excel

[6] Build‑system composition – https://github.com/snowleopard/build/blob/43b18b9a362d7d27b64679ea4122e4b8c5dfedd9/src/Build/System.hs

[7] Shake author – https://ndmitchell.com/

[8] Buck2 vs Cloud Shake discussion – https://www.reddit.com/r/rust/comments/136qs44/comment/jipq5pj/

[9] Remote execution – https://buck2.build/docs/users/remote_execution/

[10] DICE documentation – https://github.com/facebook/buck2/blob/3fd1dbec7a212291c222b803a43a1355b48c3fb9/dice/dice/docs/index.md

[11] gulp – https://gulpjs.com/

[12] grunt – https://gruntjs.com/

[13] Buck2 on DICE – https://ndmitchell.com/downloads/paper-implementing_applicative_build_systems_monadically-01_jan_2022.pdf

[14] Affected‑based incremental (Rspack) – https://github.com/web-infra-dev/rspack/discussions/8243

[15] Self‑Adjusting Computation quote – https://arc.net/l/quote/yjoxhuft