How Docker’s OverlayFS Powers Efficient Container Storage

Using Docker is simple—you just need to build, run, inspect, pull and push containers and images—but understanding how Docker achieves this requires knowledge of its underlying union file system.

What is a Union File System

A union mount creates a merged view of multiple directories without altering the original sources, allowing files from different locations to appear as a single directory. Various implementations exist, each with different performance and maturity characteristics.

UnionFS – the original project, no longer actively developed.

aufs – a reimplementation of UnionFS with added features, formerly Docker’s default driver before being replaced by OverlayFS.

OverlayFS – included in the Linux kernel since 3.18 and used by Docker’s default overlay2 driver.

ZFS – provides features like checksums, snapshots, compression and deduplication, but is not part of the mainline kernel.

Btrfs – a GPL‑licensed, kernel‑integrated file system offering block‑level operations, defragmentation and writable snapshots.

Unless you have a specific need, you should stick with the default overlay2 driver, which the article uses for the remaining demonstrations.

Why Docker Uses a Union File System

Container images can be large; copying an entire image for each container would be costly. Union file systems let Docker add a thin writable layer on top of read‑only image layers, sharing the bulk of data across containers, reducing storage use and startup time.

The read‑only image layers are accessed via copy‑on‑write (CoW): when a container modifies a file, the file is copied to the writable upper layer, leaving the original unchanged.

How Does It Work?

In a typical union mount, directories called "branches" are assigned priorities. When files with the same name exist in multiple branches, the one from the highest‑priority branch is shown.

.
├── upper
│   ├── code.py  # Content: `print("Hello Overlay!")`
│   └── script.py
└── lower
    ├── code.py  # Content: `print("This is some code...")`
    └── config.yaml

Mounting these branches with OverlayFS merges them into a single view where the upper layer takes precedence:

~ $ mount -t overlay \
    -o lowerdir=./lower,\
       upperdir=./upper,\
       workdir=./workdir \
    overlay /mnt/merged

~ $ ls /mnt/merged
code.py  config.yaml  script.py

~ $ cat /mnt/merged/code.py
print("Hello Overlay!")

When a file in the merged view is modified, the copy‑on‑write mechanism copies it to the writable upper layer, where changes are made without affecting the lower read‑only layer.

Docker applies the same principle: image layers form the lower (read‑only) directories, while a container adds an upper writable layer on top.

Try It

First clean the workspace and pull an image:

~ $ docker image prune -af
...
Total reclaimed space: ...MB
~ $ docker pull nginx
Using default tag: latest
latest: Pulling from library/nginx
... (output truncated) ...

Inspect the image to see its layer directories:

~ $ docker inspect nginx | jq .[0].GraphDriver.Data
{
  "LowerDir": "/var/lib/docker/overlay2/.../diff:/var/lib/docker/overlay2/.../diff",
  "MergedDir": "/var/lib/docker/overlay2/.../merged",
  "UpperDir": "/var/lib/docker/overlay2/.../diff",
  "WorkDir": "/var/lib/docker/overlay2/.../work"
}

Run a container and inspect its layers, which include the image’s lower directories plus a new writable upper directory:

~ $ docker run -d --name container nginx
~ $ docker inspect container | jq .[0].GraphDriver.Data
{
  "LowerDir": "...",
  "MergedDir": ".../merged",
  "UpperDir": ".../diff",
  "WorkDir": ".../work"
}

Manually recreate the merged view using the same directories:

~ $ mount -t overlay -o \
lowerdir=.../diff:.../diff,\
upperdir=.../diff,\
workdir=.../work overlay /mnt/merged

~ $ ls /mnt/merged
bin  dev  etc  ...
~ $ umount overlay

The Docker daemon builds the same mount command internally, substituting the appropriate lower, upper and work directories before invoking unix.Mount.

Summary

While Docker’s high‑level interface may seem like a black box, understanding its union file system reveals how layers are efficiently combined, how copy‑on‑write saves space and speeds up container startup, and how this knowledge can guide performance tuning and security decisions. Other Docker internals such as cgroups and namespaces are also worth exploring.

Source: Cloud Native Technology Enthusiasts Community