Understanding Container Image Storage: Configuring storage.conf and Drivers

For container users, how images are stored on disk is often a mystery. This article explores container image storage mechanisms and tools like Podman that can directly handle these images.

Container Image Storage Evolution

Early Docker architecture hid image storage details behind the daemon, making it hard to access images directly. The Red Hat container runtime team created atomic mount and atomic scan commands to mount and scan images without daemon interference.

New container engines aim to build storage systems that operate without a controlling daemon, allowing multiple tools to share storage concurrently.

File‑system locking is used to control access. The first step is separating the container storage from Docker's graphdriver, which implements various copy‑on‑write (COW) drivers such as overlay, devicemapper, btrfs, xfs, vfs, and aufs.

Note that container storage is unrelated to the Container Storage Interface (CSI); CSI manages volumes, while container storage handles read‑only image layers.

Storage configuration resides in /etc/containers/storage.conf for root‑run engines, or $HOME/.config/containers/storage.conf for rootless tools like Podman.

Driver

# This file is the configuration file for all tools that use the containers/storage library.
# See man 5 containers-storage.conf for more information
[storage]
# Default Storage Driver
driver = "overlay"

The default driver is overlay. In Docker, both overlay and overlay2 exist; most users now use overlay2, which the configuration aliases to overlay if needed.

graphroot

# Temporary storage location
runroot = "/var/run/containers/storage"
# Primary Read/Write location of container storage
graphroot = "/var/lib/containers/storage"

graphroot

defines where actual images are stored. Allocate ample space here, as image collections grow over time. Use standard Linux commands to provision storage, preferably mounting a large device at /var/lib/containers.

storage.options

Each storage driver offers many options; the article highlights a few useful ones.

additionalimagestores

# AdditionalImageStores is used to pass paths to additional Read/Only image stores
# Must be comma separated list.
additionalimagestores = []

This feature lets you add read‑only image stores, such as an NFS share containing many overlay images, which can be shared across all container engines without each node pulling large images.

size

# Size is used to set a maximum size of the container image. Only supported by certain drivers.
size = ""

Setting a size quota helps prevent users from pulling excessively large images, a practice used by platforms like OpenShift Online.

mounting

# Path to a helper program for mounting the filesystem instead of mounting directly.
# mount_program = "/usr/bin/fuse-overlayfs"
# mountopt specifies comma separated list of extra mount options
mountopt = "nodev"

The nodev option prevents devices from appearing inside container images, enhancing security.

Remap‑UIDs/GIDs

# Remap-UIDs/GIDs maps container internal IDs to host IDs.
# remap-uids = 0:1668442479:65536
# remap-gids = 0:1668442479:65536

Remapping allows containers to run with non‑root UIDs/GIDs, improving isolation.

Others

Additional options support drivers like devicemapper and thin‑pool configurations. Refer to /etc/containers/storage.conf and the storage.conf(5) manual for details.

Using Container Storage

Container engines and tools such as Podman, Buildah, CRI‑O, and Skopeo share the same storage. They can see each other's images and coordinate via file locks, but may maintain separate metadata databases.

# podman create -ti --name fedora-ctr fedora sh
# podman mount fedora-ctr
/var/lib/containers/storage/overlay/.../merged
# ls /var/lib/containers/storage/overlay/.../merged
bin  boot  dev  etc  home  lib  lib64  lost+found  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

Tools like Skopeo can preload images into the shared storage without a daemon, using standard filesystem operations.