Mapping the Docker Ecosystem: From Early Jails to the Moby Project

Container technologies before Docker

Containerization predates Docker and includes a variety of OS‑level isolation mechanisms:

Chroot jail – the original Unix chroot command (1979) that changes the root filesystem for a process.

FreeBSD jail – early OS‑level virtualization that isolates processes, network stacks and file systems.

Linux VServer – kernel‑based virtualization that creates multiple virtual servers on a single Linux kernel.

Solaris Zones/Containers – resource‑controlled OS containers for x86 and SPARC platforms.

OpenVZ – Linux kernel patch that provides multiple isolated VPS containers.

Process containers (cgroups) – Google’s initial implementation that groups processes for resource accounting.

LXC (Linux Containers) – uses namespaces and cgroups to run several independent Linux systems on one kernel.

Warden – Cloud Foundry’s early container runtime built on LXC.

LMCTFY – Google’s “Let Me Contain That For You” project, later superseded by libcontainer.

Docker – introduced in 2013, popularised portable application containers.

rkt (Rocket) – security‑focused container engine that follows open standards.

Key Docker‑related concepts and timeline

Docker & LXC → libcontainer

Docker’s first releases (up to version 0.9) used LXC as the execution environment. Starting with Docker 0.9, the project switched to its own libcontainer library, which directly interfaces with Linux kernel primitives (cgroups, namespaces, netlink, netfilter).

2015 – runC

Docker extracted the low‑level runtime into the runC binary. runC is a lightweight, OCI‑compliant runtime that can launch containers without the full Docker Engine, making the runtime portable across platforms.

2015 – Open Container Initiative (OCI)

Docker, CoreOS and other industry leaders founded OCI to define a common image format and runtime specification. OCI ensures that a container built with Docker can run on any OCI‑compatible engine.

2016 – containerd

Docker split the container management logic into a separate daemon called containerd. This daemon handles image transfer, storage, snapshotting and the lifecycle of containers, allowing the Docker Engine to focus on higher‑level orchestration.

Docker component diagram

How Docker runs a container (step‑by‑step)

Docker Engine builds a container image (layers + metadata).

The image is handed to containerd. containerd starts a containerd‑shim process for the container.

The shim invokes runC, which uses libcontainer to create the namespaces, cgroups and mount points.

After the container process is started, containerd‑shim exits, leaving the container running independently.

This architecture allows Docker Engine upgrades without stopping existing containers.

Moby project

The Moby project is an open‑source collection of modular components (“Lego‑like” building blocks) for building custom container‑based systems. It provides the foundation for Docker’s production model.

Core Moby components

containerd – a daemon that implements the OCI runtime and image specifications, managing the full lifecycle of containers on Linux and Windows.

LinuxKit – a toolkit for building minimal, immutable, and secure OS images that run containers. It supports Hyper‑V, VMware, AWS, Azure and other hypervisors.

Infrakit – a Go library for declarative, immutable and self‑healing infrastructure. It automates provisioning of distributed systems and can be used by higher‑level orchestrators.

Libnetwork – provides a container network model (CNM) and a consistent API for composable networking, enabling plug‑in drivers for overlay, bridge, MACVLAN, etc.

Docker Swarm

Docker Swarm is Docker’s native orchestration engine, integrated into the Engine since version 1.12. Users create a Swarm cluster with the Docker CLI ( docker swarm init, docker swarm join) and deploy services ( docker service create) that are scheduled across the nodes.

Docker integration with Kubernetes

In response to the rise of Kubernetes, Docker added first‑class support for Kubernetes. Docker Compose files can be converted to Kubernetes manifests, and Docker Desktop includes a built‑in Kubernetes cluster. This enables developers to choose either Swarm or Kubernetes for container orchestration.

Growth of Docker Hub image pulls (2013‑2017)

All images are sourced from the public Internet.