How to Build a Minimal Linux Container from Scratch: A Hands‑On Guide

Introduction

Containers have become mainstream in backend development. Although Docker is popular, container technology is essentially a thin wrapper around several Linux kernel APIs.

Container vs Virtual Machine

Virtual machines emulate hardware (CPU, memory, disk, NIC) so the guest OS believes it runs on real hardware. Containers share the host kernel and only isolate processes, file systems, network, etc., effectively “soft‑locking” a process inside a confined environment.

Key differences: VMs simulate hardware; containers manipulate kernel namespaces to give the illusion of a separate OS.

Building a Minimal Container (Filesystem)

First create a minimal root filesystem, either with busybox or debootstrap. Example using busybox:

apt-get install busybox-static
mkdir rootfs; cd rootfs
mkdir dev etc lib usr var proc tmp home root mnt sys
/bin/busybox --install -s bin

Or using debootstrap for a Debian wheezy rootfs:

apt-get install qemu-user-static debootstrap binfmt-support
mkdir rootfs
debootstrap --foreign wheezy rootfs
cp /usr/bin/qemu-arm-static rootfs/usr/bin/

After creation, the rootfs can be inspected:

Network Setup

Three typical network modes for containers:

No network – isolated filesystem only.

Host network – container shares the host’s network stack.

Bridge mode – Docker’s default, using a virtual bridge (docker0) and virtual NICs.

Bridge mode is created with brctl and ifconfig to connect container veth pairs to the bridge.

Process Isolation (Namespaces)

Linux provides several namespace types: UTS, PID, mount, network, user, IPC. They can be combined in a single clone system call:

clone(process_func, stack, CLONE_NEWUTS|CLONE_NEWPID|CLONE_NEWNS|CLONE_NEWNET|CLONE_NEWUSER|CLONE_NEWIPC, NULL)

In Go, the exec.Cmd struct with SysProcAttr.Cloneflags is used:

cmd := exec.Command("./container", args...)
cmd.SysProcAttr = &syscall.SysProcAttr{
    Cloneflags: syscall.CLONE_NEWUTS | syscall.CLONE_NEWPID |
                syscall.CLONE_NEWNS | syscall.CLONE_NEWNET,
}
cmd.Run()

Mounting Filesystems

The container’s “god process” mounts the rootfs and essential pseudo‑filesystems before pivoting root:

syscall.Mount(rootfs, tmpMountPoint, "", syscall.MS_BIND, "")
syscall.Mount(procpath, tmpMountPointProc, "proc", 0, "")
syscall.Mount(syspath, tmpMountPointSys, "sysfs", 0, "")
syscall.Mount("udev", tmpMountPointDev, "devtmpfs", 0, "")
syscall.PivotRoot(tmpMountPoint, pivotDir)

Resource Limits with cgroups

cgroups (control groups) limit CPU, memory, and other resources. For example, to restrict a container to two CPU cores on an 8‑core host, write the appropriate values to /sys/fs/cgroup files before launching the container.

Example fork bomb (do not run):

int main(){
    while(fork());
}

Putting It All Together

A simple container consists of two Go programs:

wocker.go – an HTTP server that receives parameters (rootfs path, hostname, target process) and starts the container via exec.Cmd.

startContainer.go – the “god process” that mounts filesystems, sets hostname, and finally runs the user’s init script.

The init script might look like:

#!/bin/bash
service ssh start
/bin/bash

After mounting the rootfs, the script is executed with:

cmd := exec.Command("/root/start_container.sh")
cmd.Run()

Running the container and connecting via SSH (port 23322) demonstrates that the process sees the isolated environment while the host remains unaffected.

Thoughts on Docker

Docker builds on the same primitives (namespaces, cgroups, AUFS, etc.) and adds a rich ecosystem for image distribution and orchestration. Compared with VMs, containers start faster, use resources more efficiently, but isolation is weaker; cgroups enforce only upper limits, not strict guarantees.

References

Rootfs creation guide: https://olimex.wordpress.com/2014/07/21/how-to-create-bare-minimum-debian-wheezy-rootfs-from-scratch/

cgroups article: http://tech.meituan.com/cgroups.html

Linux namespaces manual: http://man7.org/linux/man-pages/man7/namespaces.7.html