From VMs to Containers: Achieving Million‑QPS Elasticity with Immutable Images

What a Service Needs to Run

A service requires several stacked layers: the application code and its direct dependencies (e.g., a JAR and third‑party libraries), a runtime (JVM, Node, Python, etc.), system libraries and toolchains (shared objects, certificates, timezone data), and finally the operating‑system kernel. Moving the packaging boundary downward strengthens environment consistency but makes the package heavier.

Physical‑Machine Era

Early deployments ran directly on bare metal. One server hosted a single core application, leading to typical CPU utilization of only 10 %–20 % and provisioning cycles of weeks (hardware purchase, rack‑mount, OS install, network wiring). The main pain points were rigid resource allocation and slow delivery.

Virtual Machines

VMs introduced a hypervisor layer that virtualizes CPU, memory, disk, and network. Each VM carries a full guest OS, consuming hundreds of megabytes to several gigabytes of RAM and storage, and boot times range from tens of seconds to minutes. While VMs provide strong isolation, fine‑grained resource partitioning, and rapid provisioning via cloud providers, they do not solve the “application‑environment consistency” problem: the same “works on my machine” issue appears at the VM level because the runtime, libraries, and configuration still need to be installed inside each VM.

Containers

Containers share the host kernel and isolate processes using Linux namespaces (PID, network, mount, etc.) and cgroups for resource quotas. Because they do not bundle a separate kernel, container images are typically tens to a few hundred megabytes, and start‑up latency is measured in milliseconds to a few seconds, enabling rapid scaling.

Namespace and Cgroup

Namespace determines what a container can see. For example, a PID namespace makes processes start at PID 1 and hides host processes. Network, mount, UTS, IPC, and user namespaces provide independent views of networking, filesystem, hostname, inter‑process communication, and user IDs. Cgroup enforces limits on CPU, memory, and I/O; a container can be restricted to 2 CPU cores, 4 GB memory, and throttled I/O. Together they make a regular process appear as if it has its own machine.

Immutable Images

An image packages application code, runtime, system libraries, and configuration into a read‑only layered artifact identified by a content hash. Because the same image runs on every host, “works on my machine” disappears. Layered images enable cache reuse: only the top layer changes when code changes, reducing build, transfer, and storage costs. Immutable images also make deployments replace‑instead‑modify: upgrading means launching new containers from a new image, rolling back means switching back to the previous tag.

Benefits at Million‑QPS Scale

Elasticity speed: containers launch in seconds, allowing thousands of instances to be added within minutes, whereas VM boot times (tens of seconds to minutes) are too slow for second‑scale scaling.

Density and cost: removing the duplicated kernel raises host utilization from ~10 % (physical‑machine era) to 40 %–50 %.

Standardized delivery: the same immutable image is used by developers and operators, providing a uniform unit for orchestration tools such as Kubernetes.

Pitfalls

Isolation trade‑off: shared kernel gives weaker isolation than VMs; security‑critical workloads may need lightweight VMs (Kata, gVisor) or revert to VMs.

Stateful services: containers assume discardability; databases or stateful middleware require externalized state and graceful shutdown handling.

Resource limits: forgetting to set cgroup limits lets a container exhaust host resources; runtimes such as the JVM must be configured to respect cgroup memory limits.

Observability: short‑lived containers require external logging, monitoring, and tracing pipelines because host‑local logs disappear on container termination.

Conclusion

The evolution from physical machines to VMs to containers reflects a consistent drive to make the deployment unit lighter, faster, and more standardized. Each step trades increased density and speed for reduced isolation, while immutable images lock down environment consistency. Together they enable millisecond‑scale elasticity, high‑density deployment, and automated platform‑level orchestration for services handling tens of millions of queries per second.