How PouchContainer Enables In‑Place Container Upgrades for Stateful Services

Background

PouchContainer is an open‑source lightweight container engine that provides strong isolation, high portability, and low resource consumption. It is used for stateful services that require in‑place upgrades while preserving network configuration, storage volumes, and other container settings.

Container Storage Architecture

PouchContainer is built on Containerd v1.0.3. Instead of Docker’s GraphDriver/Layer model, it uses the Containerd concepts of Snapshotter and Snapshot . A snapshot represents a read‑only image layer, while a writable snapshot stores the container’s mutable layer. All metadata is kept in BoltDB for fast restarts.

Upgrade Feature Requirements

Data consistency – network settings, all volumes, and selected configuration fields (Env, Labels, etc.) must remain unchanged after upgrade.

Flexibility – allow changes to CPU, memory, Entrypoint, and addition of new volumes.

Robustness – support automatic rollback on failure.

Upgrade API Definition

The upgrade request is expressed by the ContainerUpgradeConfig struct, which embeds ContainerConfig and optionally a HostConfig. This struct is sent to the POST /containers/upgrade endpoint.

type ContainerUpgradeConfig struct {
    ContainerConfig
    // host config
    HostConfig *HostConfig `json:"HostConfig,omitempty"`
}

Upgrade Process

Backup the existing container metadata for potential rollback.

Merge the new configuration parameters into the old container specification.

Entrypoint handling: use the new Entrypoint if provided; otherwise inherit the old container’s Entrypoint (if it was set via config); if neither is available, fall back to the image’s default Entrypoint.

If the container is running, stop it, then create a new writable snapshot based on the new image.

When the new snapshot is ready, restart the container if it was originally running.

Delete the old snapshots and persist the final configuration.

Rollback Mechanism

If any step fails (e.g., snapshot creation or container start), the system restores the backed‑up metadata and attempts to recreate the original container. Errors are logged for diagnostics.

defer func() {
    if !needRollback {
        return
    }
    // rollback to old container.
    c.meta = &backupContainerMeta
    if err := mgr.createContainerdContainer(ctx, c); err != nil {
        logrus.Errorf("failed to rollback upgrade action: %s", err.Error())
        if err := mgr.markStoppedAndRelease(c, nil); err != nil {
            logrus.Errorf("failed to mark container %s stop status: %s", c.ID(), err.Error())
        }
    }
}()

Demo

Create a container from the ubuntu:14.04 image:

$ pouch run --name test -d -t registry.hub.docker.com/library/ubuntu:14.04 top
43b75002b9a20264907441e0fe7d66030fb9acedaa9aa0fef839ccab1f9b7a8f

$ pouch ps
Name   ID       Status          Created          Image                                         Runtime
test   43b750   Up 3 seconds   3 seconds ago    registry.hub.docker.com/library/ubuntu:14.04   runc

Upgrade the container to busybox:latest while preserving all other settings:

$ pouch upgrade --name test registry.hub.docker.com/library/busybox:latest top
test

$ pouch ps
Name   ID       Status          Created          Image                                         Runtime
test   43b750   Up 3 seconds   34 seconds ago   registry.hub.docker.com/library/busybox:latest   runc

The upgrade API swaps the image without altering network, volume, or other configurations.

Conclusion

In large‑scale production environments, in‑place container upgrades are as frequent as scaling operations, yet neither Moby nor upstream Containerd provide a dedicated upgrade API. PouchContainer’s upgrade feature fills this gap, enabling stateful service updates with data consistency, flexibility, and automatic rollback.

GitHub repository: https://github.com/alibaba/pouch