Spring Boot Containerization: Docker Image Slimming from 800MB to 200MB & K8s Probe Tuning

Two Classic Incidents

Incident 1: Release stuck 30 minutes copying image layers — 800MB fat-jar image transferred fully each time across data centers.

Incident 2: Rolling updates caused intermittent 502s. Old Pod received SIGTERM and exited in 5 seconds, dropping in-flight requests. K8s default terminationGracePeriodSeconds is 30s, but the app died in 5s.

Root cause: default values left unchecked. Base images inherit heavy layers; probes don't match Spring Boot's startup curve. 80% of containerization effort is correcting defaults.

Three Eras of Dockerfile

Era 1: Fat JAR Copy (Source of All Problems)

FROM eclipse-temurin:21
COPY target/app.jar /app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]
# Issues:
# 1. Full JDK (~400MB) includes compiler
# 2. App code in single layer — one line change invalidates whole 100MB+ layer
# 3. JVM params rely on defaults; old JDK miscalculates container memory

Era 2: Multi-Stage + Slim JRE (Standard Answer)

# ---- Stage 1: Build ----
FROM maven:3.9-eclipse-temurin-21 AS build
WORKDIR /build
COPY pom.xml .
RUN mvn dependency:go-offline
# Dependency layer fixed first: code changes no longer re-pull deps
COPY src ./src
RUN mvn package -DskipTests

# ---- Stage 2: Runtime (JRE not JDK) ----
FROM eclipse-temurin:21-jre-alpine
RUN addgroup -S app && adduser -S app -G app
WORKDIR /app
COPY --from=build /build/target/app.jar ./
USER app
ENTRYPOINT ["java", "-jar", "app.jar"]

Three key moves: multi-stage build (build tools excluded from final image), dependency layer first (pom unchanged → dependency layer cache hit, only code layer ~tens of KB transferred), non-root user (security baseline, K8s PSA policies reject root containers).

Era 3: Layered JAR / jlink / CDS (Advanced)

Layered JAR: Spring Boot extracts dependencies/spring-boot-loader/snapshot-dependencies/application into four layers, each becoming a Docker layer — dependency and code cache granularity fully separated.

jlink/jdeps: Trim JRE to only used modules; runtime image can shrink to ~120MB.

CDS/AppCDS: Archive class-loading snapshot at startup; subsequent starts reduce class-loading time by 30–40% — significant for frequently scaled short-lived Pods.

# Build-time extract four layers
RUN java -Djarmode=tools -jar app.jar \
  extract --layers --destination /ext

FROM eclipse-temurin:21-jre-alpine
WORKDIR /app
# Four COPYs: least-changing first
COPY --from=build /ext/dependencies/ ./
COPY --from=build /ext/spring-boot-loader/ ./
COPY --from=build /ext/snapshot-dependencies/ ./
COPY --from=build /ext/application/ ./
# ENTRYPOINT (single line):
# java org.springframework.boot.loader.launch.JarLauncher

Key takeaway: Slimming ledger: fat-jar era ~800MB (base JDK 400 + fat jar 200+); JRE-alpine multi-stage ~300MB; layered + jlink ~120–180MB. Release transfer drops from "change one line → push full image" to "change one line → push only classes layer".

Startup Acceleration: Pod Scaling Can't Wait 2 Minutes

K8s scale-out experience = image pull + startup + readiness probe pass. Last two are Java weak points:

Spring AOT / GraalVM Native Image: Compile to native executable, millisecond startup, half memory — but reflection config cost high, ecosystem compatibility needs verification; suits Serverless & high-frequency scale scenarios.

CDS: Low cost, quick win, one JVM flag.

Lazy initialization: spring.main.lazy-initialization=true speeds startup but slows first request; avoid on critical paths.

# JVM startup optimization combo
-XX:+UseSerialGC
-Xshare:on -XX:SharedArchiveFile=app.jsa
# Build-time generate CDS archive
# Runtime -Xshare:on forces use
# Archive invalid → hard error, avoids "thought it worked but didn't"

# Container CPU awareness (JDK 10+ auto, older manual)
-XX:ActiveProcessorCount=2
# Align with Pod CPU limit; GC/thread pools calculate from this

Core Flow: A Pod's Life in K8s

Scheduling: Based on resources.requests — request is scheduling basis, limit is runtime ceiling.

Image Pull: imagePullPolicy defaults to IfNotPresent (fixed tag); latest becomes Always.

Container Start: ENTRYPOINT runs, Spring Boot initializes.

Probes Take Over: startup → readiness → liveness polled per config.

Traffic Enters: readiness passes → endpoint added to Service.

Exit: Receive SIGTERM → preStop + terminationGracePeriod grace → force kill (ties to graceful shutdown article).

request vs limit pitfall: request too low → scheduled onto overcommitted node, CPU throttled → JVM jitter; limit too high → node overcommit. CPU limit especially dangerous for Java — GC threads throttled prolongs all pauses. Many teams' practice: no CPU limit (or 2–4× request), memory limit mandatory .

Probes: Three Switches Most Often Misconfigured

5.1 Probe Semantics Reiterated

startupProbe: Is app startup complete? Before it passes, liveness/readiness don't run — slow-start Java apps' lifesaver .

readinessProbe: Can accept traffic? Failure removes endpoint, no container restart.

livenessProbe: Still alive? Failure restarts container — only put "process self-state", never DB connections (per graceful shutdown iron law).

5.2 Actuator Probe Endpoint Source-Level Structure

// HealthEndpoint grouping (Boot 2.3+)
management.endpoint.health.probes.enabled=true
management.endpoint.health.group.liveness
  .include=livenessState
management.endpoint.health.group.readiness
  .include=readinessState,db

// livenessState from event enum:
// CORRECT / BROKEN -> process self
// readinessState:
// ACCEPTING_TRAFFIC / REFUSING_TRAFFIC
// -> decouples "accept traffic" from "is healthy"

Source-level key: probe state is essentially two AvailabilityChangeEvent enums — publishing "pause traffic" (REFUSING_TRAFFIC) won't trigger liveness restart. This is the mechanism behind "drain traffic then exit" during rolling updates. DB down → readiness fails → drain traffic; put in liveness → cluster-wide restart storm.

5.3 Production-Validated Probe Config

livenessProbe:
  httpGet: {path: /actuator/health/liveness, port: 8080}
  periodSeconds: 10
  failureThreshold: 3   # 30s to declare dead
startupProbe:
  httpGet: {path: /actuator/health/liveness, port: 8080}
  periodSeconds: 5
  failureThreshold: 60  # wait up to 5 minutes
readinessProbe:
  httpGet: {path: /actuator/health/readiness, port: 8080}
  periodSeconds: 5
  failureThreshold: 2   # drain in 10s

Three numbers rationale: startup failureThreshold × periodSeconds must exceed slowest app startup ; readiness must be fast (drain quicker the better); liveness must be dull (prevent mis-kill, 30s verdict sufficient).

5.4 Complete Rolling Update Orchestration

spec:
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
      # start new first, kill old after drain — zero capacity loss
  template:
    spec:
      containers:
      - name: app
        lifecycle:
          preStop:
            exec:
              command: ["sleep", "5"]
            # wait for iptables routing rules to converge
            # avoid last batch of requests in drain window
        terminationGracePeriodSeconds: 45
        # preStop 5s + graceful shutdown 30s + buffer

This combo plus graceful shutdown trio zeros 502s: maxUnavailable=0 guarantees capacity, preStop sleep covers routing convergence window, grace period covers preStop + graceful timeout .

ConfigMap / Secret & Resource Quotas

Config Injection: ConfigMap for config, Secret for sensitive data (base64 only; real encryption needs Vault/KMS); Spring side uses spring.config.import to mount.

Graceful Reload: Config change triggers restart (ArgoCD/Flux rollout) beats hot reload — simpler, reliable.

Resource Ledger: Memory limit = heap + metaspace + off-heap + thread stacks + buffer (previous article's 70% rule applied here).

resources:
  requests:
    cpu: "1"
    memory: "3Gi"
  limits:
    memory: "4Gi"   # heap 3G + non-heap 1G
    # CPU no limit, prevent GC thread throttling

Release Practice: Before/After Comparison

Transformation on an 8GB memory, 10M DAU service:

Image: 800MB → 210MB (JRE-alpine + layered), publish transfer 8 min → 40 sec.

Startup: 105s → 38s (CDS + dependency layer cache warm), startupProbe from "luck" to precise wait.

Release: Rolling update occasional 502 → zero 502 (maxUnavailable=0 + preStop + graceful).

Scale-out: HPA adds one Pod from "2 min to serve traffic" to 45s ready; flash-sale scaling window no longer panic.

Key takeaway: Suggested order: first probes & shutdown orchestration (stop 502 bleeding), then image slimming (save bandwidth/time), finally startup acceleration (scale-out experience). First two high ROI, low risk.

Eight High-Frequency Pitfalls

Pitfall 1: Full JDK base image. Wastes 300MB; JRE or jlink-trimmed is correct runtime shape.

Pitfall 2: COPY before dependencies. Code change invalidates whole layer cache, deps re-pulled, build 10 min.

Pitfall 3: No startupProbe. Slow-start app killed by liveness mid-start → CrashLoopBackOff.

Pitfall 4: liveness probes DB. DB hiccup = cluster-wide restart storm. DB only in readiness.

Pitfall 5: latest image tag. imagePullPolicy becomes Always, no rollback/version pinning; tag with git SHA.

Pitfall 6: CPU limit too tight. GC/Netty threads throttled, P99 spikes. Loosen or omit CPU limit.

Pitfall 7: Forgot terminationGracePeriodSeconds. Default 30s may be less than preStop + shutdown grace sum; grace gets hard-cut.

Pitfall 8: JVM in container without container support. Old JDK calculates heap from host memory, eats whole node.

Troubleshooting Three-Step Method

Step 1: describe pod check Events — image pull slow / probe fail / OOMKilled obvious.

Step 2: Probe fail → container logs + curl probe endpoint, distinguish "app slow" vs "endpoint actually down".

Step 3: OOMKilled → exit code 137, cross-check memory limit vs JVM heap config (recall GC article 70% rule).

Learning Path & Interview Questions

9.1 Four-Stage Roadmap

Beginner (1 week): Multi-stage Dockerfile, image layer principles, K8s Pod basic lifecycle.

Intermediate (2 weeks): Layered JAR, three probe semantics & params, resources request/limit, rolling update strategy.

Deep (1 month): CDS/AOT, image scanning & signing (security), HPA/VPA, Service Mesh traffic drain.

Practice (ongoing): Full slim + probe overhaul on existing service; establish "release checklist" (non-root / probes / grace / resources four checks).

9.2 Twelve Interview Questions

What problems does multi-stage build solve? How does image layer cache hit?

Spring Boot layered JAR principle & benefits?

jlink JRE trimming principle? How small can it go?

Why is startupProbe Java's lifesaver?

Consequences of liveness vs readiness probing DB?

Full causal chain of rolling update 502? How to zero it?

Difference between request and limit? CPU limit risks?

What does preStop sleep 5 seconds wait for?

OOMKilled vs Java OOM difference? Exit code?

GraalVM Native Image trade-offs? When worth it?

Why does CDS speed startup? What happens on invalidation?

Why must image use non-root user?

9.3 One-Page Cheat Sheet

Image three moves: multi-stage + JRE/jlink + layered jar
Build speed: dependency layer first, code layer last
Security baseline: non-root + fixed tag (git sha)
Probe three rules: startup wait 5 min
readiness fast drain (10s)
liveness dull verdict (30s) process only
DB only in readiness!
Release four pieces: maxUnavailable=0 + preStop 5s
+ graceful 30s + grace 45s
Resource two rules: memory must limit (heap 70%)
CPU loosen prevent throttle
One sentence: containerization engineering,
eight parts correcting defaults

Summary: Containerization upgrades "runs" to "ships well, scales well, recycles well": image three moves govern transfer efficiency, probe three rules govern life/death verdicts, release four pieces govern zero-downtime updates, resource two rules govern fair scheduling. This closes the loop from code to container. Next: distributed consistency deep dive — Spring Boot Distributed Locks: From Redisson Watchdog to RedLock Controversy .