JVM Production Mastery: Parameter Skeleton, Launch Checklist, Incident Cheat Sheet & Promo Plan
This final article in the JVM series assembles a production-ready parameter skeleton for JDK17 G1, a 10-item automated launch check script, a 10-category incident cheat sheet, and a big-promotion contingency timeline, all grounded in source-code analysis of parameter parsing, container awareness, OOM handling, and configuration drift detection.
Introduction: Assembling the Production Machine
The previous nine articles dissected every JVM component: memory layout, GC, object allocation, class loading, code optimization, and diagnostics. This concluding episode assembles those pieces into a machine ready for the production line: a full parameter skeleton, an automated launch check script, a 10-category incident cheat sheet, a big-promotion contingency timeline, plus a complete knowledge map of the series.
Three opening accidents illustrate the stakes:
Accident 1: Container without a budget. A 4 GB container with no MaxRAMPercentage gave the JVM only 1 GB heap; GC ran every second, CPU at 90%. An 8 GB bare-metal machine defaulted to 2 GB heap unchecked. The lesson: heap is a budget; services without a budget will inevitably hit performance issues.
Accident 2: Big promotion without a plan. A sporadic Full GC at peak caused chaos — some demanded heap dumps, others wanted to tweak IHOP, others restarted. The dump during peak froze the instance, P99 spiked. Postmortem: every action needed at peak must be frozen into a plan at T-1; decisions made on the spot are almost always wrong.
Accident 3: Recurring "new" problems. A team rediscovered Metaspace leaks quarterly and class conflicts semi-annually, each time spending a full day. Root cause: no postmortem-to-cheatsheet mechanism. One accident's value is turning it into a cheatsheet entry so the next occurrence takes 10 minutes.
1. Production Parameter Skeleton: Five Zones
1.1 Parameter Map: Each Zone Owns One Account
The skeleton covers "safety and correctness", not "ultimate performance". It guarantees no low-level mistakes in container, GC, and diagnostics; squeezing P99 from 80 ms to 60 ms relies on code-level allocation profiling and hotspot optimization (articles 06/08), not more parameters. The skeleton is the "passing-grade equipment"; performance tuning is the "bonus" — order cannot be reversed.
For JDK 17 + G1 default line, the skeleton divides parameters into five zones:
Heap Zone — Set the budget
GC Zone — Set the strategy
Metaspace Zone — Set the ceiling
Container Zone — Set the eyes
Diagnostics Zone — Set the insurance
Zone-based troubleshooting: heap issues → budget zone, GC behavior → strategy zone, class issues → metaspace zone, environment sensing → container zone, forensics → diagnostics zone. Configuration reviews follow the same zones: each zone either has a value with a rationale, or it doesn't. Most teams' problem is "a pile of parameters but nobody knows which does what"; zoning eliminates that.
Zoning also cures "copying someone else's parameters". Big-tech blog GC params were tuned for their heap size, traffic shape, container spec — copying them directly usually fails, especially MaxGCPauseMillis and IHOP which depend on traffic shape at pixel level. Correct posture: copy zone conclusions , not specific values — e.g., copy "diagnostics zone five segments fully enabled", not "MaxGCPauseMillis=100". Numeric parameters must always be validated against your own GC logs; others' numbers are just their experimental conclusions.
1.2 Full Skeleton Configuration
The skeleton is not invented; it directly distills the conclusions of the previous nine articles: heap and metaspace caps from article 02's memory ledger, G1 three parameters from article 04's tuning conclusions, container trio from article 01's container derivation chain, diagnostics five segments from article 09's always-on config. Every line has a source — that's why the skeleton claims to "handle 90% of scenarios".
A complete, copy-paste-ready skeleton (JDK 17, G1 default, 8 GB service instance example):
# ==== JVM Production Skeleton (JDK17 + G1) ====
# ---- Heap Zone: Budget ----
-Xms8g -Xmx8g
-XX:MaxMetaspaceSize=512m
-XX:CompressedClassSpaceSize=256m
-XX:MaxDirectMemorySize=1g
# ---- GC Zone: Strategy ----
-XX:+UseG1GC
-XX:MaxGCPauseMillis=200
-XX:InitiatingHeapOccupancyPercent=45
-XX:G1HeapRegionSize=2m
# ---- Container Zone: Eyes ----
-XX:+UseContainerSupport
-XX:MaxRAMPercentage=70.0
-XX:ActiveProcessorCount=4
# ---- Diagnostics Zone: Insurance ----
-Xlog:gc*,safepoint:file=/logs/gc.log
:time,uptime,level,tags
:filecount=5,filesize=50m
-XX:StartFlightRecording=maxsize=250m
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/logs/heapdump.hprof
-XX:+ExitOnOutOfMemoryError
-Xlog:class+unload=info
:file=/logs/class-unload.logZone-by-zone rationale:
Heap Zone — Xms=Xmx eliminates dynamic heap resize pauses and jitter (article 04); metaspace and direct memory caps make any leak "die fast, die early, leave evidence" (article 02).
GC Zone — G1 default line stays conservative: 200 ms pause target is an empirical sweet spot; smaller targets fragment collection and increase frequency (article 04 pause-target backfire). IHOP=45 works with GC log observation of concurrent mark timing for fine-tuning.
Container Zone — MaxRAMPercentage=70 reserves 30% for off-heap and OS; ActiveProcessorCount explicitly pins core count in quota/shared-pool environments to prevent GC/JIT thread miscalculation.
Diagnostics Zone — Direct landing of article 09's five-segment always-on config; observability installed before incidents.
The skeleton deliberately stays "minimal and precise": ~20 explicit parameters cover 95% of production scenarios; the rest are left to ergonomic derivation. Parameter stuffing is an anti-pattern — every extra parameter is future upgrade compatibility debt and adds to "nobody can explain why this is set". To use a parameter, you must first articulate its rationale. These four anti-patterns plus the five zones form the series' parameter knowledge delivery: one skeleton, one set of rationales, one process.
Three "anti-concepts" to remember:
"More parameters = more professional" — opposite is true: fewer explicit parameters, each explainable, is more professional.
"New version parameters can be pre-configured" — don't take medicine you don't need; add new params only when logs prove necessity.
"Tuning = adjusting parameters" — the series repeatedly proves most performance root causes are in code (allocation profiling, lock granularity, cache design); parameters are only the last-mile micro-adjustment. Fix code first, then parameters; read logs first, then parameters — these two sentences are the parameter philosophy's guiding principle.
1.3 Container Special: JVM's Three-Piece Eye Set
Containers are the default deployment form and the hotspot for parameter chaos — Accident 1's default heap issue, OOMKilled, GC thread count distortion all happen here, warranting a dedicated section:
# ==== Container Special: JVM's Three-Piece Eye Set ====
# 1. Memory Awareness (JDK10+ default on)
-XX:+UseContainerSupport
# MaxRAMPercentage relative to container limit
-XX:MaxRAMPercentage=70.0
# 2. CPU Awareness: default reads cpu quota
# quota jitter or shared pool → explicitly pin:
-XX:ActiveProcessorCount=4
# 3. requests = limits (most critical)
# inconsistency makes cgroup limit vs scheduler
# actual resource mismatch, derivation fails entirelyThree frequent crash points:
requests ≠ limits — in oversold elastic environments, cgroup limit and actual resource mismatch; heap derived from large limit gets squeezed into small resources, ending in OOMKilled.
Quota core-count distortion — 0.5 core quota rounded to 1 core by JVM, GC/JIT threads double and contend for CPU (echoes article 01 container params).
Misinterpreting 30% headroom as waste — that 30% is off-heap (metaspace, thread stacks, direct memory, native) lifeline; heap eating 100% of container dies by kernel OOM Killer without warning. This verification routine runs in five minutes after every spec change and catches most container-side JVM ghosts.
Add a habit: after container spec changes (memory expansion, core addition), add "JVM eye re-verification" to the change checklist — eyes read cgroup; if cgroup changes, eyes must re-verify. Doing this real eliminates half of container-side parameter issues upfront.
Four additional verification steps after every spec change:
Check limit vs heap relation (
PrintFlagsFinalMaxHeapSizematches 70% expectation).
Check core derivation ( Runtime.availableProcessors or ActiveProcessorCount actual value; GC thread count derives from it).
Check OOMKilled records (container exit code 137 = kernel OOM Killer; first reaction check off-heap, not heap).
Check cgroup version (v1/v2 file paths and semantics differ; old kernel + old image combos are perception-failure hotspots).
Five minutes for four checks catches most container-side JVM mysteries.
When to touch each zone's parameters:
Heap Zone — only on capacity change (big-promo sustained traffic step-up) or container spec change.
GC Zone — only after GC logs prove a specific behavioral issue (e.g., IHOP triggers too late causing to-space exhausted).
Metaspace Zone — only increase, never lightly decrease.
Container Zone — follows deployment shape; re-review on every environment change.
Diagnostics Zone — only add, never remove — observability items have no "over-configured" problem, only "not configured" problem.
Each zone's change frequency differs; review cadence should too: GC zone half-yearly is enough; container zone every deployment change.
1.4 Parameter Change Process: Review, Canary, Rollback
After skeleton is set, daily work is "changing parameters" — a high-frequency source of JVM production incidents, must be proceduralized:
# ==== Parameter Change SOP (One Variable at a Time) ====
# 1. Review: write rationale + expected effect + rollback condition
# Rationale template: current_state(log_evidence) -> change
# -> expected(which_metric_improves) -> risk
# 2. Canary: 1 instance first, observe 1 GC cycle
# Metrics: pause/frequency/throughput/FGC four views
# 3. Sampling comparison: before/after each keep
# PrintFlagsFinal and GC log samples
# 4. Full rollout: batches 20% -> 50% -> 100%
# 5. Rollback: restore original value = rollback, no
# code rollback needed — parameter changes are naturally
# reversible, their biggest advantage over code optimizationThe most skipped step in SOP is "pre-change sample" : many change params only to realize no baseline exists for comparison; "did it improve?" becomes pure feeling. Before changing, keep a GC log sample and PrintFlagsFinal output — cost ten seconds, gain an evidence-backed experiment record.
Parameter changes are naturally reversible (restore original value = rollback) — this is their biggest advantage over code optimization. Use this property well and JVM tuning can move in small, fast steps.
Change metrics must come in pairs: every change logs a triplet "change, expectation, actual" — expected pause reduction 20%, actual reduction how much, explain gap if large. After several rounds, this logbook becomes the team's own "parameter experience base": which params work under your traffic shape, which don't, why they don't — all inside. Next year's tuning efficiency depends on whether this year you kept this ledger seriously.
2. Launch Check & Incident Cheatsheet
If the parameter skeleton is "weapons loaded", this section is "test-fire" and "target paper": check answers "is the gun usable", cheatsheet answers "where to aim in battle". Both are low-cost, high-leverage — one check ten minutes, one cheatsheet lookup few minutes, exchange for no panic at critical moments.
2.1 Ten-Point Check: All Green Before Traffic
First eight items are config verification; items 9-10 are behavioral verification — scripts installed but not rehearsed, parameters written but not validated, are "paper checks" . Automate checks into a script; launch review shifts from "verbal confirmation" to "run once, get result":
#!/bin/bash
# jvm-doctor.sh: automated launch check
PID=$(jps -l | grep app.jar
| awk '{print $1}')
pass=0; fail=0
ck(){ if eval "$2"; then
echo "PASS $1"; pass=$((pass+1))
else
echo "FAIL $1"; fail=$((fail+1))
fi }
ck "Heap cap" "jcmd $PID VM.flags
| grep -q MaxHeapSize"
ck "JFR" "jcmd $PID JFR.check
| grep -q running"
ck "GC log" "ls /logs/gc.log*
>/dev/null"
ck "Dump headroom" "test \$(df
--output=avail /logs | tail -1)
-gt 8000000"
echo "RESULT pass=$pass fail=$fail"
# fail != 0: no trafficHalf the script's value is checking, half is forcing result retention : paste pass/fail into launch ticket, review has evidence, retrospective "did we actually check then?" stops being a blame game. Script customization cost is tiny — each check is a jcmd or file test, just append to ck.
Beyond launch check, routine patrol is its sibling, different frequency and content: weekly GC log weekly report (pause distribution, frequency trend, safepoint sync time), monthly histogram and Metaspace curve for trend reconciliation, quarterly diagnostic drill (article 09's quarterly routine). Launch check answers "healthy at launch"; patrol answers "health silently eroding" — Metaspace creeping +5% weekly, P99 pause slowly climbing — only patrol trend graphs expose these chronic diseases; acute launch check can't catch them.
Next step: embed check into release pipeline: add "jvm-doctor runs on new instance, fail blocks release" to pre-release automated checks — check moves from human discipline to pipeline gate, miss rate drops to zero. Add PrintFlagsFinal diff to pipeline: parameter baseline lives in repo, each release auto-diffs new instance actual values, drift triggers alert. Discipline's endpoint is automation — turn "humans must remember" into "system blocks", never bet on human memory again.
2.2 Ten-Category Incident Cheatsheet: First Action Always First
The cheatsheet condenses the previous nine articles' ten main lines (each category's full methodology in its article; cheatsheet keeps only "first action" index): heap OOM and Metaspace growth back to articles 02/06 leak investigation; CPU 100% and hang use article 09's four-step method plus multi-snapshot comparison; frequent Full GC and promotion storm are articles 03/04's generational ledger; safepoint leak in article 09 source segment; class conflict article 07's three axes; direct memory OOM via NMT diff (article 02); slow startup via AppCDS (article 07). The cheatsheet's first column is always "first action" — stop-gap before cure, evidence before action; these two sequence disciplines are worth more than ten tricks.
Correct cheatsheet usage: "match seat then expand immediately":
Step 1: match symptom column to row (don't jump to conclusions; wrong visual match wastes everything after).
Step 2: execute first-check column commands (seconds to minutes, grab first evidence).
Step 3: expand along root-cause direction column into corresponding article's full methodology .
Three actions align to one principle: clarify "what it is" first, then decide "how to treat". Counterexample is Accident 2's scene — symptoms not clear yet everyone rushing to act, all "treating", none "checking" first.
Further upstream: grade incidents:
P0 (platform impacted) — stop-gap only, minimal forensics, postmortem later.
P1 (single instance anomaly) — drain traffic + full forensics; most rehearsed scenario.
P2 (chronic degradation) — no rush, follow patrol trends and weekly reports.
Grading determines forensics depth vs action speed ratio — P0 doing full dump forensics on scene is itself a secondary incident. Grading standards and per-grade action lists go on page one of on-call handbook.
On-call vocabulary also worth standardizing — same anomaly, whole team uses same script:
CPU high → first report "four-step result + whether GC threads".
API slow → first report "multi-snapshot conclusion + trace located which layer".
Memory anomaly → first report "histogram top + growth rate".
Standardized vocabulary's value at escalation : receiving expert gets structured info in 30 seconds, not a few minutes of free-form narration. Vocabulary = protocol, protocol = efficiency.
Three "most easily misjudged" categories from the ten, with reminders:
CPU 100% misjudged as business code — four-step catches GC threads, must immediately switch to memory side; many teams spend half hour analyzing GC thread stacks (article 09 switch principle).
API hang misjudged as deadlock — most hangs are slow downstream + lock contention "chain reaction"; real deadlock shows deadlock block at jstack end; absent means not deadlock.
Metaspace growth misjudged as heap leak — heap histogram can't see metaspace; two investigation lines use completely different tools (article 02); first use VM.metaspace to confirm battlefield before acting.
Misjudgment costs more than no investigation: wrong direction, harder you work, further you go.
Cheatsheet must stay alive to have value: after each postmortem update corresponding row (new root cause, new stop-gap action added), quarterly purge outdated entries; print on on-call desk, electronic version on runbook homepage. A cheatsheet untouched for three months means postmortem mechanism is idling — sheet's lifecycle = team's JVM capability lifecycle.
3. Big Promo Plan: Rehearse Accidents in Advance
Big promo is a stress test at scale and a high-incidence window for JVM issues: traffic 3x, allocation rate, class loading volume, lock contention all amplify 3x; chronic diseases hidden at normal times erupt collectively. The plan answers not "how to tune" but "who, at what threshold, executes what action" — move human judgment upstream into process judgment.
3.1 Escort Timeline & Red Lines
Escort's essence is moving all "on-scene decisions" forward into "plan execution" : T-7 complete stress test and baseline archive (flame graph, histogram, GC log — all three required; no baseline = no comparison); T-1 freeze parameter changes and rehearse plan scripts (degradation switches, scale-out paths, drain sequences all walked through); T-0 only monitor and execute plan, three red lines (no dump, no param change, restore first then investigate) written into on-call handbook; T+1 postmortem against baseline to find regressions, parameters and plan iterate into archive.
Plan granularity must reach "one command executable": trigger table writes clear threshold and action, e.g., "single instance P99 continuous 5 min > 2x baseline → drain 30% traffic and preserve scene for forensics"; "Full GC frequency > 1/min → confirm if big-promo traffic pattern; if yes wait for G1 adaptive, else execute scale-out per plan" — the more a plan reads like a command, the less T-0 execution resembles gambling. A plan without executable actions equals no plan. Plan's essence is "decision forward": thresholds set at T-7, actions rehearsed at T-1, T-0 follow the script — three steps done, every decision on big-promo night has a source.
3.2 Plan Checklist: Six Action Categories Worth Pre-seeding
List all actions worth putting in plan by type, review each category during evaluation:
# ==== Big Promo Plan Checklist (Review by Category) ====
# 1. Capacity: scale-out path / warm-up traffic ramp curve
# 2. Degradation: non-core feature switch list & owners
# 3. Rate-limit: entry thresholds & trigger conditions
# 4. Forensics: JFR short-record / histogram commands
# (lightweight forensics usable at peak, not dump)
# 5. Drain: single instance removal operation sequence
# 6. Rollback: param / config / image three-tier rollback
# Every item must have: trigger threshold + exec command + ownerMost commonly missed category is forensics : peak can't dump, but 30-second JFR short-record and one histogram are perfectly doable (article 09 cost conclusion); write them as plan entries so escort doesn't get stuck between "want evidence" and "afraid to act". Next is rollback : param, config, image three-tier rollback paths all must be rehearsed — half-baked recovery that rolls back param but not config is harder to debug than no rollback.
Postmortem also deserves a fixed template, six columns one page:
Timeline (when detected, when stop-gap, when recovered)
Evidence chain (log slices, flame graphs, histogram screenshots)
Root cause (one sentence explaining mechanism)
Stop-gap action (what done, effect)
Cure action (owner + deadline)
Cheatsheet update (new entry number)
Six columns fit one page — unfinished postmortem means forensics wasn't captured then; that's the template's second role: force on-call to preserve full evidence.
Escort observation actions also worth codifying into a mnemonic:
One: watch GC dashboard (pause & frequency hugging baseline?)
Two: watch allocation rate (alloc flame graph new hotspots?)
Three: watch traffic-GC coupling (traffic 3x, GC frequency 10x = allocation anomaly signal)
Four: watch Metaspace & off-heap (peak dynamic classes & direct memory both amplify).
All four hugging baseline → peacefully execute plan; any deviation → article 09's three-layer tool progression for forensics — escort forensics always uses lightweight means; that's red line #1 in daily practice.
Scheduling and escalation paths are also part of the plan:
Escort on-call at least two people (one executes, one cross-checks; peak solo ops easily distort actions).
Three-level escalation — on-call 5 min no progress → escalate to JVM expert; 15 min no progress → escalate to architect + sync degradation decision; eliminate classic tragedy of "one person silently toughing it out for half hour".
Cross-team actions pre-aligned — DBA, network, middleware on-call contacts written in plan; finding people during incident is the most expensive time cost.
4. Core Source Code: How Parameters Take Effect
4.1 JVMFlag: A Parameter's Life
Every -XX parameter goes through "parse → validate → ergonomic" three steps; understanding this explains why some params error at startup, some silently fail at runtime:
// Parameter's life: parse → validate → ergonomic
bool Arguments::parse_argument(...) {
// Dispatch by JVMFlag type: bool/uintx/
// string each have setter, illegal values error here
}
// Unspecified params go through ergonomic derivation:
// Derive defaults by memory, cores, container env
// Mutual constraints checked uniformly in validate:
// NewSize > MaxNewSize → refuse start
// Conclusion: explicit > ergonomic > defaultKeywords for each step:
Parse layer — recognizes characters; typo in param name fails immediately (good thing).
Ergonomic layer — guesses numbers; unconfigured params derived from environment — Accident 1's default heap eating container memory is this layer's result.
Constraint layer — conflict detection; params have many mutual constraints (e.g., NewSize cannot exceed MaxNewSize). Understanding three layers explains why "reviewing VM.flags actual values" is more truthful than "reading startup script" — script holds intent, VM.flags holds fact.
4.2 Container Awareness: JVM's Eyes
// JVM's eyes: cgroup awareness (OSContainer)
size_t os::Linux::available_memory() {
if (OSContainer::is_containerized()) {
// Read cgroup memory limit,
// take min(physical, container limit)
return OSContainer::memory_limit();
}
return physical_memory();
}
// MaxRAMPercentage discounts on limit:
// MaxHeapSize = limit × 70%
// CPU side same: read quota/cpuset: wrong core count
// → GC threads, JIT threads, TLAB all derive wrongThis source code is the underlying basis for the container trio: JVM has no magic; it just honestly reads cgroup files — /sys/fs/cgroup memory.limit and cpu quota. That's why "requests=limits" is so critical: limits change → JVM's eyes change; requests lie to scheduler; misalignment = Accident 1 shape. Bare-metal path falls back to physical_memory; same config behaves differently in two environments — migration across deployment shapes must re-run parameter review.
4.3 OOM's Last Mile: Dump & Exit
// OOM tail action sequence
void report_java_out_of_memory(
const char* message) {
// 1. Execute OnOutOfMemoryError command
// (alert hook hangs here)
// 2. HeapDumpOnOutOfMemoryError:
if (HeapDumpOnOutOfMemoryError) {
HeapDumper::dump_heap();
// Reuse article 09 mechanism: inside safepoint
// full heap traversal write HPROF
}
// 3. ExitOnOutOfMemoryError:
// os::exit to restart relaunch
}This code explains the OOM trio's real behavior:
Dump is synchronous — at OOM moment process still parks in safepoint for tens of seconds writing HPROF; that's the real price of "preserve scene", and why HeapDumpPath disk must reserve heap-size space.
Exit is the safety net — post-OOM heap is shattered; continuing is more dangerous than dying; hand to restart relaunch is most services' correct answer. Sequence fixed: alert first, forensics second, exit last — incident scene's value hangs on those tens of seconds of persistence. Also OnOutOfMemoryError hook must not do heavy work — it runs at the problem scene too; urgent alerts go external monitoring for stability.
4.4 PrintFlagsFinal: Parameter Review's Evidence
// View "actual effective" params (review god-tool)
java -XX:+PrintCommandLineFlags -version
# Outputs explicitly specified key items
java -XX:+PrintFlagsFinal -version
| grep HeapSize
# Full flag table, source three tags:
# {product} = factory default
# {ergonomic} = environment derived
# {command line} = explicitly specified
# Check item 10 basis: diff expected vs actualConcluding source code gave implementation basis for check item 10: parameter's real source splits into three tags ; any item you thought explicitly configured but shows ergonomic is a review must-ask. Diff PrintFlagsFinal output against review baseline = simplest "configuration drift detection" — more trustworthy than any config management platform screenshot, because it comes from JVM's own testimony.
Drift governance goes further: feed PrintFlagsFinal output into patrol diff — weekly auto-diff each instance's actual params against review baseline; any drift (someone manually edited startup script, a release brought new param) gets caught. Drift isn't scary; undetected drift is: most "same service different instances behave differently" ghosts end up in flag diff. Config credibility comes from "verifiable", not "I remember setting it".
5. Seven Closing Accounts: JVM Total Cost Ledger
The closing seven accounts tally the "series total cost view" — previous nine articles did single-point accounts; here they merge into one master table, each account citing source article:
Parameter Account. Explicit > ergonomic > default; ~20 explicit params cover 95% scenarios — parameter rationale matters more than parameter count (this article 1.2/1.4).
Observability Account. Always-on config ~1% overhead, buys "post-incident replay" and "early leak detection"; skipping observability saves now, pays back with interest at first incident (article 09 seven accounts observability part).
Capacity Account. Heap at 70% of container memory golden line: heap too large → long STW & crowds off-heap; heap too small → frequent GC; 30% headroom is off-heap lifeline, not waste (articles 01/02 container derivation chain).
Timing Account. Parameter changes go through window + canary, one variable at a time; big-promo escort freezes params — any "quick param tweak" is gambling (this article 1.4 SOP & red lines).
Rehearsal Account. SOP scripts run on real machine once beats docs written ten times; check item 9 "rehearsed" is highest gold content of the ten (this article 2.1).
Postmortem Account. One incident → one cheatsheet entry; teams with monthly "new problems" owe postmortem debt. Knowledge not archived → repeatedly pay tuition (this article 2.2 & Accident 3).
Mindset Account. Whole series converges to three questions — Memory Budget (heap/metaspace/direct boundaries & reclamation), Execution Optimization (class loading/JIT/inlining/escape analysis gains & backfires), Observability Discipline (always-on, light-first, multi-snapshot comparison, postmortem archival) . All JVM production problems are variants of these three (this article section 7 knowledge map).
6. 20 Interview Questions (Closing Edition)
The article includes 20 Q&A covering container heap sizing, requests/limits mismatch, Xms=Xmx rationale, MaxGCPauseMillis too small backfire, ActiveProcessorCount when to set, OOM trio config & order, most missed check item, verifying actual parameter effectiveness, big-promo three red lines, Full GC frequent investigation path, heap vs direct memory OOM distinction, MaxMetaspaceSize sizing, ten incident investigation common principles, when to use AppCDS, correct parameter tuning order, key GC log fields, JFR vs GC log division, building effective postmortem mechanism, when to migrate G1→ZGC, and three-question series summary. Each answer is concise, grounded in earlier articles, and includes specific thresholds, commands, and rationales.
7. Full Series Knowledge Map
Ten articles aren't ten islands; they're ten stations on three main lines:
Memory Line — object's life (born into TLAB, survive via generations, reclaimed via concurrency — articles 02/03/04/05/06).
Execution Line — code's life (byte stream → Class, interpreted → compiled, optimization brings backfire — articles 07/08, meshing with 06's inlining/escape).
Observability Line — problem's life (phenomenon, forensics, attribution, stop-gap, cure, archive — articles 09/10). Article 01 panorama is the three-line intersection index.
Reading suggestion: re-read by main line yields higher return than sequential — binge memory line four articles, TLAB, generations, Region, concurrent cycle concepts auto-explain each other; execution line same. Isolated knowledge forgets easily; main-line structure sticks — that's why each article opens with "continuing from previous": knowledge only becomes a net, then flexible application.
Three lines also intersect; intersections are high-frequency interview & practice zones:
Memory × Execution — escape analysis decides object off-heap (06), inlining decides escape analysis visibility (08), class loading decides JIT compilation target (07).
Execution × Observability — JIT storms need 09's profiler & compilation logs (08/09), class conflicts need sc -d & class+load logs (07/09).
Memory × Observability — GC tuning prerequisite is logs & flame graphs (03-05 & 09), direct memory needs NMT (02/09).
Intersection zones are where "integration" happens — reading single article isn't mastery; cross-referencing at intersections means you've truly wired the JVM into one machine.
One-sentence essence per article as series index:
01 Panorama — runtime five memory zones & param system, JVM's map page.
02 Memory Layout — each zone's boundary params & NMT observation, who owns which ledger.
03 GC Basics — reachability analysis & generational hypothesis, recycling's "why".
04 G1 — Region-based generations, pause-target-driven concurrent collection.
05 ZGC — colored pointers & concurrent compaction, pause-floor cost model.
06 Allocation — TLAB & escape analysis, art of making objects "not enter heap".
07 Class Loading — byte stream × loaders & parent delegation, conflict & cold-start cure.
08 JIT — tiered compilation & inlining, all secrets of code running faster.
09 Diagnostics — three-layer tools & safepoint mechanism, cost of seeing & intervening.
10 Practice — assemble everything into skeleton, checklist, plan & map.
To make series "read thick": memory line pair with Deep Understanding of JVM (ch2/3 ↔ 02-05, ch7 ↔ 07); execution line supplement RednaxelaFX's JIT series; observability line run 09's profiler five modes on your own service — one real flame graph beats ten readings .
8. One-Page Cheatsheet (Ten Closing Sentences)
JVM Production Practice Ten Sentences (Series Guiding Principle): Heap is budget: Xms=Xmx , container 70%, off-heap trio set caps. GC keeps default line: pause target 200ms, IHOP & logs co-tune. Container trio: UseContainerSupport , MaxRAMPercentage , requests=limits . Observability installed before incident: GC/safepoint logs, JFR, OOM auto-dump always on. Check ten items all green before traffic; item 9 "rehearsed" highest gold. Incident first check cheatsheet: stop-gap before cure, evidence before action. Big promo escort: T-7 baseline, T-1 freeze, T-0 execute, T+1 archive; red lines no dump, no param change. Parameter change one variable at a time; PrintFlagsFinal is the only truth. Postmortem must archive: one incident → one cheatsheet entry. Three-question guiding principle: Memory Budget × Execution Optimization × Observability Discipline — all JVM problems are their variants.
Usage: print & wall-mount ①③④⑤⑥ (daily & launch), on-call handbook enlarge ⑥⑦ (incident & promo), new hires start with ⑩ (three-question principle, later nine sentences hang on its branches). Cheatsheet is index not full text — each sentence backs a full article's methodology; forget details, follow index back to article.
Compressed for time-pressed readers:
Heap budget (70%) GC default (200ms) observability first always-on (logs + JFR + OOM dump), check ten items all green release; incident first check table (stop-gap before evidence before), big promo relies on plan (T-7 rehearse T-0 execute); parameter one change at a time, postmortem every entry archived; Memory Budget, Execution Optimization, Observability Discipline — three questions answered, JVM graduated.
9. Closing: Series Finale
Looking back at three opening accidents: container without budget → answer is five-zone parameter skeleton; big promo without plan → answer is T-7 rehearsal + T-0 execute; monthly recurring "new" problems → answer is cheatsheet + postmortem archive.
Production practice's entire mindset in one sentence: eliminate all uncertainties before launch — parameters have rationale, observability is always-on, plan is executable, knowledge archives; achieve these four, JVM ceases to be mysticism and becomes a transparent machine with predictable behavior and explainable problems.
This methodology doesn't depend on any specific version — JDK 21 virtual threads, future GC evolutions change the "three questions" specific answers, not the "budget first, observability first, plan first" answering discipline. Internalize discipline into team habit; tech evolution is just swapping ammo, not relearning the gun.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
