Understanding Service Jitter and JVM JIT Compilation

When a service is deployed, intermittent latency spikes ("jitter") may occur, causing response times to rise from a few milliseconds to seconds.

The article describes a case where a compute‑intensive service, cached with Redis and Guava, handled ~1300 QPS with 1‑3 ms latency, but later experienced jitter during iterations, with response times exceeding 50‑100 ms.

Investigation revealed that the JVM’s Just‑In‑Time (JIT) compiler was triggered, leading to high CPU usage, increased thread count, and long compilation times.

It explains the role of the interpreter and JIT, the difference between C1 (client) and C2 (server) compilers, and the tiered compilation mechanism introduced in Java 8.

Hot‑spot detection methods (sampling vs. counters) and the thresholds for method‑call and back‑edge counters are detailed, showing how they trigger JIT compilation.

The compilation process consists of three stages: building a high‑level intermediate representation (HIR), converting to low‑level IR (LIR), and register allocation with machine‑code generation.

To mitigate jitter, the article suggests pre‑warming hot code before traffic arrives, disabling tiered compilation with "-XX:-TieredCompilation -server", or adjusting compilation thresholds.

Additional JVM options such as -Xint, -Xcomp, and background compilation settings are provided for fine‑grained control.