Backend Development 22 min read

JVM Crash Caused by JIT Compilation in a Scheduled Task System and How to Resolve It

A scheduled task system that frequently accesses MySQL and cache crashes at night due to a SIGSEGV in the HotSpot JVM caused by JIT compilation of hotspot code, and the article explains the JVM internals, JIT behavior, and provides configuration and driver downgrade solutions to prevent the failure.

JD Retail Technology
JD Retail Technology
JD Retail Technology
JVM Crash Caused by JIT Compilation in a Scheduled Task System and How to Resolve It

The system is a scheduled‑task service that periodically runs business code accessing MySQL and a cache. After running normally for a while, the Java process disappears at midnight, leaving only a crash log indicating a fatal SIGSEGV in the HotSpot JVM.

cat: /proc/1/environ: Permission denied [admin@host-11-40-38-52 ~]$ more hs_err_pid231.log # A fatal error has been detected by the Java Runtime Environment: # SIGSEGV (0xb) at pc=0x00007f21a8c21325, pid=231, tid=139779725313792 # JRE version: Java(TM) SE Runtime Environment (8.0_20-b26) (build 1.8.0_20-b26) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.20-b23 mixed mode linux-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x858325] LoadKlassNode::make(PhaseGVN&, Node*, Node*, TypePtr const*, TypeKlassPtr const*)+0x45 # Core dump written. Default location: /home/admin/core or core.231 # ... (full crash log omitted for brevity) ...

After the JVM finishes initialization, bytecode is first interpreted; when a method becomes hot, the Just‑In‑Time (JIT) compiler translates it to native machine code for better performance. HotSpot uses two JIT compilers: the fast, locality‑focused C1 (Client) compiler and the optimizing C2 (Server) compiler.

Hotspot detection relies on two counters per method: an invocation counter and a back‑edge counter. When the sum exceeds a threshold (default 1500 for C1, 10000 for C2, configurable via -XX:CompileThreshold ), the method is considered hot and JIT compilation is triggered.

In this case, the scheduled task repeatedly executed the same code, causing the JIT compiler to compile it. During compilation an internal error occurred, leading to the JVM crash.

Two main remediation approaches are suggested:

Adjust JVM options to control JIT behavior, for example: java -Xint -version # interpreter‑only mode java -Xcomp -version # compile‑only mode java -Xmixed -version # mixed mode (default) -XX:+PrintCompilation # print JIT compilation info -XX:CompileCommand=exclude,package/Class,methodName # exclude specific method from JIT -XX:CompileCommand='compileonly,java/lang/StringBuffer.*' # compile only specific methods -XX:CompileThreshold=1000 # lower the compilation trigger threshold

Rollback the MySQL driver to a stable version, as the issue appeared after upgrading to the driver bundled with spring-boot-2.1.6.RELEASE . Using an older, proven driver eliminates the crash.

By either limiting JIT compilation for the problematic method or using a stable JDBC driver, the scheduled task can run continuously without causing the JVM to abort.

backendjavaJVMperformanceJITcrash
JD Retail Technology
Written by

JD Retail Technology

Official platform of JD Retail Technology, delivering insightful R&D news and a deep look into the lives and work of technologists.

0 followers
Reader feedback

How this landed with the community

login Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.