Tagged articles
performance debugging
60 articles · Page 1 of 1
This guide shows test developers how to customize Nginx log formats and use concise awk commands to quickly locate slow requests, error codes, abnormal traffic, and common pitfalls, turning dense log files into actionable debugging data.
Nginxawkerror monitoring
0 likes · 11 min read

After optimizing a Spring Boot channel service, a mysterious extra ~100 ms appeared per request; detailed tracing with Arthas revealed that embedded Tomcat repeatedly loads Swagger UI META‑INF resources, causing a 70 ms overhead, which can be eliminated by upgrading Tomcat or removing the Swagger dependencies.
ArthasEmbedded Tomcat BugSpring Boot
0 likes · 30 min read

This guide explains the five common sources of GPU memory consumption in large‑model inference services, provides a step‑by‑step diagnosis workflow—from static usage and KV‑Cache analysis to concurrency and K8s scheduling—offers concrete command‑line checks, scripts, configuration examples, and actionable remediation and monitoring recommendations.
GPU memoryKV CacheLLM OOM
0 likes · 28 min read

A Spring Boot channel service showed an unexpected 100 ms latency; by systematically checking network, using curl, and employing Arthas to trace Spring MVC and Tomcat internals, the author discovered a Tomcat‑embed bug that repeatedly loads Swagger‑UI JAR resources, which is resolved by upgrading Tomcat.
ArthasEmbedded TomcatSpring Boot
0 likes · 15 min read

This article explains why mobile observability often falls into a black‑hole, outlines a four‑step solution that makes the mobile client the first hop of a distributed trace by sharing a common Trace ID, and demonstrates the approach with a real‑world slow‑query debugging case using Alibaba Cloud RUM.
APMCloud NativeObservability
0 likes · 13 min read

During Pinterest’s migration of its custom search platform Manas to the PinCompute Kubernetes environment, a rare latency spike—one request per million taking 100 times longer—was traced to cAdvisor’s memory‑intensive smaps scans, revealing hidden resource contention and prompting a targeted fix.
KubernetesPinComputecAdvisor
0 likes · 13 min read

This guide explains why Java developers need profiling, introduces IntelliJ IDEA’s built‑in Profiler (powered by Async Profiler and JFR), and provides step‑by‑step instructions with screenshots for CPU, memory, and thread analysis to diagnose slow endpoints, high CPU usage, memory leaks, and concurrency bottlenecks.
CPU analysisIntelliJ IDEAJProfiler
0 likes · 12 min read

A developer encountered a sudden CPU spike caused by excessive JVM garbage collection in a single Kubernetes pod, and by using Linux monitoring tools, thread‑ID conversion, jstack analysis, and file transfer techniques pinpointed a flawed Excel export implementation that created massive in‑memory lists, ultimately fixing the issue.
JVMKubernetesLinux
0 likes · 6 min read

This guide explains how CPU scheduling works, why 100% CPU usage occurs, and provides a step‑by‑step troubleshooting workflow—including monitoring with top/vmstat, identifying offending threads, analyzing stack traces, and applying both quick‑fix and long‑term remediation techniques—to keep systems stable.
CPU SchedulingLinuxSystem Monitoring
0 likes · 19 min read

Go 1.25 introduces Flight Recorder, a lightweight in‑memory trace buffer that captures recent execution data and can be snapshotted on demand, enabling developers to retroactively investigate latency spikes in long‑running services without the overhead of continuous tracing.
BackendFlight RecorderGo
0 likes · 10 min read

During Pinterest’s migration of its high‑traffic Manas search platform to the PinCompute Kubernetes environment, engineers observed an extremely rare latency outlier—one in a million requests took 100 times longer—prompting a deep investigation that traced the root cause to cAdvisor’s memory‑intensive smaps scans interfering with leaf node processing.
Cloud NativeKubernetescAdvisor
0 likes · 14 min read

This article walks through a real‑world production‑environment debugging case, showing how tracing with SkyWalking, deep GC log analysis, mis‑configured c3p0 pools and Finalizer behavior can combine to cause intermittent time‑outs and high latency in a distributed Java service.
Connection PoolGCmemory analysis
0 likes · 24 min read


ITPUBJun 24, 2025 · Backend Development
This guide explains why Java applications can hit 100% CPU in production, introduces the two primary diagnostic tools jstack and Arthas, and provides step‑by‑step commands and code examples to locate the exact source line responsible for the CPU spike.
ArthasCPU profilingJava
0 likes · 13 min read

This article explains how to use SQL coloring in MyBatis by implementing a lightweight interceptor or an AspectJ weave to annotate each SELECT statement with its mapper ID and execution stack, enabling rapid identification of performance bottlenecks during high‑traffic events.
AspectJJavaMyBatis
0 likes · 29 min read

This tutorial introduces the BPF Compiler Collection (BCC) suite, explains how to install it, lists essential Linux commands, and provides step‑by‑step examples of each BCC tool for fast performance analysis, fault isolation, and network troubleshooting on Linux systems.
BCCLinux ObservabilitySystem Tracing
0 likes · 17 min read

This guide explains how to locate the cause of sudden CPU spikes in a Java program by using the top command to identify the offending process and thread, converting the thread ID to hexadecimal, and analyzing stack traces with jstack to pinpoint the problematic code line.
BackendCPU profilingJava
0 likes · 4 min read

When a production Java service repeatedly restarts and triggers full GC and OutOfMemoryError alerts, this guide shows how to capture heap dumps using JVM flags, install and configure Eclipse Memory Analyzer (MAT), and systematically analyze the dump to pinpoint memory leaks, high usage, and problematic code.
Heap DumpJavaMemory Analyzer
0 likes · 6 min read

After deploying the full‑precision DeepSeek‑R1 model on a 2×8‑GPU ACS cluster, repeated stress tests showed GPU memory usage continuously rising without release; this article details the investigation, reproduces the behavior, examines vLLM logs, Prometheus metrics, and reveals PyTorch’s caching allocator as the root cause, offering mitigation tips.
DeepSeekGPU memoryMemory Cache
0 likes · 21 min read

This article explains Java thread run states, analyzes why threads appear in WAITING or TIMED_WAITING during concurrent tasks, shows flame‑graph diagnostics, and offers concrete code‑level optimizations to reduce CPU pressure and improve overall backend performance.
ConcurrencyFlame GraphJava
0 likes · 7 min read

This article outlines a comprehensive approach to identifying and resolving both error‑related and slow‑request problems in online systems by leveraging trace data, log correlation, method‑stack analysis, unified entity models, and large‑language‑model assistance to accelerate root‑cause diagnosis.
APMLLMRoot Cause Analysis
0 likes · 12 min read

Kyanos is an open‑source command‑line utility that leverages eBPF to provide low‑overhead, kernel‑compatible network tracing and performance analysis for HTTP, MySQL, and Redis traffic, offering simple watch and stat commands that replace slow tcpdump workflows with seconds‑level diagnostics.
Observabilitycommand-line tooleBPF
0 likes · 11 min read

An October 2023 outage of the bfe‑customer‑application‑query‑svc revealed how traffic spikes, downstream latency, and a mis‑configured custom thread pool saturated Tomcat's worker threads, causing health‑check failures, pod restarts, and a complete zone failure.
MicroservicesTomcatcapacity planning
0 likes · 33 min read

This article documents the step‑by‑step investigation of an unexpected ~100 ms response delay in a Spring Boot channel service, showing how network checks, curl tests, Arthas trace/stack commands, and source analysis revealed a Tomcat‑embed bug caused by Swagger‑ui resources and how upgrading Tomcat resolved the issue.
ArthasBackendJava
0 likes · 14 min read

The article details how repeatedly calling FileSystem.get(uri, conf, user) created distinct UserGroupInformation objects, inflating the static FileSystem cache and causing a heap‑memory leak that triggered an Out‑Of‑Memory error, and explains that using the two‑argument get method or explicitly closing instances resolves the issue.
HadoopJavaMemory Leak
0 likes · 13 min read

This article demonstrates how to quickly locate and resolve frequent full GC and CPU spikes in Java backend services by combining Arthas‑integrated async‑profiler flame graphs with VisualVM, GCEasy analysis, and practical step‑by‑step deployment procedures.
ArthasCPU profilingGC
0 likes · 7 min read

This article details the systematic investigation of intermittent availability drops in a high‑throughput Go service, covering hypothesis formulation, extensive profiling with pprof, gctrace, strace, fgprof, go trace, heap analysis, the discovery of a gcache LFU bug, and the final remediation steps.
BackendGCGo
0 likes · 10 min read

This tutorial demonstrates how to employ the open‑source Java diagnostic tool Arthas to quickly locate and analyze a high‑CPU problem in a running JVM by leveraging commands such as dashboard, thread, jad, watch, and ognl, complete with code examples and step‑by‑step instructions.
ArthasCPU profilingJava
0 likes · 7 min read

After receiving an alarm that a production container’s CPU usage surged past 90%, I investigated the JVM metrics, discovered excessive young and full GCs in a single pod, and walked through the detailed troubleshooting steps—including top, thread analysis, jstack, and code fixes—that resolved the issue.
CPU spikeGCJVM
0 likes · 7 min read

This guide shows how to use the open‑source Java diagnostic tool Arthas to attach to a running JVM, view real‑time metrics, and trace method execution without inserting manual timing code, enabling quick pinpointing and optimization of performance bottlenecks.
ArthasJava profilingSpring Boot
0 likes · 8 min read

This guide walks you through using IntelliJ IDEA Ultimate’s built‑in Profiler to analyze Java CPU and memory performance, covering quick start steps, visualizations such as hotspot maps, call trees, method lists, timelines, real‑time charts, and exporting results as .jfr and .hprof files.
CPU analysisIntelliJ IDEAJava profiling
0 likes · 5 min read

An investigation of a 100k‑QPS Go monolith revealed that intermittent availability drops were caused by a memory‑leak in the third‑party gcache LFU implementation, which inflated GC work and produced long mark phases; upgrading gcache eliminated the leak and restored 0.999+ availability, highlighting the need for thorough observability and dependency monitoring.
GoTracinggarbage collection
0 likes · 10 min read

The article explains how null or empty query values caused MongoDB to ignore the intended index, leading to massive slow queries and service timeouts, and details the step‑by‑step investigation, plan‑cache invalidation, and the corrective addition of a compound index that restored normal performance.
MongoDBPlan CacheQuery Optimization
0 likes · 21 min read

This article walks through the step‑by‑step investigation of an unexpected ~100 ms latency in a Spring Boot channel service, using network checks, curl timing, and the Arthas Java diagnostic tool to pinpoint a Tomcat‑embed bug caused by Swagger jars, and then shows how upgrading Tomcat or Spring Boot resolves the problem.
ArthasSpring BootTomcat
0 likes · 14 min read

Using lightweight dynamic‑tracing tools that record allocator calls and page‑fault events, the authors diagnose a production MySQL‑proxy memory leak in TDSQL, generate focused flame‑graphs with custom memstacks and pgfaultstacks, and demonstrate a fast, source‑independent alternative to gdb or Valgrind.
Flame GraphLinuxMemory Leak
0 likes · 13 min read

After releasing a new version of a VSCode extension, intermittent freezes were traced to a memory leak; the investigation used llnode and heapdump, tackled Electron version mismatches, extended DevTools parsing limits, and ultimately identified recursive socket callbacks as the root cause.
DevToolsMemory LeakVSCode
0 likes · 15 min read

This article details a step‑by‑step investigation of a Java Spring‑Boot service that suffered nightly response‑time alerts, revealing that misconfigured Logback file paths caused cross‑volume log rotation, thread blocking, and ultimately a production outage, and shows how gray‑deployment and environment fixes resolved the issue.
JavaKubernetesLogback
0 likes · 13 min read

This tutorial demonstrates how to download, attach, and use the Arthas Java diagnostic tool—leveraging commands like dashboard, thread, jad, watch, and ognl—to quickly locate and fix high CPU problems caused by parallel stream code in a Java application.
ArthasCPU profilingJava
0 likes · 7 min read

This article shares practical methods and real‑world case studies for IT professionals to analyze, locate, and fix system runtime issues, service timeouts, file‑handle leaks, JVM memory overflows, and performance bottlenecks, emphasizing hypothesis testing, boundary narrowing, and systematic post‑mortems.
IT OperationsJVM memoryfile handle leak
0 likes · 31 min read

A Spring Boot channel service suffered an unexplained 100 ms extra latency, which was traced using Arthas to Tomcat’s handling of META‑INF resources from Swagger’s JAR, and the issue was resolved by upgrading the embedded Tomcat version.
ArthasJavaSpring Boot
0 likes · 29 min read

This article demonstrates how to combine Linux top with -H -p options and jstack to correlate Java thread IDs with their logical names, explains the underlying JDK thread‑naming bug, and provides practical recommendations for performance debugging on different JDK versions.
JDKJavaLinux
0 likes · 6 min read

After migrating a project to the MDP framework based on Spring Boot, the system repeatedly reported excessive swap usage; the investigation revealed that native memory allocated by the Spring Boot classloader’s Reflections scanning and InflaterInputStream caused 700 MB–800 MB of off‑heap memory to remain unreleased, which was eventually resolved by limiting the scan path and updating Spring Boot.
Memory LeakNative MemorySpring Boot
0 likes · 12 min read

This article walks through the step‑by‑step investigation of an unexpected ~100 ms latency in a Spring Boot channel service, using network checks, curl timing, Arthas trace and watch commands to pinpoint a Tomcat‑embed bug caused by Swagger‑UI JAR loading, and then shows how upgrading Tomcat resolves the issue.
ArthasEmbedded ServerJava
0 likes · 30 min read

This article explains the evolution from a monolithic architecture to microservices, outlines the new pain points such as fault isolation, performance bottlenecks and scaling inefficiencies, and presents a practical, low‑cost distributed tracing solution with unified frameworks, components, configuration management, data collection, and visualization.
Configuration ManagementObservabilitydistributed-tracing
0 likes · 31 min read

This article details the step‑by‑step investigation of an unexpected ~100 ms response delay in a Spring Boot‑based channel system, showing how network checks, curl measurements, and deep tracing with the Arthas Java diagnostic tool pinpointed a Tomcat‑embed bug caused by Swagger‑UI resources and how upgrading Tomcat resolved the problem.
ArthasJavaSpring Boot
0 likes · 30 min read

This article explains how to reproduce a slow‑response SpringBoot service using JMeter, then quickly locate the bottleneck in production by leveraging Alibaba's open‑source Arthas tool and its trace command to analyze the full call chain.
ArthasJMeterperformance debugging
0 likes · 9 min read

Learn how to export a live heap dump from a Java service, use JProfiler 9.x to open the .hprof file, identify large objects such as Zipkin's InMemoryReporterMetrics, and pinpoint memory leaks that can consume gigabytes of RAM.
Heap DumpJProfilerJava
0 likes · 2 min read

The article details a step‑by‑step investigation of abnormal swap memory consumption in a Spring Boot project, revealing that native memory allocated by the Inflater during JAR scanning was not released promptly, leading to apparent memory leaks that were ultimately resolved by configuring package scanning and updating Spring Boot.
JavaMemory LeakNative Memory
0 likes · 12 min read

This article explains how to generate a heap dump with jmap, import it into Eclipse Memory Analyzer (MAT), and interpret the various reports such as Histogram, Dominator Tree, Top Consumers, and Leak Suspects to identify memory leaks or insufficient heap sizing in Java applications.
Heap DumpJavaMAT
0 likes · 8 min read

The article details a step‑by‑step investigation of why a Spring Boot service migrated to the MDP framework consumed far more physical memory than its 4 GB heap, revealing native‑code allocations, memory‑pool behavior of glibc and tcmalloc, and how limiting MCC scan paths or upgrading Spring Boot resolves the off‑heap leak.
JavaMemory LeakNative Memory
0 likes · 11 min read

Using the open‑source Arthas tool, the author traced a Java server’s 99 % CPU usage to two runaway threads, inspected their stack traces, discovered a cyclic bucket in a HashBiMap caused by unsynchronized cache updates, and resolved the issue by adding a synchronized keyword to the cache‑sync method.
ArthasCPU profilingJava
0 likes · 10 min read

This article explains the principles behind Android Systrace, details its implementation using ftrace, and shows how to add custom trace categories and parsers via Catapult to visualize kernel‑level information in Chrome's tracing UI.
AndroidCatapultCustom Trace
0 likes · 8 min read

This article details a step‑by‑step investigation of unusually high native memory consumption in a Spring Boot service, covering JVM configuration, system‑level diagnostics with jcmd, pmap, gperftools, strace, GDB, and jstack, and explains how the MCC component’s default package scanning caused the leak and how configuring scan paths or upgrading Spring Boot resolved the issue.
JVMLinux toolsMemory Leak
0 likes · 11 min read

A Spring Boot channel service suffered an unexplained ~100 ms delay per request, which was traced through network checks, curl timings, and deep Arthas analysis to reveal a Tomcat‑embed bug caused by repeated loading of Swagger static resources, ultimately fixed by upgrading Tomcat.
ArthasEmbedded Tomcat BugSpring Boot
0 likes · 30 min read

This article explains why Dubbo's default thread pool can become exhausted, demonstrates how to reproduce the issue, and shows how to use Arthas's dashboard and thread commands to pinpoint and resolve thread‑pool problems in Java services.
ArthasDubboJava
0 likes · 10 min read

This article introduces common memory‑leak detection tools, explains how to analyze heap memory consumption of a C++ process using AddressSanitizer, Valgrind and the core_analyzer utility, and provides practical optimization suggestions for reducing memory bloat and fragmentation.
C++Heap Analysiscore analyzer
0 likes · 18 min read

After moving to Spring Boot, the application consumed up to 7 GB of native memory because Meituan’s MCC package scanner invoked Spring’s ZipInflaterInputStream, which allocated large off‑heap buffers during JAR decompression that were only freed by the JVM finalizer and retained by glibc’s 64 MB arenas; restricting the scan scope or upgrading to Spring Boot 2.0.5 eliminated the excess usage.
JVMMemory LeakNative Memory
0 likes · 13 min read

This article walks through setting up a Vagrant‑based experiment that injects an intentional infinite loop into a simple Spring MVC service, then demonstrates step‑by‑step how to identify the offending process using top, examine JVM heap with jstat, and trace the problematic thread with jstack to resolve a CPU‑100% issue.
CPUJVMJava
0 likes · 4 min read

After a recent feature release caused a sharp decline in a key data metric, the team followed a systematic, fourteen‑step troubleshooting process—including verification, code review, DBA involvement, local debugging, environment comparison, logging, packet capture, service restarts, request mode changes, load testing, and partition resizing—to identify and resolve a Kafka‑related throughput bottleneck.
Kafkaasynchronous vs synchronousdata pipeline
0 likes · 8 min read

This article walks through a real‑world incident where a data platform’s CPU usage surged to 98%, detailing how to pinpoint the high‑load process, trace the offending Java thread, uncover a time‑utility method bottleneck, and apply a concise fix that reduced load by thirtyfold.
CPUJavaoperations
0 likes · 7 min read

This article demonstrates how to use strace, including its -c, -T, and -e options, to identify kernel‑level system calls such as clone that cause high CPU consumption in PHP processes on a Linux server, providing step‑by‑step commands and interpretation of the results.
LinuxPHPperformance debugging
0 likes · 4 min read
