Why a Hidden 70 ms Delay in Spring Boot Was Caused by Swagger’s JAR – and How to Fix It

Background

The company has a channel system that bridges third‑party channels, mainly handling message conversion and parameter validation. After code optimizations the response time still fell short, showing a mysterious ~100 ms extra delay between the recorded processing time and the caller's observed response time.

Investigation Process

Code Analysis

The channel system is a typical Spring Boot web project using the embedded Tomcat. No special filters or interceptors were found, so business code was initially ruled out.

Call Flow Analysis

Network path:

Nginx → reverse proxy → channel system

Ping tests showed no latency between the client and Nginx or between Nginx and the channel server.

Directly calling the service via localhost still incurred ~73 ms for an empty endpoint on the first request, while a subsequent immediate request took only ~3 ms, suggesting a caching effect.

Since the issue could not be reproduced locally, the Arthas diagnostic tool was employed.

Using Arthas to Diagnose

Arthas is an open‑source Java diagnostic tool. The trace command was used to monitor method execution times.

Trace the Spring MVC entry point: trace org.springframework.web.servlet.DispatcherServlet * Identify that Spring MVC consumed only ~18 ms of the total 115 ms observed by the client.

The remaining time was traced to Tomcat. Further tracing of org.apache.coyote.http11.Http11Processor.service revealed a ~129 ms cost, with a significant 74 ms spent in TomcatJarInputStream.getNextJarEntry(), which loads META‑INF resources from JAR files.

Watching TomcatJarInputStream.createZipEntry showed it repeatedly loading resources such as META-INF/ and Swagger UI files.

Root Cause

The embedded Tomcat (version 8.5.31) repeatedly parsed Swagger UI static resources from the JAR, causing the latency. The issue disappeared after removing the Swagger dependencies.

Solution

Upgrade the embedded Tomcat to version 8.5.40 or later (or upgrade Spring Boot to a version that includes a newer Tomcat). For Maven projects, override the tomcat.version property:

<properties>
    <tomcat.version>8.5.40</tomcat.version>
</properties>

Alternatively, upgrade Spring Boot to 2.1.0.RELEASE or newer, which bundles Tomcat 8.5.40+.

Tomcat Embed Bug Analysis & Fix

The bug resides in org.apache.catalina.mapper.Mapper#internalMapWrapper, which unnecessarily checks static resources on each request. Tomcat caches these resources in org.apache.catalina.webresources.Cache with a default TTL of 5000 ms, explaining why consecutive requests were fast.

Local reproduction failed because the Spring Boot packaging method changes how Tomcat accesses static resources.