Hidden 70 ms Delay in Spring Boot: Tomcat Embed Bug Triggered by Swagger UI

Background

The company runs a channel system built with Spring Boot and an embedded Tomcat server. Its only responsibilities are message conversion and parameter validation. After code optimizations the response time was still about 100 ms slower than expected: the server logged ~150 ms processing time while the client observed ~250 ms.

Localization Process

Analyze the code – No custom filters or interceptors were found, so the problem was not in business logic.

Analyze the call flow – The request path is Nginx → Channel System. Ping tests showed no network latency. Directly calling the service via curl on localhost:7744 still took ~73 ms for the first request and ~3 ms for the second, indicating a caching effect.

Use Arthas for deep tracing

Arthas Tracing

Arthas, Alibaba’s Java diagnostic tool, was used to trace the execution path.

Trace Spring MVC DispatcherServlet

[arthas@24851]$ trace org.springframework.web.servlet.DispatcherServlet *

The trace showed only about 18 ms spent inside Spring MVC, far less than the total latency.

Stack the DispatcherServlet

[arthas@24851]$ stack org.springframework.web.servlet.DispatcherServlet *

The stack revealed a long call chain ending in org.apache.coyote.http11.Http11Processor.service.

Trace Http11Processor.service

[arthas@24851]$ trace org.apache.coyote.http11.Http11Processor service

This trace reported a total cost of 269 ms, with a 129 ms segment inside org.apache.catalina.core.Adapter.service.

Trace CoyoteAdapter.service

[arthas@24851]$ trace org.apache.catalina.webresources.AbstractArchiveResourceSet getArchiveEntries

The trace highlighted a 74 ms cost in TomcatJarInputStream.getNextJarEntry, which loads JAR entries repeatedly.

Watch createZipEntry

[arthas@24851]$ watch org.apache.catalina.webresources.TomcatJarInputStream createZipEntry "{params[0]}"

The watch output listed many META-INF resources such as swagger-ui static files being loaded on each request.

Root Cause

Tomcat’s embedded version (8.5.31, used by Spring Boot 2.0.2.RELEASE) loads static resources from JARs on every request when handling META-INF entries. Swagger UI adds dozens of such resources, causing a ~70 ms overhead per request. The overhead disappears after the cache (TTL 5000 ms) expires, which explains why consecutive requests are fast.

Why It Was Not Reproducible Locally

The issue appears only when the application is packaged with Spring Boot’s repackaged JAR. The repackaging changes the class‑loading mechanism, so the bug does not manifest in a plain IDE run.

Solutions

Upgrade the embedded Tomcat version to 8.5.40 or newer (or upgrade Spring Boot to 2.1.0+ which bundles a newer Tomcat).

Override Tomcat version in Maven by adding a properties section:

<properties>
  <tomcat.version>8.5.40</tomcat.version>
</properties>

Remove Swagger UI dependencies if they are not needed; after removing springfox-swagger2 and springfox-swagger-ui the extra latency disappears.

Key Takeaways

Embedded Tomcat can become a hidden performance bottleneck when loading many static resources from JARs.

Arthas trace and watch commands are effective for pinpointing the exact method and parameters causing the delay.

Upgrading the server library or eliminating unnecessary dependencies resolves the issue.