How We Tackled High CPU and Full GC Issues in a Custom Microservice Gateway

Background

CoolHome started a service‑oriented transformation in early 2016. Because of special constraints they could not use mainstream Dubbo or Spring Cloud, the R&D team built a custom microservice framework on top of open‑source code and quickly launched it.

Like other microservice frameworks, the custom one includes a dedicated service gateway, and this article discusses several problems encountered in the gateway.

What is a service gateway?

The service gateway encapsulates internal architecture and provides a customized API for each client. It also offers authentication, rate limiting, circuit breaking, security, etc. All clients access microservices through a unified gateway, which handles non‑business functions and typically exposes a REST/HTTP API. The gateway forwards client requests to the appropriate microservice.

Below is a simple diagram of the gateway’s position and role in a typical system.

We will gradually introduce the various pitfalls we encountered while iterating the microservice framework, focusing on the gateway.

Incident Review

High CPU Usage

On a day in 2016, the gateway continuously triggered CPU alarms. Investigation showed many threads performing the same operation, mainly string processing besides I/O.

Initially the gateway simulated Spring MVC request mapping to route requests, which required traversing all API patterns and performing many regex matches, leading to high CPU usage, especially with numerous Restful APIs containing PathVariable.

We observed that RESTful URLs share a static prefix (namespace) followed by dynamic parts at the end. Using a trie (prefix tree) to organize URL patterns dramatically reduced matching time.

private static class Node { Map<String, Node> children; // child nodes List<T> patterns; // patterns matching this prefix }

Example trie diagram:

Routing algorithm:

Split the URL by “/” into tokens.

Traverse the trie from the root to find the longest matching path.

From the matched node’s patterns collection, iterate to find the final matching URL pattern.

After rewriting the routing logic with a trie, CPU usage dropped significantly.

Full GC

In 2017, the gateway began experiencing intermittent Full GC lasting 2‑3 seconds, causing brief load spikes.

Even though code and traffic were unchanged, heap dumps before and after Full GC showed that int[] and SimpleScalar objects were being collected.

Further analysis revealed many humongous objects (size ≥ half a G1 region) allocated in the old generation, which were quickly reclaimed before the dump, explaining the smaller dump size.

Monitoring and GC logs indicated that the large number of int arrays originated from Matcher and Pattern objects created during regex‑based routing.

Pattern compilation is expensive and creates many temporary objects. The previous implementation used Spring 3’s PathMatcher, which compiled a Pattern for each request.

Optimization focused on reducing Pattern compilation and Matcher creation. Spring 4’s AntPathMatcher caches Patterns and Matchers, disabling the cache only when it exceeds a threshold (default 65536).

After deploying the optimization, Full GC ceased.

Further Optimizations

Additional improvements include disabling unused suffix pattern matching and trailing slash matching in Spring, and moving towards prefix‑based routing similar to Nginx location blocks.

Proposed changes:

Each microservice defines its own routing rule; the gateway first matches based on these rules and forwards directly.

If no rule matches, fall back to the original traversal method for legacy APIs.

Clients can specify the target service name via a special header; the gateway forwards accordingly.

Conclusion

While code optimizations solved the PathMatcher issue, API naming also contributes to the problem. Long Restful paths with many PathVariables increase trie depth and pattern count, making routing costly.

The core bottleneck is regex matching, driven by Restful PathVariable usage. Reducing reliance on such patterns or converting them to request parameters can improve performance, though Restful URLs remain important for SEO.