Boost Third‑Party API Performance in Kubernetes with Nginx Caching

This article discusses how to improve performance and stability of third‑party service access in Kubernetes by configuring Nginx as a caching gateway. The approach requires no code changes, but lacks write‑caching support and high‑availability for the cache.

Third‑party dependencies

Modern tech companies increasingly rely on external services as part of their application stack. While this accelerates development, failures or latency in those services can degrade reliability.

At Back Market, a product catalog is provided by a third‑party service. To ensure fast and reliable access, the team uses Nginx as a gateway proxy to cache all internal calls to the third‑party API.

Results

After a few days of operation, only about 1% of internal read requests needed to wait for the third‑party response. The cache status distribution over a week is shown below:

HIT : Valid response in cache – used directly.

STALE : Expired response – used while backend fetches fresh data.

UPDATING : Expired response with background update – used.

MISS : No cached response – synchronous call to third‑party.

Even when the third‑party was down for 12 hours, 96% of requests were served from cache, keeping end‑users unaffected. Direct calls to the Europe‑based API from the US were significantly slower than cached responses.

Below is the configuration and deployment guide for Nginx.

Nginx cache configuration

Key directives (see official Nginx documentation for details):

proxy_cache_path ... max_size=1g inactive=1w;

proxy_ignore_headers Cache-Control Expires Set-Cookie;

proxy_cache_valid 1m;

proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;

proxy_cache_background_update on;

The goal is to minimize read (GET) requests to the third‑party service.

If a response is missing, the request must wait for the third‑party, which we aim to avoid. This can happen when a URL has never been requested or when a cached entry expires (e.g., inactive=1w) or is evicted due to size limits ( max_size=1g).

When a cached response is older than one minute ( proxy_cache_valid 1m) and proxy_cache_background_update on is set, Nginx returns the stale response immediately and refreshes it in the background.

For URLs that return absolute self‑referencing links, we rewrite them using sub_filter and disable gzip because sub_filter does not work with gzipped responses:

proxy_set_header Accept-Encoding "";

Kubernetes deployment

The Nginx configuration is packaged in a non‑privileged container image and deployed like any other web application in the cluster. Environment variables replace hard‑coded values at runtime.

The gateway is exposed via a Kubernetes Service, and the number of gateway pods can scale dynamically. Since Nginx cache relies on a local filesystem, persistence across pod restarts requires special handling.

Persisting cache across non‑fixed pods

We synchronize the local cache directory with an S3 bucket. Each Nginx pod runs two sidecar containers sharing an /mnt/cache emptyDir volume:

An init container pulls the cached data from S3 before Nginx starts.

A sidecar container continuously syncs the local cache back to S3 every 10 minutes.

aws s3 sync s3://thirdparty-gateway-cache /mnt/cache/complete

while true; do
  sleep 600
  aws s3 sync /mnt/cache/complete s3://thirdparty-gateway-cache
done

We enable use_temp_path=on to avoid uploading partially written cache entries:

proxy_cache_path /mnt/cache/complete ... use_temp_path=on;
proxy_temp_path /mnt/cache/tmp;

Bucket lifecycle rules can automatically delete old entries (e.g., after 8 days).

<LifecycleConfiguration>
  <Rule>
    <ID>delete-old-entries</ID>
    <Status>Enabled</Status>
    <Expiration>
      <Days>8</Days>
    </Expiration>
  </Rule>
</LifecycleConfiguration>

Limitations

This solution works well for read‑heavy workloads where eventual consistency is acceptable. It does not support write requests (POST, DELETE) and cannot provide custom abstractions on top of the third‑party API.

Cache results are shared across all client services unless authentication information is included in the cache key. In our scenario this is acceptable because the data is publicly readable within the internal network.

Security considerations include ensuring the S3 bucket is private and only authorized personnel can access it. Centralized cache storage also means the cache is not highly available; future work may explore a primary/secondary cache architecture.

Source: Blog post by charlieroro (original link: https://www.cnblogs.com/charlieroro/p/17116296.html)