How Varnish Works: High‑Performance In‑Memory HTTP Caching Explained

What is Varnish

Varnish is a high‑performance open‑source reverse‑proxy and HTTP‑caching server. Its functionality is similar to Squid, but Varnish stores cached data in memory rather than on disk, which makes it faster while the cache is lost on server restart.

How Varnish Works

Initialization Process

The master process reads the configuration file, creates a storage space of the size specified by the administrator (e.g., 2 GB), and launches child processes. Child processes handle subsequent tasks and allocate threads for accepting HTTP requests, allocating cache space, cleaning expired objects, and performing space‑fragmentation cleanup.

HTTP Request Handling

A dedicated thread constantly listens on the request port. When a request arrives, it wakes a worker thread. The worker parses the request URI, checks the cache for the object, and if found returns it directly. If not, the request is forwarded to the backend server, the response is cached, and then returned to the client.

Cache Allocation

When an object needs to be cached, Varnish searches the free cache area for the smallest suitable free block. The object is placed there; any remaining space becomes a new free block. If no free space exists, Varnish evicts objects based on a least‑recently‑used policy to make room.

Cache Release

A dedicated thread periodically scans cached objects and removes those that have not been accessed within a configured lifetime, freeing their memory. After freeing space, the thread checks adjacent free blocks and merges them into larger blocks to reduce fragmentation.