Mastering Varnish: Boost Web Performance with Caching and Load Balancing

1. Introduction

Web caching stores resources such as HTML, JS, CSS, and images on both the web server and the client browser. When a request arrives, Varnish decides whether to serve the content from its local cache or forward the request to the backend server based on expiration and content changes. Unlike the memory‑based cache memcached, Varnish offers high‑speed caching and is widely adopted by large‑scale websites, reducing backend pressure and enhancing user experience.

2. Varnish Architecture and Configuration

Architecture: Varnish consists of a management process and one or more child processes. The management process provides a command‑line interface, compiles VCL, monitors health, and controls child processes. Child processes handle worker threads, generate cache logs, and manage cache expiration.

VCL (Varnish Configuration Language): VCL is a state‑engine based language where each engine uses return to exit the current state and transition to the next. Engines are isolated but can interact through defined states.

Request Flow: Requests are classified as cacheable or non‑cacheable. Cacheable requests may hit the cache; a hit returns the cached response, while a miss forwards the request to the backend, caches the result, and then serves it. Non‑cacheable requests are passed directly to the backend.

Below are diagrams illustrating the architecture, VCL processing flow, and request flow.

3. Varnish Load Balancing and Static‑Dynamic Separation

Experiment Environment: Two backend web servers (websrv1 and websrv2) are used. When implementing load balancing, the path must be set to bypass caching and retrieve data directly from the backend.

After configuring VCL, use varnish_reload_vcl or varnishadm to compile and apply the configuration. The load‑balancing diagram shows how requests are distributed between the two backends.

Static‑Dynamic Separation: Configuration examples demonstrate routing dynamic pages to websrv1 and static pages to websrv2, achieving effective separation of content types.

Testing diagrams confirm that dynamic pages are served from websrv1 and static pages from websrv2, validating the separation strategy.

Conclusion: Varnish caches content by hashing URLs. It intercepts user requests, stores responses on the Varnish server, and serves cached data when it is still valid, dramatically reducing backend server load and improving response times.