Master Varnish: Step-by-Step Installation and Configuration Guide

1. Installing Varnish

Prepare the environment on two CentOS 5.4 hosts (Varnish‑server 192.168.12.246 and Web‑server 192.168.12.26). Create a dedicated varnish user and group, then create cache and log directories:

useradd -s /sbin/nologin varnish
mkdir /data/varnish/cache
mkdir /data/varnish/log
chown -R varnish:varnish /data/varnish/cache
chown -R varnish:varnish /data/varnish/log

Download Varnish 2.1.2 from the official site, then install PCRE (required for regular‑expression support):

tar zxvf pcre-7.9.tar.gz
cd pcre-7.9/
./configure --prefix=/usr/local/pcre/
make && make install

Extract and compile Varnish:

tar -zxvf varnish-2.1.2.tar.gz
cd varnish-2.1.2
export PKG_CONFIG_PATH=/usr/local/pcre/lib/pkgconfig
./configure --prefix=/usr/local/varnish \
    --enable-dependency-tracking \
    --enable-debugging-symbols \
    --enable-developer-warnings
make
make install
cp redhat/varnish.initrc /etc/init.d/varnish
cp redhat/varnish.sysconfig /etc/sysconfig/varnish

Installation is now complete.

2. Configuring Varnish

VCL (Varnish Configuration Language) defines request handling policies. It resembles C/perl syntax, using operators such as =, ==, !, &&, and regular‑expression matching with ~. VCL is not a full programming language—it lacks loops and user‑defined variables.

Key built‑in VCL functions include:

vcl_recv : entry point for incoming requests; decides Pass, Pipe, or Lookup.

vcl_pipe : forwards the request unchanged to the backend.

vcl_pass : fetches from backend without caching.

lookup : searches the cache and dispatches to vcl_hit or vcl_miss.

vcl_hit : serves cached content.

vcl_miss : fetches missing objects from backend.

vcl_fetch : decides whether to store the fetched object.

vcl_deliver : final processing before sending to client.

vcl_timeout and vcl_discard : handle expiration.

Common VCL built‑in variables are listed in tables (e.g., req.backend, client.ip, beresp.ttl, obj.cacheable, resp.status).

The following flowchart illustrates the Varnish request processing pipeline:

3. Sample VCL Configuration (Varnish 2.1.2)

Save the default file as /usr/local/varnish/etc/varnish/vcl.conf and include the following sections:

backend webserver {
    .host = "192.168.12.26";
    .port = "80";
}

sub vcl_recv {
    if (req.http.x-forwarded-for) {
        set req.http.X-Forwarded-For = req.http.X-Forwarded-For ", " client.ip;
    } else {
        set req.http.X-Forwarded-For = client.ip;
    }
    if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" &&
        req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" &&
        req.request != "DELETE") {
        return (pipe);
    }
    if (req.request != "GET" && req.request != "HEAD") {
        return (pass);
    }
    if (req.http.host ~ "^(.*).ixdba.net" || req.http.host ~ "^(.*).ixdba.cn") {
        set req.backend = webserver;
    }
    if (req.url ~ "\\.(jsp|do)($|\\?)") {
        return (pass);
    } else {
        return (lookup);
    }
}

sub vcl_pipe { return (pipe); }
sub vcl_pass { return (pass); }

sub vcl_hash {
    set req.hash += req.url;
    if (req.http.host) {
        set req.hash += req.http.host;
    } else {
        set req.hash += server.ip;
    }
    return (hash);
}

sub vcl_hit {
    if (!obj.cacheable) { return (pass); }
    return (deliver);
}

sub vcl_miss { return (fetch); }

sub vcl_fetch {
    if (!beresp.cacheable) { return (pass); }
    if (beresp.http.Set-Cookie) { return (pass); }
    if (req.url ~ "^/servlet/") { return (pass); }
    if (req.url ~ "^/services/") { return (pass); }
    if (req.request == "GET" && req.url ~ "^/upload(.*)$") { set beresp.ttl = 300s; }
    if (req.request == "GET" && req.url ~ "\\.(png|xsl|xml|pdf|ppt|doc|docx|chm|rar|zip|bmp|jpeg|swf|ico|mp3|mp4|rmvb|ogg|mov|avi|txt|gif|jpg|css|js|html|htm)$") {
        set beresp.ttl = 600s;
    }
    return (deliver);
}

sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT from www.ixdba.net";
    } else {
        set resp.http.X-Cache = "MISS from www.ixdba.net";
    }
    return (deliver);
}

This configuration sets up a backend, handles forwarding, defines caching rules for static assets, and adds an X-Cache header to indicate hit or miss status.