Varnish (now Vinyl Cache)

Varnish is an open-source HTTP caching reverse proxy. In the project's own words, it is a "web application accelerator". You put it in front of your origin. It answers most requests out of its own store before traffic ever reaches the backend. Its policy is not a config file of on/off switches. Instead it uses VCL, a domain-specific language. The daemon compiles VCL to binary code when you load it. The project started in 2005 for the Norwegian newspaper Verdens Gang. Version 1.0 shipped in September 2006. That is why so much cache vocabulary traces back to it.

It is worth being clear about what Varnish is not. It is a reverse proxy, not a forward proxy. It accelerates content you publish, not requests made on behalf of arbitrary clients. It is not a CDN either. It is one caching node you operate. It has no global footprint, anycast or DNS steering of its own. CDNs, though, are built out of nodes like it. And it does not terminate TLS. That is a deliberate design decision, so something else has to do it in front. Finally, since 2026 the name alone no longer tells you which software you have. The open-source project renamed itself Vinyl Cache. Varnish Software now ships a separate downstream build called Varnish Cache.

How it works

For each request, Varnish builds a hash and looks it up. By default the cache key is only the URL plus the Host header. It is the URL plus the server IP if there is no Host. The built-in VCL lowercases the host for you. If a fresh object is found, it is delivered without contacting the backend. If not, Varnish fetches from the origin, inserts the response into cache, and serves it.

Concurrent misses for the same object are coalesced. Only one backend fetch runs at a time. The other pending requests wait for its result. This is what stops a thundering herd hitting your origin when a popular object expires.

Freshness has three dials. TTL comes from the response by default. Varnish reads Cache-Control max-age to compute it, or you set it yourself as beresp.ttl. Grace lets Varnish serve the stale object after its TTL has expired. This happens while it refetches asynchronously in the background. This is exactly stale-while-revalidate semantics. Keep retains a stale object longer so it can be revalidated. Revalidation uses a conditional If-Modified-Since or If-None-Match request and gets a 304 answer. The values add up. Grace of 10 seconds plus keep of one minute means the object survives 70 seconds past its TTL.

VCL is where you write the policy. It is organised into subroutines that run at different stages: vcl_recv for the incoming request, vcl_hash for the cache key, vcl_backend_response for what the origin returned, and vcl_deliver for a last edit on the way out. Your file is appended to a built-in VCL. If you do not return an action, sensible defaults still apply. Loading transpiles the VCL to C, compiles it to a shared object and loads it into the running process. No restart is needed. Several VCLs can be loaded at once. vcl.use switches the active one instantly. Requests already in flight finish on the VCL they started with. Storage is pluggable ("stevedores"). The default is malloc, or umem where available. That default is virtual memory the operating system may page out to swap. The file backend is instead an mmap'd file on disk.

Why it matters for a CDN

Varnish is where the vocabulary of the programmable cache tier was worked out. Nearly every dial you set in it has a counterpart in a managed CDN under a different name. Those counterparts include cache key, TTL, grace or serve-stale, ESI fragment assembly, purge and ban invalidation. Fastly's VCL is a direct descendant of Varnish's, so the skill transfers literally rather than by analogy. It is also the thing you actually reach for when you build your own caching tier. That could be a self-hosted edge, or an origin shield that absorbs misses in front of a fragile application.

What CDNs do

• Fastly documents its configuration language as "a domain-specific programming language that evolved from the Varnish proxy cache". Every Fastly service runs on VCL, even when you only click in the web interface. Clicking in the interface just generates the VCL for you. Responses still carry a Via header naming varnish. But Fastly's VCL evolved from an early Varnish, so the dialects are not portable. Fastly's boilerplate uses vcl_fetch and "set req.hash +=". Vinyl Cache 9 instead uses vcl_backend_response and hash_data().
• Managed CDNs split edge freshness from browser freshness with a proprietary header. Fastly's version is Surrogate-Control. Fastly strips that header before the response reaches the client. This lets origin Cache-Control stay conservative. In Varnish you get the same separation by setting beresp.ttl in vcl_backend_response and leaving the client-facing Cache-Control alone.
• Varnish gives you cache behaviour as VCL you write yourself. A CDN gives you the same thing as a switch and an API. Fastly exposes "Serve stale content on origin failure" as a toggle. It also exposes purge-all and per-URL soft purge as API calls. The Varnish equivalents are grace, and PURGE requests plus regex bans. You have to build and protect those yourself.

Watch out for

• The name is now ambiguous. Vinyl Cache is the continuation of the old FOSS project, with the same maintainers. It was first released under the new name as 9.0 on 16 March 2026. Varnish Cache is now a downstream distribution governed by Varnish Software. Varnish Enterprise is that company's separate commercial product. Check the date on anything you read. Before late 2025, "Varnish Cache" means what is now Vinyl.
• The 9.0 rename touched more than the docs. The daemon is now called vinyld. The tools are vinyllog, vinylstat and vinylncsa. The default VCL directory moved from /etc/varnish to /etc/vinyl-cache. The Server and Via headers now say Vinyl-Cache instead of Varnish. X-Varnish became X-Vinyl. Monitoring, log parsers and VCL that match on those old names break silently.
• Uncacheable by default is usually your headers, not the engine. The built-in vcl_recv passes any standard method other than GET or HEAD. It also passes any request carrying a Cookie or Authorization header. The built-in vcl_backend_response marks a response uncacheable, called "hit-for-miss". This happens when the computed TTL is at or below zero, or when there is a Set-Cookie header. It then sets beresp.ttl to two minutes. So for those two minutes, every request goes to the backend with no coalescing. Read the origin's response before blaming Varnish.
• The default key is thinner than people assume. Cookies, device class or geo do not enter the hash unless you add them with hash_data(). Variants the backend declares with Vary must match byte for byte. So a missing space in an Accept-Language value creates a separate object. Vary: User-Agent will shred your hit ratio unless you normalise it first.
• No TLS. Varnish has never terminated TLS. The project has argued since 2011 that it should not, because doing so would mean fencing crypto into a separate process anyway. So a terminator sits in front. You accept its connections with the PROXY protocol on the listen socket. This is needed if you still want the real client IP in VCL.
• Sizing is not just the cache size. The size you give the malloc backend is a net figure. Because of allocator fragmentation, the real memory used can be two to four times higher. Varnish also needs roughly another 1KB per object outside the storage engine. Size for the gross figure or you will swap.

Best practice

• Set TTL and grace explicitly in vcl_backend_response instead of inheriting whatever the application happens to emit. Add keep if your origin can answer conditional requests with a 304. This saves you the body on revalidation.
• Make grace depend on origin health. Keep a long beresp.grace on the object. Then cap the effective grace with req.grace when std.healthy() says the backend is fine. This way a long stale window only opens when the origin is actually sick.
• Normalise before you hash. Canonicalise the host and sort the query string. Drop analytics-only cookies. Collapse the headers your backend varies on to a small set of values.
• Validate offline, then load, then switch. A VCL that fails to compile is rejected at load time and never becomes active. So the change that hurts you is the one that compiles and is wrong. Keep an emergency VCL loaded, and roll back with vcl.use on the previous name.
• Use purge when you know the exact object. Use a ban when you only have a pattern. Gate both behind an ACL so the internet cannot empty your cache.
• Know which release you are on and how long it lives. The project ships every six months. 9.0.1 is current, with end of life on 16 March 2027. 6.0.18 is the long-lived LTS and is still supported. 8.0 goes end of life on 15 September 2026.