Negative Caching
Negative caching stores an HTTP error response (typically 404 or 410, and 5xx where enabled) at the edge for a short, explicitly set TTL, so repeated requests for a broken or missing resource are answered from cache instead of the origin. It trades freshness for origin protection.
Also known as error caching.
Full Explanation
Negative caching is the deliberate storing of an HTTP error response at the edge. It stays there for a short, explicitly chosen time. Repeated requests for a broken or missing resource are then answered from the cache. They do not reach the origin again. It normally covers 404 and 410. It covers the 5xx family only where you switch it on. The trade is freshness for protection. While an error is cached, a fix at the origin stays invisible until that copy expires or is purged. It is not DNS negative caching. DNS negative caching is a resolver storing the knowledge that a name does not exist (RFC 2308 section 1). The two share a name and nothing else. It is not stale-if-error. Stale-if-error serves an older successful copy when the origin fails, instead of storing the new error. On several platforms that stale copy wins. There, negative caching only applies when there is nothing stale to serve. And it is not an outage remedy. It shields the origin during an incident. It does not shorten one.
How it works
A cache may store a response only under three conditions. The status code must be final. no-store must be absent. The response must carry at least one storage trigger. A storage trigger is one of these: an Expires header, a Cache-Control max-age or s-maxage directive, a public directive, a cache extension, or a status code that HTTP defines as heuristically cacheable (RFC 9111 section 3). Among the error statuses, only 404, 405, 410, 414 and 501 are heuristically cacheable. 500, 502, 503 and 504 are not. So a bare 5xx arrives with no license to be stored at all. RFC 9110 section 15.1 gives that list. It also lists the successful and redirect codes 200, 203, 204, 206, 300, 301 and 308.
Negative caching depends on one rule. Any response with a final status code can be cached if it is given explicit freshness information (RFC 9110 section 16.2.2). An error rarely carries freshness of its own. So the CDN supplies it: a short, fixed TTL assigned per status code. That assigned TTL is what turns an error response into a cacheable object.
- The origin returns an error, say a 404, for a URL.
- The edge stores that response under the URL's normal cache key. It stamps the response with the TTL configured for that status.
- While the stored error is fresh, later requests for the same URL are satisfied from it. The origin is not contacted (RFC 9111 section 4.2).
- When the TTL expires, the next request goes forward again to see whether the resource works now.
Without an assigned TTL, the error is simply passed through. Every repeat is another miss. The cache has to forward each one to the origin (RFC 9111 section 4.1).
Why it matters for a CDN
A CDN funnels a whole region's traffic through a small set of edge nodes. So identical requests arrive in bulk. A popular resource might break, or a crawler might walk thousands of nonexistent paths. Each of those requests is a miss. It reaches the origin at the moment the origin is least able to answer. Request collapsing only merges the misses that arrive simultaneously for one object. The sequential repeats that follow are exactly what negative caching absorbs. One cheap, short-lived copy of the error per edge changes the math. The origin sees roughly one request per URL per TTL window, instead of every repeat. A scanner hammering a dead URL adds no origin load at all.
What CDNs do
Defaults differ sharply. Several status codes are opt-in. The scope of "cached by default" differs too. Confirm the behavior on the platform you run, rather than assuming it.
- Cloudflare gives 404 and 410 an Edge Cache TTL of 3 minutes when the origin sends no cache-control or expires header. It caches no other error status by default. (200, 206 and 301 get 120 minutes; 302 and 303 get 20 minutes.) That default only applies where Cloudflare caches at all. Cloudflare caches by file extension. HTML and JSON are not cached by default. So a 404 on a page or an API path is not negative-cached until a Cache Rule covers it. Per-status TTLs come from the Edge TTL "Status Code TTL" setting in a Cache Rule, or from cacheTtlByStatus on a Worker fetch. There, 0 expires the entry immediately, and a negative value means do not cache.
- Amazon CloudFront caches a fixed list of error statuses: 404, 414, 500, 501, 502, 503 and 504. The default error-caching duration is 10 seconds. It is adjustable per status as the Error Caching Minimum TTL. 400, 403, 405, 412 and 415 are cached only when the origin sends Cache-Control max-age or s-maxage. An origin max-age, s-maxage or Expires wins when it is greater than that minimum. no-store, no-cache and private are honored on 404, 410, 414 and 501. They are ignored for every other error code. 416 is never cached. With an Amazon S3 origin, an error-caching TTL of 0 is still floored at 1 second, to protect the bucket.
- Akamai caches 204, 305, 404, 405 and 501 for 10 seconds by default. 500, 502, 503 and 504 are cached only when the Cache HTTP Error Responses behavior is enabled. That behavior is also where the time is changed. It does nothing if Caching is set to no-store. Negative-cached responses are capped at 64 KB by default. That cap is the usual reason a 404 is not cached. Since Q2 2023, a 400 is not cached from origin unless the property defines the variable PMUSER_AK_INT_CACHE_400 with the value true.
- Fastly treats only 200, 203, 300, 301, 302, 404 and 410 as cacheable by status. So 404 and 410 are negative-cached with the service's fallback TTL. A 5xx is not stored at all. A 500 carrying Cache-Control max-age=300 is not cacheable, because of its status code. That 300-second TTL is simply ignored. Caching a 5xx means overriding beresp.cacheable in VCL.
- In the origin-side tier a CDN sits in front of, nginx sets the time per code with proxy_cache_valid. For example: proxy_cache_valid 404 1m; and proxy_cache_valid 500 502 503 5s;. Error codes have to be named explicitly, because giving only a time caches nothing but 200, 301 and 302.
- Varnish exposes beresp.status and beresp.ttl for writing in vcl_backend_response. That is the one place where object attributes can still be changed before the object is stored. So an error is given its own short TTL there. When the origin sets no s-maxage, max-age or Expires, beresp.ttl falls back to the default_ttl parameter. That default is two minutes out of the box, far longer than an error should live.
Watch out for
- Serving stale usually beats caching a fresh error on 5xx. The precedence is vendor-specific. Akamai edge servers keep and serve stale cached objects on 500, 502, 503 and 504 by default (the Preserve Stale Objects option). You have to disable that option to negative-cache the error instead. On Cloudflare, stale-if-error triggers only on 5xx. A 404 is not an error for that purpose. So the fresh 404 replaces the stale copy. Work out which mechanism is answering before tuning either.
- A long error TTL can outlive the fault. CloudFront's own warning: with a long error-caching duration it "might continue to respond to requests with an error response or your custom error page for a long time after the object becomes available again".
- Too short a TTL on a 5xx forwards more requests to an origin that is already failing. In CloudFront's words, it "might aggravate the problem that originally caused your origin to return an error".
- The cached error hides origin state from monitoring. While it lives, dashboards and uptime checks that sample the CDN keep reporting the error. They do so even after the origin has recovered. Monitor the origin directly. Know when the edge copy expires.
- One cached error exists per URL. It is an ordinary cache entry under its own cache key. So clearing it is per object. CloudFront's guidance: if the origin returned errors for several objects, each one is invalidated separately.
- A cached error can entrench an attack. Akamai's documentation warns that a request containing a header with an invalid character may be passed to the origin. The resulting error response may be cached at the edge as if it were valid. So a poisoning payload is served for the whole TTL. WAF rules that reject invalid characters are the defense.
- Private errors can leak through a shared cache. A shared cache must not reuse the response to a request that carried an Authorization header, unless the response explicitly permits it (RFC 9111 section 3.5). A Set-Cookie header does not inhibit caching, though (RFC 9111 section 7.3). So a session-derived 403 can be stored and served to everyone, unless the origin marks it no-store.
- Vendor documentation can contradict itself. CloudFront's error-caching page names 404, 410, 414 and 501 as the codes where no-store is honored. Its list of cached status codes is different: 404, 414, 500, 501, 502, 503 and 504, with no 410 in it. Check the response headers on a real request. Do not just trust a table.
Best practice
- Keep error TTLs short and class-dependent. Use seconds for a transient 5xx. Use longer TTLs for a 404 or 410 that will not come back soon. The vendor defaults are a fair anchor: 10 seconds for errors at CloudFront and Akamai, 3 minutes for 404 and 410 at Cloudflare.
- Set the error TTL explicitly per status code. Never let an error inherit a general content default, such as Varnish's two-minute default_ttl or a CDN fallback TTL. That is how a cached error outlives the incident that produced it.
- Check first whether the error path is cached at all on your platform. Cloudflare's extension-based default leaves HTML and JSON errors uncached until a Cache Rule covers them. Fastly and Akamai both need an explicit opt-in before a 5xx is stored.
- Purge the cached error, per URL, the moment the fix ships. Do not wait out the TTL.
- Never negative-cache 401, 403 or 429. Also never negative-cache any error body carrying user or internal detail. Mark those no-store at the origin, since no-store bars storage outright (RFC 9111 section 3).
- Decide the 5xx precedence between serving stale and caching the error deliberately. Let negative caching be the fallback for when no stale copy exists.
- Treat the error TTL as a shield, not a recovery path. Health checks, origin failover and alerting must not depend on a cached error going away.
Examples
Set CloudFront negative caching with Terraform:
resource "aws_cloudfront_distribution" "cdn" {
custom_error_response {
error_code = 404
error_caching_min_ttl = 10 # seconds
}
custom_error_response {
error_code = 500
error_caching_min_ttl = 5
}
custom_error_response {
error_code = 502
error_caching_min_ttl = 5
}
}Cache errors in Nginx:
# Cache 404s for 10 seconds, 500s for 5 seconds
proxy_cache_valid 404 10s;
proxy_cache_valid 500 502 503 5s;
# Or in Varnish
sub vcl_backend_response {
if (beresp.status == 404) {
set beresp.ttl = 10s;
} else if (beresp.status >= 500) {
set beresp.ttl = 5s;
}
}Frequently Asked Questions
Negative caching stores an HTTP error response (typically 404 or 410, and 5xx where enabled) at the edge for a short, explicitly set TTL, so repeated requests for a broken or missing resource are answered from cache instead of the origin. It trades freshness for origin protection.
Set CloudFront negative caching with Terraform:
resource "aws_cloudfront_distribution" "cdn" {
custom_error_response {
error_code = 404
error_caching_min_ttl = 10 # seconds
}
custom_error_response {
error_code = 500
error_caching_min_ttl = 5
}
custom_error_response {
error_code = 502
error_caching_min_ttl = 5
}
}Cache errors in Nginx:
# Cache 404s for 10 seconds, 500s for 5 seconds
proxy_cache_valid 404 10s;
proxy_cache_valid 500 502 503 5s;
# Or in Varnish
sub vcl_backend_response {
if (beresp.status == 404) {
set beresp.ttl = 10s;
} else if (beresp.status >= 500) {
set beresp.ttl = 5s;
}
}Yes. Negative Caching is also known as error caching. Negative caching stores an HTTP error response (typically 404 or 410, and 5xx where enabled) at the edge for a short, explicitly set TTL, so repeated requests for a broken or missing resource are answered from cache instead of the origin. It trades freshness for origin protection.
Related CDN concepts include:
- max-age — max-age is a Cache-Control response directive giving the seconds a stored response may be reused …