Middle Mile

The middle mile is the network path between a CDN's edge and the origin. It also includes the hops between the CDN's own sites. Cloudflare's working definition is “the path from an origin server to any proxies or other network hops”. The middle mile is not the last mile. The same page calls the last mile “the final hop from the ISP to the user”. The middle mile is also not the origin server itself (Cloudflare blog).

The term has a second, wider sense, which is the original one. Akamai co-founder Tom Leighton used it in Communications of the ACM in February 2009, for the whole public path “between origin server and end user”. He called the name a misnomer “in that it refers to a heterogeneous infrastructure that is owned by many competing entities and typically spans hundreds or thousands of miles” (CACM). Both senses point at the same problem: the stretch of the path that nobody in particular owns. Nobody is paid to expand it. A CDN's answer is to shorten it, cache in front of it, or route around it.

The practical consequence is narrow but expensive. Content served from cache at the edge never crosses the middle mile at all. So the middle mile governs precisely the traffic the edge cannot serve: misses, revalidations, dynamic and personalised responses, and API calls.

How it works

Delivery is conventionally split into three legs. Cloudflare names the first as the first mile: “the path from an origin server to the data that you are requesting”. It names the second as the middle mile: the path to “any proxies or other network hops”. It names the third as the last mile: “the final hop from the ISP to the user”. Leighton puts the first mile more plainly. He calls it “origin infrastructure”.

The middle mile is crossed only when the edge cannot answer on its own. On a hit, the response leaves the nearest point of presence. No middle-mile traffic occurs on a hit. On a cache miss, a revalidation, or a response that is not cacheable, the edge forwards to the origin. Google documents the sequence for Cloud CDN: “When a cache miss occurs, the GFE forwards the request to the external Application Load Balancer. The load balancer then forwards the request to one of your origin servers” (Google Cloud docs).

Because it is metered separately from edge traffic, this leg has its own names on your bill. Google splits the two directions: “Data transfer from a cache to a client is called cache egress. Data transfer to a cache is called cache fill.” Middle-mile bytes are the cache-fill side of that meter. Akamai's word for traffic between its own servers is midgress. A cache hierarchy means “regular edge servers need to communicate with these servers during a request. This is what's called midgress traffic”. Once tiering is on, “traffic data is separated into regular edge traffic and midgress traffic” (Akamai TechDocs).

On that leg a CDN assembles three transport components and a cache.

• Its own backbone runs between its sites. This is the only part it genuinely owns.
• Peering happens at internet exchange points. It hands traffic straight to the other network. This avoids paying a carrier to move it.
• Paid transit covers everything it does not reach itself: other people's networks, bought by the bit.
• A mid-tier or shield cache consolidates what is left. With Origin Shield in use, AWS says: “all requests from all of CloudFront's caching layers to your origin go through Origin Shield, increasing the likelihood of a cache hit”. Requests that miss there are “consolidated with other requests for the same object, resulting in as few as one request going to your origin” (AWS docs). That is request coalescing applied to the middle mile. Cloudflare divides its data centres “into a hierarchy of lower-tiers and upper-tiers” so that “only the upper-tier can ask the origin for content”. This also “concentrates connections to origin servers so they come from a small number of data centers rather than the full set of network locations” (Cloudflare docs).

Two more things happen on this leg that are easy to miss. Connections are held open rather than rebuilt. Cloudflare pairs tiering with connection reuse, “which packages multiple requests into a single connection to your origin” (Cloudflare docs). Akamai keeps persistent connections to origin open by default. It ends an idle one after five minutes. It keeps them open because “establishing a TCP connection starts with a small congestion window”, and holding the established connection lets you “maintain the optimal flow” (Akamai TechDocs). This is why an indirect route can beat a direct one. Fastly notes shielding “reduces connection setup latency for MISS and PASS requests that must be served from origin (this feels counter-intuitive, but takes advantage of the fact that all Fastly POPs always have a pool of open connections to all other Fastly POPs)” (Fastly docs). The route itself can be chosen rather than inherited. BGP selects by policy, not by measured speed. Leighton calls it “simple and scalable but not very efficient or robust”. So an overlay with servers on many networks can measure candidates and relay through a faster one.

Why it matters for a CDN

The middle mile sets the performance ceiling for exactly the traffic a CDN cannot cache. Leighton's argument is that length itself is the defect, because the intervening peering points “are often overburdened, causing packet loss and service degradation”. So “the longer data must travel through the middle mile, the more it is subject to congestion, packet loss, and poor performance”.

The reason it stays bad is economic rather than technical. Money, he wrote, “flows into the networks from the first and last miles, as companies pay for hosting and end users pay for access”. In the middle, “there is very little incentive to build out capacity. If anything, networks want to minimize traffic coming into their networks that they don't get paid for.” The middle mile is a commercial boundary as much as a distance. It can fail as one: when Cogent and Telia de-peered over a business dispute in March 2008, for more than a week customers of each “could not reach certain Web sites at all”. Owning the middle mile is a way of buying out of that negotiation.

The cost is more than the added delay. This is because latency and throughput are coupled through TCP. RFC 7323 states that “TCP performance problems arise when the bandwidth * delay product is large”. It names such a path a “long, fat network” (LFN). On such a path, “the receive window limits the maximum throughput of the TCP connection over the path, i.e., the amount of unacknowledged data that TCP can send in order to keep the pipeline full”. It adds that “packet losses in an LFN can have a catastrophic effect on throughput” (RFC 7323, section 1.1). Leighton names the user-visible version the Fat File Paradox. It explains why a fat file is slow to cross the country “even if the network is not congested”. He gives the same mechanism: “latency and throughput are directly coupled”, so “throughput is effectively throttled by network round-trip time (latency)”. For magnitude, RFC 6349's worked example puts maximum achievable throughput on a 44.21 Mbps T3 with a fixed 64 KB window at 42.8 Mbps for a 10 ms round trip. It puts throughput at 20.5 Mbps for a 25 ms round trip (RFC 6349, section 3.3.1). This is one link and one fixed window, not a law, since modern stacks scale their windows. But the direction is structural: distance costs bandwidth, not only delay.

The gain from fixing it is real. But the widely quoted figure is a dated vendor claim. In that same 2009 article, Akamai stated that with a highly distributed network “you can actually speed up uncacheable communications by 30% to 50% or more, by using routes that are faster and much less congested”. Treat it as one vendor's measurement of its own overlay in one era, not a specification. The structural alternative is placement rather than routing: “a highly distributed CDN delivers content from the right side of the middle-mile bottlenecks, eliminating peering, connectivity, routing, and distance problems”. This means it serves from the user's side of them.

What CDNs do

Three different products are sold under this one heading. They are not interchangeable: carrying the traffic on the provider's own network, choosing a better path across other people's networks, and putting a cache tier in front of the origin.

• Google Cloud CDN uses its own network. This is not optional. The current Network Service Tiers documentation states that “Cloud CDN is always Premium Tier” and that “You cannot use Standard Tier with Cloud CDN”. Premium Tier “delivers traffic on Google's premium backbone” while Standard Tier “uses regular ISP networks” (Google Cloud docs). For an origin outside Google Cloud, reached through an internet network endpoint group, Google's claim is hedged. The far hop is still public. The benefit is to “Deliver traffic to your public endpoint across Google's private backbone, which improves reliability and can decrease latency between client and server”. Google “strongly recommend[s]” HTTPS or HTTP/2 to that backend, “so that communication between the load balancer and the backend is encrypted and authenticated when transiting the public internet” (Google Cloud docs).
• Akamai SureRoute is path selection, not private fibre. It “tests multiple routes between ​Akamai​ edge servers and your origin server to identify an optimal path for performance and establish alternative routes in cases of potential request failures”. A race is triggered by an end-user request for non-cacheable content. The race runs over three routes at once to fetch a small test object after the user has been served. The winner becomes the primary route until the next race. Race frequency and the lifetime of race results are configurable (Akamai TechDocs). It is aimed squarely at uncacheable content: “Requests for cacheable content don't trigger SureRoute races, as cacheable content is typically served directly from the ​Akamai​ edge server.” It is a per-property behaviour the customer adds. The races “cause a small increase in origin traffic”. The Performance optimization type “is available for selected ​Akamai​ products”.
• Cloudflare Argo Smart Routing is also path selection. It “detects real-time network issues and routes your web traffic across the most efficient network path, avoiding congestion”. The benefits “are most apparent for users farthest from your origin server” (Cloudflare docs). It does this by using “alternative network locations and providers” to put traffic on “a less direct, but faster, route”. It began as a pure middle-mile product: “When it launched, Argo was entirely focused on the ‘middle mile,’ speeding up connections from Cloudflare to our customers' servers”. It later added the client-to-Cloudflare leg (Cloudflare blog, September 2021). So an Argo figure today spans both senses of the term. It is a paid add-on, now offered as part of Smart Shield.
• Fastly Origin Connect is an actual private circuit: “a direct fiber connection between your origin servers and a Fastly shield POP thus reducing the number of organizations (and by association, the number of servers) handling your data” (Fastly docs). Note the shield POP. Note also how narrow eligibility is: servers in the same data center as that shield POP, Enterprise-level support, a publicly routed ASN, BGP peering provisioned on each interconnect, and a cross connect you pay the facility for. Fastly publishes no latency figure for it.
• Amazon CloudFront Origin Shield is a cache tier. It also draws the on-net boundary explicitly. This is the question worth asking any provider: “For origins in an AWS Region, CloudFront network traffic remains on the high throughput CloudFront network all the way to your origin. For origins outside of AWS, CloudFront network traffic remains on the CloudFront network all the way to Origin Shield, which has a low latency connection to your origin” (AWS docs). AWS also notes you “incur additional charges for using Origin Shield”.

Watch out for

• Two definitions. One is the CDN's edge-to-origin leg. The other is the whole public path between origin and end user. This is the sense in which the term was coined and called a misnomer. Vendor material and academic material use different ones. Say which you mean before comparing numbers.
• “Smart routing” is not private fibre. Argo and SureRoute measure and then choose among paths other networks own. Argo may deliberately take “a less direct, but faster, route”. A provider backbone or a cross connect is a physically different product with different failure modes. Know which you are buying.
• Where the origin sits decides how much is really on-net. AWS states the on-net path reaches the origin only for origins in an AWS Region. For origins outside AWS, it reaches Origin Shield instead. The remaining hop is then a separate connection. The same asymmetry applies to Google Cloud CDN with an off-Google origin, and to any “private backbone” pitch.
• A private leg is best-effort, not contractual. Fastly states: “In the event of Origin Connect service degradation, congestion, or a failure of one of these interconnects, public internet transit will be used for origin connectivity”. Fastly also states “There is no Service Level Agreement (SLA) available for Origin Connect.” Google is equally candid about Premium Tier. Where a user's network peers with it in several places, “Google Cloud doesn't guarantee that outbound traffic remains on the Google global network until it as close as possible to the internet user”. Plan for the day the traffic is back on the public internet.
• It does nothing for hits. Fastly is explicit: shielding “is only applicable to requests that are forwarded to a backend”. Shielding “is not involved when the request is handled entirely at the edge”. The cache hit ratio bounds the payoff before you spend anything.
• A cache tier aimed at uncacheable traffic makes things worse. Akamai states Tiered Distribution “is intended for cached objects”. If caching is off, you should disable it. Otherwise “a request is routed through a longer, Tiered Distribution-specific server path and then to your origin”, which “creates longer response times for your end users”. AWS agrees that Origin Shield “may not be a good fit” for “dynamic content that is proxied to the origin, content with low cacheability, or content that is infrequently requested”. AWS charges for it as “always an incremental layer” on dynamic requests. Route selection, not tiering, is the tool for uncacheable content.
• You cannot pre-warm the cost away. Google states: “cache fill happens only in response to a client-initiated request. You cannot preload caches except by causing the individual caches to respond to requests.” The first request at each cache location pays the middle mile. A wider footprint multiplies first misses rather than removing them. This is what mid-tier and shield layers exist to absorb.
• Vendor percentages are vendor measurements. Akamai's 30% to 50% figure is its own co-founder writing about its own overlay in 2009. Cloudflare's “average of 33% off HTTP time to first byte” is its own benchmark against a single origin in the central United States. Treat both as directional, not a forecast for your traffic.
• Shields have side effects. With Fastly shielding, your edge code “typically executes twice”: at the edge POP and again at the shield. This can cause redundant processing, duplicate headers or cache poisoning if you do not branch on execution context. Cloudflare warns that changing origin IPs or DNS records under Smart Tiered Cache may reassign upper tiers, “resulting in an increased MISS rate as cache is refilled in the new upper tiers”.
• It is billed in several places. Cache fill and midgress are metered separately from edge egress. Shield tiers add per-request charges. A private interconnect needs a cross connect you pay for. Route races add a little origin traffic. Fastly asks you to contact sales if “your traffic doesn't meet our minimum threshold for Origin Connect”. A private middle mile pays back only at volume.
• A good middle mile does not make you reachable. Routing failures break reachability wholesale, whoever's fibre you are on. Leighton's example is February 2008. Pakistan tried to block access to YouTube from within the country by broadcasting a more specific BGP route. This “accidentally caused a near-global YouTube blackout, underscoring the vulnerability of BGP to human error” (CACM). Private capacity on the middle mile buys performance, not immunity from the control plane.
• Availability is per plan and per product. Argo's inclusion in Smart Shield depends on package tier. SureRoute's Performance optimization type is offered only for selected Akamai products. Origin Shield exists only in AWS Regions that have a regional edge cache. Read the contract, not the product page.

Best practice

• Raise the cache hit ratio first. Every request served at the edge removes a middle-mile round trip outright. No routing product can match that.
• Put the tier or shield near the origin, not near your users. Let the provider measure. AWS advises enabling Origin Shield in the Region “that has the lowest latency to your origin”. AWS notes requests made in the same Region as the origin bypass it. Cloudflare's Smart Tiered Cache selects the upper tier by measured latency to the origin. This cannot work if your origin sits behind a cloud provider's anycast address. Set a cloud region hint in that case.
• Match the mechanism to the traffic. Akamai documents SureRoute for non-cacheable content and Tiered Distribution for cacheable content. Akamai states that if both apply to the same request, “they won't conflict”. Sustained high-volume origin traffic out of one facility is what justifies a private interconnect. A slow dynamic API is what justifies route selection.
• Leave edge-to-origin persistent connections on. Keep the origin's own idle timeout longer than the CDN's. This way the warm connection is not closed from the origin side.
• Measure the legs separately, per region. A client-side TTFB conflates all three. Compare edge TTFB on hits against misses to isolate the middle plus origin. Time the origin's own response at the origin to split its think-time from the network. Watch midgress or cache-fill volume next to origin volume, so you can confirm a shield traded the way you intended. Fixing the wrong leg is the common failure.
• For genuinely uncacheable traffic, trial route optimisation with the vendor's own before and after analytics on your own traffic rather than accepting a published percentage.
• Ask the questions the vendors' own documentation answers: which hops stay on their network for an origin in your location, what happens when the private path fails, whether there is an SLA, and what is billed as fill rather than egress. A claim of “control” that survives those questions is worth paying for. Run this same checklist per provider in a multi-CDN setup. Each CDN has its own middle mile.