Bandwidth

Bandwidth is the highest rate at which a network link can carry data. It is measured in bits per second: Mbps, Gbps. It is the link’s capacity: a ceiling you buy and plan against, not a promise about the traffic that actually flows. Bandwidth is not throughput. Throughput is the rate that really moves, and it always sits below the ceiling. Bandwidth is not latency either. Latency is how long a single trip takes. On a hosting or CDN invoice, the word almost always means something else again: the volume of data you moved that month. Bandwidth caps how much video a point of presence can serve. It does almost nothing for a page built from hundreds of small objects.

How it works

A link’s ceiling comes from its physical medium and from the interface rate you buy. It only means something once you say which layer you are measuring. RFC 5136 calls the physical figure the nominal physical link capacity: “the theoretical maximum amount of data that the link L can support”. An OC-3 link is capable of 155.520 Mbit/s. RFC 5136 is explicit that this value only provides an upper bound on capacity at the IP layer. Line coding and the framing bits that the physical and link layers add both eat into it. So a port sold as 1 Gbps never delivers 1 Gbps of IP payload. As RFC 5136 puts it, it is meaningless to speak of link capacity without qualifying exactly what is meant.

Next, the units trap. Rates are quoted in bits per second. Files and bills are counted in bytes. There are 8 bits in a byte, so 1 Gbps is at most 125 MB/s. Volumes use decimal SI multiples rather than binary ones. Fastly, for example, documents that in its calculations 1 gigabyte (GB) = 10⁹ bytes and 1 terabyte (TB) = 10¹² bytes. A decimal GB is about 7% smaller than a gibibyte (1 GiB = 2³⁰ bytes). So a “100 GB” allowance buys less than a file manager reporting “100 GB” would suggest.

A single connection rarely reaches the ceiling on its own. To keep a path full, a sender has to hold roughly bandwidth × RTT of unacknowledged data in flight. This is the bandwidth-delay product. RFC 7323 obsoletes the original RFC 1323, and it makes the consequence concrete. TCP reports its receive window in a 16-bit header field. So without the Window Scale option, the largest usable window is 64 KiB. As RFC 7323 states, “for LFN paths where the bandwidth * delay product exceeds 64 KiB, the receive window limits the maximum throughput of the TCP connection over the path” (an LFN being a long, fat network). If the window is smaller than the product, throughput is capped well below link capacity no matter how wide the link is. That is why a 10 Gbps link across an ocean does not by itself give you 10 Gbps.

That produces the split that matters in practice. Large transfers are bandwidth-bound and scale with the ceiling. Cloudflare notes bandwidth “is useful for downloading large files like operating system updates and big game updates”. Short transfers are latency-bound. A page assembled from hundreds of small, dependency-ordered objects finishes each one before the pipe is ever full. As Cloudflare puts it: “the protocols will use all the bandwidth available, but often complete a transfer before all the available bandwidth is consumed. It’s no wonder then that adding more bandwidth doesn’t speed up the page load, but better latency does.” The measurements Cloudflare cites put the point of diminishing returns for page load at about 20 Mbps. By contrast, every 200 ms of latency saved cuts more than a second off page load time.

Why it matters for a CDN

A CDN sizes bandwidth in two places. One is egress from the edge toward users. The other is the path that fills the edge from origin. Video dominates the first. Netflix recommends 3 Mbps or higher for 720p HD, 5 Mbps or higher for 1080p, and 15 Mbps or higher for 4K/UHD. So a 1 Gbps link “could stream more than 60 Netflix shows in 4K at the same time”. A PoP serving 10,000 concurrent 4K viewers needs at least 150 Gbps for video alone (10,000 × 15 Mbps, a floor, since the recommendation is “or higher”). At that scale, interconnection, not the server, is the constraint.

Bandwidth is also where the money goes. Cloudflare states that three costs together make up about 80% of its cost of goods sold: bandwidth, hardware and colocation. It also states that “first and largest is bandwidth: the traffic that traverses our network”. The lever on that number is to stop buying the traffic from third parties. An ISP can “choose to peer directly with Cloudflare and exchange traffic at no cost”, cutting out a paid transit provider. So CDNs peer aggressively at internet exchanges and take private interconnects once volume justifies it. Cloudflare will consider a PNI when a network exchanges more than 10 Gbps of peak traffic with it in a single location. Cloudflare currently supports N×100G LR4 ports and typically does not allow new 10G connections.

What CDNs do

CDNs differ far more in how they bill the traffic than in the physics of carrying it.

• Cloudflare does not charge CDN delivery per gigabyte. On the Free plan, bandwidth is unmetered. This works because that traffic rides off-cycle headroom: “we serve the traffic from across our network, wherever there is spare room.”
• AWS CloudFront offers two models. Pay-as-you-go “charges for data transfers out from its edge locations, along with HTTP or HTTPS requests”, on volume tiers that fall as usage grows and vary by region. Currently, for the United States, the first 1 TB each month is free, the next 9 TB is $0.085 per GB, and the rate falls to $0.020 per GB above 5 PB. Flat-rate plans instead bundle a monthly data-transfer allowance with no overage charges: 100 GB on the $0 plan and 50 TB on the $15, $200 and $1,000 plans. Data transferred from an AWS origin into CloudFront edge locations is free. So on CloudFront the origin-fill leg costs nothing when the origin is on AWS.
• Fastly charges “for egress traffic from our POPs”, billing each response as one request while “the response size in bytes is billed as bandwidth”. It bills the origin-facing leg too: bandwidth for traffic sent from the CDN to customers’ origins. Usage is recorded in bytes and presented in decimal GB.

Watch out for

• Bits, not bytes. Mbps and Gbps count bits. Divide by 8 for MB/s, so 1 Gbps is at most 125 MB/s. A billing GB is 10⁹ bytes, about 7% smaller than the 2³⁰-byte GiB your tooling may display.
• “Bandwidth” on an invoice is data transfer. Cloudflare’s own learning material is blunt that in web hosting the word really means data transfer: the bytes moved, not the width of the link. Hosts typically allot a monthly volume and charge on “either ingress (data going in) or egress (data going out), whichever is higher”. CDNs, by contrast, generally bill the egress side.
• Peak sets the transit bill, not average. IP transit is normally sold on 95th-percentile billing. The provider samples link utilisation in 5-minute bins across the month and bills the 95th percentile of those samples, forgiving the top 5% of bursts. A few busy hours therefore set the monthly rate. CAIDA’s study of a decade of transit data confirms it is the “de facto standard for transit providers”. It also finds the method imperfect: the 95th percentile “does not necessarily reflect the cost burden to the provider”.
• A wider pipe will not fix a slow site. If pages are many small, dependency-ordered requests, they are latency-bound. Buy round trips, not bandwidth.
• “Unmetered” still has rules, and they are wider than the Free plan. Cloudflare’s current Service-Specific Terms require every non-Enterprise customer, Free, Pro and Business, to use paid products such as Images or Stream “in order to serve video and other large files via the CDN”. The terms also reserve the right to limit CDN use for serving “a disproportionate percentage of pictures, audio files, or other large files”. Read the terms before treating unmetered as unlimited.
• A rated port speed is not deliverable IP capacity. Coding and framing overhead sit between the two. A capacity figure is only meaningful relative to the layer it was measured at.

Best practice

• Size edge egress and origin-fill capacity for peak concurrency rather than monthly average. Keep headroom for spikes and for the failover case where one PoP absorbs another’s load.
• Measure achievable throughput instead of trusting a rated figure. iperf3 is “a tool for active measurements of the maximum achievable bandwidth on IP networks”. The example below shows a nominal 1 Gbps path actually delivering about 938 Mbits/sec.
• Work out which regime you are in before buying capacity. If RTT dominates, spend on connection reuse, fewer round trips and closer PoPs. Add bandwidth only once throughput is genuinely the bottleneck.
• Raise cache hit ratio. Every byte served from the edge is a byte you do not pay to move from origin. That is the main lever on a data-transfer bill.
• When contracting, establish exactly what is billed: per byte, flat with an allowance, unmetered under an acceptable-use policy, or a billed percentile. Model your own spike profile against that rule before signing.