TCP Fast Open (TFO)

TCP Fast Open (TFO) is an experimental extension to TCP, defined in RFC 7413. It lets a client put application data in the very first SYN packet of a connection to a server it has visited before. The server can then hand that data to the application before the three-way handshake completes. The whole benefit is one round trip of connection-setup latency saved, and no more. It is not a new transport, not a faster network path, and not a security feature. TFO is ordinary TCP with one rule relaxed. It buys nothing on a first visit, nothing from a client that does not implement it, and nothing until an operator switches it on. RFC 7413 is published as an Experimental protocol, not a standards-track specification. Its practical reach has narrowed since 2014. Mainstream browsers no longer send TFO SYNs at all. Firefox removed its implementation in version 87 (Bug 1689604, landed February 2021). Chromium's TCP socket implementation carries no Fast Open code today. On a CDN edge it is therefore an optimisation for non-browser clients and proxy-to-origin hops, not for web page loads.

How it works

Standard TCP already allows data in a SYN but forbids the receiver from delivering that data to the application until the handshake completes. The handshake is what catches old and duplicate SYNs. TFO removes that one restriction and puts a Fast Open cookie in its place. The cookie is a message authentication code (MAC) tag generated by the server and opaque to the client. It authenticates the client's source IP address. It rides in its own TCP option, option kind 34. The option is 6 to 18 bytes overall, carrying a 4-to-16-byte cookie.

1. Earn a cookie. On an ordinary connection the client sends a Fast Open option whose Length field is 0. That is an empty cookie field. The server's SYN-ACK may return a cookie, if that listening port currently supports TFO. The client must cache it. It is advised to also cache the MSS the server advertised alongside it, and it keeps at most one cookie per client-IP and server-IP pair.
2. Spend it. On a later connection the client sends a SYN carrying the cookie plus application data. The data is sized to the cached MSS. With no cached MSS, it defaults to 536 bytes for IPv4 and 1220 for IPv6.
3. The server decides. If the cookie is valid, the server acknowledges the SYN and the data together. It delivers the data to the application. It may also send its response before the handshake finishes, with congestion control bounding how much. Otherwise it drops the data and acknowledges only the SYN sequence number.
4. Fall back cheaply. A refused cookie costs nothing. RFC 7413 notes there is no latency penalty when the server does not acknowledge the SYN data. The client retransmits the data in the first ACK. The exchange then continues as a normal three-way handshake.

Two limits decide whether the shortcut pays at all. The first application data unit must fit inside one segment minus the SYN's options. If it does not fit, the server waits for the remainder after the handshake anyway, and the saving evaporates. Nothing about TFO is automatic. RFC 7413 requires that TCP implementations MUST NOT use TFO by default. Implementations must use it only when an application asks, per service port. Each listening socket accordingly keeps two variables: FastOpenEnabled, off until the application turns it on, and PendingFastOpenRequests, the count of TFO connections in SYN-RCVD. If PendingFastOpenRequests passes a preset system limit, the server MUST disable TFO for new requests until it falls back. Cookies stay the server's to revoke. It can expire them at any time by rotating the key that generates them or encoding a timestamp in them.

Why it matters for a CDN

The saving is exactly one round trip to the edge. Its value scales with that round trip: near-nothing for a user milliseconds from a PoP, material on a long or mobile path. RFC 7413's own evidence is a single study of the Chrome browser over 28 days of global statistics. Chrome held idle persistent connections for 5 to 10 minutes, but the average connection carried only 3.3 transactions. The handshake accounted for up to 25% of HTTP transaction network latency. The authors estimated TFO would improve page load time by 10% to 40% on selected popular web sites. Those are one 2011 measurement, not a figure a CDN can promise.

A handshake keeps getting paid because connection reuse is imperfect. The same section puts the average across studies at 2 to 4 transactions per connection. It also records that Chrome still made 35% of HTTP requests on new TCP connections. Those numbers describe HTTP/1.1 with keep-alive. They predate HTTP/2, which multiplexes far more aggressively, so the figures do not carry over. But the structural point does: mobile power management, middlebox idle timeouts and capacity rebalancing all keep tearing connections down.

For TLS the saving does not double. RFC 7413 says it is safe and useful to include a TLS ClientHello in the SYN to save one RTT in the TLS handshake. That is the same single round trip, now spent starting the TLS exchange an RTT earlier rather than the HTTP request. The idea outlived the mechanism. QUIC builds the same capability into its own handshake as 0-RTT early data. RFC 9000 describes this as an option for clients to send data immediately, requiring some form of prior communication or configuration. It is the same bargain, and the same replay exposure.

What CDNs do

TFO lives in the kernel and the edge server, not on a CDN product surface. So it is an operator decision, and it varies. What the vendors actually document:

• Cloudflare documents that it supports TFO on user connections. That is the visitor-to-edge leg. With TFO enabled, Cloudflare can also send initial data in the SYN-ACK. RFC 7413 permits exactly that. The server MAY include data in the SYN-ACK when its response is readily available. That page does not describe TFO on the Cloudflare-to-origin leg.
• nginx takes fastopen=number on a listen directive. This both enables TFO for that socket and limits the queue of connections that have not yet completed the handshake. Its own warning: do not enable this feature unless the server can handle receiving the same SYN packet with data more than once.
• HAProxy uses two different keywords, and they are easy to conflate. tfo as a bind option enables it on a frontend's listening socket (Linux 3.7 or newer). tfo as a server or default-server option enables it toward backends (Linux 4.11 or newer). The configuration manual adds two cautions. The backend form needs retry-on with conn-failure, empty-response and response-timeout, or HAProxy cannot retry a failed connection. Many firewalls still refuse data on SYN packets, so enable it only once well tested. Frontend support dates from 1.5, backend support from 2.0.
• Linux gates everything behind the net.ipv4.tcp_fastopen bitmask. The client side (0x1) is on by default; the server side (0x2) is not. With the server bit set, each listener still opts in through the TCP_FASTOPEN socket option. Its value bounds the pending-SYN queue, much like the backlog argument to listen. Alternatively, setting 0x400 enables every listener at once. For the origin-facing direction, TCP_FASTOPEN_CONNECT (Linux 4.11) lets the edge act as a TFO client.
• A fleet behind one address must share cookie state. RFC 7413 says servers behind a load balancer accepting connections to the same server IP should use the same key, so they generate identical cookies for a given client. Otherwise the client collects a cookie the next machine cannot verify, and every Fast Open attempt falls back. On Linux that key is net.ipv4.tcp_fastopen_key. It takes a primary plus an optional backup key used only for validation, precisely so a fleet can rotate without invalidating cookies already cached by clients. This is not theoretical. A PowerDNS developer found that 8.8.8.8 issued TFO cookies but rejected the ones it had issued, because the anycast cluster did not share the cookie secret between nodes. An anycast edge fleet is exactly that shape. That makes shared keys the difference between TFO working and TFO never triggering.

Watch out for

• Replay is a protocol property, not a bug. Data in a SYN can reach the application more than once. So RFC 7413 states that a client or server that cannot handle receiving the same SYN data more than once MUST NOT enable TFO. It judges real duplication uncommon: measurements on a Tier-1 network found packet duplication rare. But the prohibition is normative regardless. Its worked example of what not to send is a POST without application-layer transaction protection, such as a unique identifier in the request header. It calls TFO particularly useful for GET.
• Paths that drop SYNs carrying data. This is the expensive failure, not the graceful one. The client learns nothing until the initial SYN timeout fires, and it retransmits a bare SYN. That costs a whole retransmission timeout, far more than the round trip TFO set out to save. RFC 7413 cited studies finding 6% of probed Internet paths dropped SYNs with data or unknown TCP options. It requires a client to cache such negative responses and stop trying Fast Open on that path. The picture has since improved enough that the reverse became the problem. In July 2021 the Linux maintainers disabled the kernel's TFO blackhole detection by default, because the logic was too aggressive and falsely triggered too often. Most middleboxes no longer drop TFO packets.
• There may be no client. Server-side TFO pays only if something sends Fast Open SYNs. Browser support has receded rather than grown. Count how much of your traffic actually arrives carrying a Fast Open option before crediting TFO with anything.
• Floods whose cookies validate. An attacker who harvests cookies from compromised hosts can flood SYNs that pass validation. Unlike a plain SYN flood, which mostly fills the listener queue, these reach the application. RFC 7413 notes they may exhaust server CPU and memory resources, surfacing in logs as an application-level DoS. The PendingFastOpenRequests limit is the defence. Past that limit the server disables TFO and downgrades new requests to regular connections, letting ordinary SYN flood defences such as SYN cookies take over. RSTs from the spoofed hosts fuel a TFO flood rather than defeating it. So a TFO server also has to watch connections being reset in SYN-RCVD.
• Amplified reflection. A cookie is valid for whoever holds that IP address. So an attacker who collects cookies while owning an address, and then releases it, can make many servers fire SYN-ACK-plus-data at whoever holds it next. RFC 7413's complete fix is for the server not to respond with data until the handshake finishes. That eliminates the amplification risk, but it also gives back the part of the saving that came from answering early.
• Client addresses that move. The cookie authenticates a source IP. So a changed address means a rejected cookie and a silent fall-back. Hosts sharing one NAT address get the same cookie, and TFO keeps working. But on carrier-grade NAT configurations where every new connection from one host uses a different public address, RFC 7413 says TFO provides no latency benefit. It adds that there is no penalty either. A shared NAT address is also how an attacker harvests cookies valid for its neighbours.
• Congestion control gets subtly weaker. When a SYN-ACK is lost, ordinary TCP shrinks the initial window before sending data. With TFO the server may already have sent an initial window. Across a fleet of mostly short connections, that blunts the response to loss. RFC 7413 flags this as a possible source of instability, and suggests Fast Open be temporarily disabled if a server sees many handshake losses. See congestion window.
• The first connection always pays. There is no cookie for a host never visited. So the opening connection runs a full handshake, and any benefit starts from the second.

Best practice

• Enable TFO only where the data that can land in a SYN is replay-safe. Keep state-changing requests out of the SYN. If you enable it toward origins, confirm the proxy is configured to retry a failed connection.
• On a Linux edge, add the server bit to net.ipv4.tcp_fastopen (0x2, giving 3 alongside the default client bit). Opt each listener in with a bounded TCP_FASTOPEN value, so valid-cookie SYNs cannot grow the pending queue without limit. Setting the sysctl alone does nothing.
• Pin one Fast Open key across every machine answering the same edge address. Rotate it through the backup-key slot, so cookies issued under the previous key still validate. A per-machine key silently reduces TFO to a plain handshake.
• Keep the first request inside one segment. A client sizes its SYN data from the MSS your edge advertises. Bloated request headers, or an unusually large advertised MSS, defeat the mechanism. RFC 7413 lets a client clamp its cached MSS to 1460 bytes for IPv4 or 1440 for IPv6, because an outsized SYN can trigger IP fragmentation and confuse middleboxes.
• If reflection risk outweighs the last round trip, withhold response data until the handshake completes: you keep the saving on the request leg and lose only the early answer.
• Verify from the second connection onward, and verify in production. Compare a repeat connection against a first. Count the SYNs arriving with a Fast Open option, and count the cookies that fail. A saving that shows up in a lab but not in production is usually a middlebox, a client that no longer implements TFO, or a request too large for one segment.