Manifest File

A manifest file is the table of contents of a stream. It is the first thing a player fetches, before any media. HLS calls it a playlist. A playlist is a UTF-8 text file. A client must be able to identify it either from a path ending in .m3u8 or .m3u, or from the content type application/vnd.apple.mpegurl or audio/mpegurl. DASH calls it a Media Presentation Description (MPD). An MPD is an XML document. It is normally served as .mpd with the content type application/dash+xml. A manifest is not the video. It carries no media. It lists the quality variants, the segment URLs, the codecs, the segment timing, and how to obtain the keys for protected content. A player that cannot load a valid manifest cannot start playback at all.

How it works

The player fetches the manifest first. Only then does it request the segments it names. RFC 8216, section 2 states the HLS rule: “To play this Playlist, the client first downloads it and then downloads and plays each Media Segment declared within it.” DASH works the same way: “In order to play the content, the DASH client first obtains the MPD”. By parsing the MPD, the client learns the timing of the program, the availability of media, the media types, resolutions and bandwidths, the DRM required, and the location of each media component on the network.

Both formats are two-level:

• An HLS master playlist lists the variants, one #EXT-X-STREAM-INF tag each. The URI line that follows the tag “specifies a Media Playlist that carries a Rendition of the Variant Stream” and “is REQUIRED”. Every such tag “MUST include the BANDWIDTH attribute” and “SHOULD include a CODECS attribute”. That lets a player tell, before playing, whether it can decode and afford a variant.
• An HLS media playlist lists the segments themselves. Each segment has an #EXTINF duration. A required #EXT-X-TARGETDURATION tag “specifies the maximum Media Segment duration”.
• A DASH MPD is one XML document. It has three levels: Periods (intervals of the program), Adaptation Sets (one per media component, such as video or one audio language), and Representations (that component’s encoded alternatives). Together these carry the timing, the quality alternatives and the segment addresses. Segment URLs may be listed individually, or “signaled using a template scheme resulting in a compact MPD”.

A minimal HLS master playlist is only three kinds of line:

• #EXTM3U: the format identifier, the first line of every playlist.
• #EXT-X-STREAM-INF:BANDWIDTH=2400000,RESOLUTION=1280x720: one variant, its peak segment bit rate and its resolution.
• 720p/playlist.m3u8: the URI line naming that variant’s media playlist. Repeat the pair per rendition. Inside that media playlist you then find #EXT-X-TARGETDURATION:10, and an #EXTINF:9.009, line, plus a segment URI for each segment.

Here is a full master playlist with three renditions:

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360
360p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2400000,RESOLUTION=1280x720
720p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=6000000,RESOLUTION=1920x1080
1080p/playlist.m3u8

Those URIs may be relative. The spec says “any relative URI is considered to be relative to the URI of the Playlist that contains it”. So the path a manifest is served from decides where its segment requests go.

The player chooses among the variants using measured throughput. A DASH client “monitors the bandwidth fluctuations of the network” and “decides how to adapt to the available bandwidth by fetching segments of different alternatives”. That is the mechanism behind adaptive bitrate playback. Encryption is announced in the manifest too: “Media Segments MAY be encrypted. The EXT-X-KEY tag specifies how to decrypt them”. The tag carries the METHOD and a URI “that specifies how to obtain the key”. It does not carry the key material itself.

VOD and live manifests then behave in opposite ways:

• VOD. The list is complete and frozen. An #EXT-X-PLAYLIST-TYPE tag “with a value of VOD indicates that the Playlist file MUST NOT change”. So it can be cached for hours or days.
• Live HLS. The server appends new segments and drops old ones as the DVR window slides. A playlist with no #EXT-X-ENDLIST tag must be republished with at least one new segment. The new segment must appear “no earlier than one-half the target duration… and no later than 1.5 times the target duration” after the previous version (RFC 8216, section 6.2.1). So the playlist is rewritten continuously. The RFC calls 10 seconds a typical target duration. Fastly reports that “typically, live stream transcoders are configured to generate 5s segments and manifests are refreshed after each new segment is available”. A client that reloads a changed playlist “MUST wait for at least the target duration before attempting to reload the Playlist file again”. It waits only one-half of that time if the playlist had not changed.
• Live DASH. An MPD instead states its own freshness. A nonzero MPD@minimumUpdatePeriod “defines the MPD validity duration of the present snapshot of the MPD”. Absence of the attribute “indicates an infinite validity (the MPD will never be updated)”. Where segment references are self-extending, clients “MAY obtain new segment references from such sequences even between MPD updates” (DASH-IF restricted timing model). So a live MPD need not be rewritten once per segment the way an HLS media playlist is. Read the manifest before assuming a TTL.

Why it matters for a CDN

The manifest is the first request of every playback session. So how the edge treats it sets startup latency, and decides whether playback is correct at all. Both protocols were designed for that traffic. HLS “offers compatibility with large-scale HTTP caching infrastructure to support delivery to large audiences”, and MPEG-DASH is a solution for streaming “using existing available HTTP infrastructure (particularly servers and CDNs, but also proxies, caches, etc.)”.

The difficulty is that one file type carries two opposite freshness requirements. A VOD playlist must never change. A live one is rewritten every few seconds. The difficulty is also that staleness has a deadline written into the protocol: a client “SHOULD assume that each Media Segment in it will become unavailable at the time that the Playlist file was loaded plus the duration of the Playlist file”. Serve a live manifest from cache past that point, and the segment URLs inside it are already gone. Cache it not at all, and every player start hits the origin. Hence the standard split: manifests and segments get separate cache policies, with the live media playlist on the shortest TTL.

The manifest is also where delivery topology lives. A DASH MPD may carry several BaseURLs, so “the same content can be available at multiple URLs, i.e. at different servers or CDNs, and the client can stream from any of them”. Multi-CDN steering is therefore a manifest decision, not only a DNS one. And because relative URIs resolve against the manifest’s own URI, any edge logic that rewrites manifest paths moves the segment traffic with it.

What CDNs do

• Akamai (Adaptive Media Delivery). Its Segmented Media Delivery Mode behavior tunes manifest TTLs by mode. In Live mode, “if no specific caching rules are defined… it assigns a very short TTL to the stream manifest file so that updates to it are retrieved in a timely manner”. Your own caching rules override it.
• CloudFront. The live-streaming guide says you “typically set up two cache behaviors for each endpoint”. One is for “the parent manifest, which is the index to your files” (path pattern *.m3u8 or *.mpd). The other is for “the segments” (*.ts or *.mp4). For each behavior it asks for a cache policy whose Minimum TTL is “set to 5 seconds or less, to help prevent serving stale content”. For LL-HLS it tells you to “specify _HLS_msn and _HLS_part as additional query string parameters for the cache policy that you use with the cache behavior for manifest requests”.
• Fastly. Its streaming configuration guidelines recommend an origin shield POP for live origins, and “setting manifest file TTLs to less than half of the video segment duration, typically 1-2 seconds for 5-second video segments”. Its sample VCL gives manifests (m3u8|mpd) a 1-second TTL and segments (aac|dash|m4s|mp4|ts) 3600 seconds. It also caches error responses for about a second.
• Cloudflare Stream. For its own managed streams, the player documentation takes the opposite line: “Manifests are dynamic assets that may be updated at any time. Do not cache, proxy, or store manifests; always read them directly from Stream”. That is because “outdated manifests may be missing features or refer to assets which have been moved”.

Watch out for

• Half-written playlists. “Changes to the Playlist file MUST be made atomically from the point of view of the clients, or playback errors MAY occur”. A partially written file that reaches cache breaks every player that gets it, for the whole TTL.
• Over-long live TTLs. The live window is deliberately short. A server “MUST NOT remove a Media Segment from a Playlist file without an EXT-X-ENDLIST tag if that would produce a Playlist whose duration is less than three times the target duration”. Once removed, a segment need only “remain available to clients for a period of time equal to the duration of the segment plus the duration of the longest Playlist file”. A manifest cached beyond that names segments the origin may already have deleted. Servers “SHOULD” also send an Expires header reflecting that planned time-to-live. An edge that honours origin headers will follow it.
• Query strings. Manifest URLs are often parameterised, so decide the cache key deliberately. Akamai’s Cache Key Query Parameters behavior defaults to “Exclude all parameters”. That means “all requests for the same URL are served from a single cache entry regardless of the query string in a request”. This is convenient for tokenised VOD URLs, but wrong when the parameter changes the response. CloudFront goes the other way. It asks you to allow the parameters that do change the response: m (MediaPackage’s modified-time tag, where “if content is already cached with a different value for this tag, CloudFront requests a new manifest instead of serving the cached version”), start and end for time-shifted viewing, aws.manifestfilter for manifest filtering, and _HLS_msn/_HLS_part for LL-HLS. Those last two are not in RFC 8216. Blocking playlist reload and the _HLS_ Delivery Directives come from HTTP Live Streaming 2nd Edition, still an Internet-Draft.
• Errors on live paths. Players routinely request segments that do not exist yet: “video players can make requests to segments not yet available or requests can return errors like 500 or 503 status codes”. Fastly’s advice is to make those responses cacheable, but only “with TTLs small enough to give sufficient time for origins to recover (around 1s)”.
• Compression. Manifests are text and compress well. “HTTP servers SHOULD transfer text files — such as Playlists and WebVTT segments — using the ‘gzip’ Content-Encoding if the client indicates that it is prepared to accept it.” Fastly enables it per manifest extension or content type.

Best practice

• Cache VOD manifests for hours or days. A VOD playlist “MUST NOT change”, so a long Cache-Control max-age is correct and cheap.
• Give live media playlists the shortest TTL in the configuration, below half the segment duration. Derive it from the target duration rather than guessing. The master playlist names only media playlists, not segments. So it carries none of that per-segment staleness risk, and it can take a longer TTL.
• For live DASH, take the TTL from the MPD’s own minimumUpdatePeriod instead of copying the HLS number.
• Split the caching by path or content type: one policy for manifests (.m3u8, .mpd), another for segments (.ts, .mp4, .m4s).
• Decide consciously which query parameters enter the manifest cache key. Include none for plain tokenised VOD. Include exactly the ones that vary the response for LL-HLS, time-shift, manifest filtering or per-session manifests.
• Cache transient 4xx/5xx responses on live paths for about a second, so a not-yet-published segment cannot stampede the origin.