The Streaming Pipeline
Every streaming service, from a Twitch broadcast to a Netflix movie, follows roughly the same chain of stages.
Encoding
Raw 1080p60 video is roughly 3 Gbit/s. Encoders compress it 300–1000× using intra-frame (spatial) and inter-frame (motion-compensated) prediction.
Packaging
Encoded streams are split into short segments (typically 2–6 s) and described by a manifest (.m3u8 or .mpd) that the player reads.
Delivery
Segments are plain HTTP objects, so CDNs can cache them at the edge and serve millions of viewers without hitting the origin.
Video Codecs
| Codec | Year | Efficiency vs H.264 | Notes |
|---|---|---|---|
| H.264 / AVC | 2003 | baseline | Universal hardware support; still the safe default. |
| H.265 / HEVC | 2013 | ~40–50% smaller | Common for 4K/HDR; complex patent licensing. |
| VP9 | 2013 | ~35–45% smaller | Royalty-free, widely used by YouTube. |
| AV1 | 2018 | ~50% smaller | Royalty-free (AOMedia); hardware decode spreading fast. |
| H.266 / VVC | 2020 | ~50% smaller than HEVC | Next generation; limited adoption so far. |
Frames come in three types: I-frames (self-contained keyframes), P-frames (predicted from previous frames) and B-frames (predicted from both directions). The distance between keyframes is the GOP size; segment boundaries must start on a keyframe.
Transport Protocols
RTMP
Legacy Flash-era protocol over TCP. Still the most common way to push a live stream from OBS to a platform.
SRT
Secure Reliable Transport over UDP with retransmission and encryption. Built for contribution over unreliable networks.
HLS
Apple's HTTP Live Streaming. Segments plus .m3u8 playlists; plays nearly everywhere. LL-HLS cuts latency to ~2–3 s.
MPEG-DASH
Open ISO standard, codec-agnostic, with an XML manifest (.mpd). Popular on Android and smart TVs.
WebRTC
Peer-to-peer real-time media over UDP with sub-second latency. Used for calls, cloud gaming and interactive live; WHIP/WHEP standardize ingest and playback.
CMAF
Common Media Application Format: one set of fragmented MP4 segments that serves both HLS and DASH, halving storage.
Example HLS master playlist
#EXTM3U #EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360,CODECS="avc1.4d401e,mp4a.40.2" 360p/index.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=2800000,RESOLUTION=1280x720,CODECS="avc1.4d401f,mp4a.40.2" 720p/index.m3u8 #EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2" 1080p/index.m3u8
Adaptive Bitrate (ABR)
The source is transcoded into a bitrate ladder: several renditions at different resolutions and bitrates. The player measures throughput and buffer level and switches renditions at segment boundaries, so playback continues smoothly as network conditions change.
| Rendition | Resolution | Typical H.264 bitrate |
|---|---|---|
| Low | 640×360 | 0.8 Mbit/s |
| SD | 854×480 | 1.4 Mbit/s |
| HD | 1280×720 | 2.8 Mbit/s |
| Full HD | 1920×1080 | 5.0 Mbit/s |
| 4K | 3840×2160 | 15–20 Mbit/s |
Typical FFmpeg command for one rendition of an HLS ladder:
ffmpeg -i input.mp4 -c:v libx264 -preset veryfast -b:v 2800k -maxrate 3000k \ -bufsize 5600k -vf scale=1280:720 -g 48 -keyint_min 48 -sc_threshold 0 \ -c:a aac -b:a 128k -f hls -hls_time 4 -hls_playlist_type vod 720p/index.m3u8
Latency Tiers
| Tier | Glass-to-glass | Typical tech | Use case |
|---|---|---|---|
| Standard | 15–45 s | HLS / DASH, 6 s segments | VOD, linear TV |
| Low | 2–5 s | LL-HLS, LL-DASH, chunked CMAF | Sports, live events |
| Ultra-low | < 1 s | WebRTC | Video calls, auctions, cloud gaming |
Latency comes from encoder look-ahead, segment duration, player buffer and network round trips. Shorter segments lower latency but raise request overhead and reduce CDN cache efficiency.