Design a Video Streaming Service (YouTube / Netflix)

An encoding ladder is the set of (resolution, bitrate, codec) tuples generated for each source video. The ladder is the central economic decision in video.

Standard ladder (illustrative)

Codec choices

• H.264 (AVC): universal compatibility. ~40% bitrate of older codecs. Free for most uses.
• H.265 (HEVC): 50% more efficient than H.264 for same quality. Patent thickets - licensing $$.
• AV1: open, ~30% more efficient than H.265, but encoding cost is 5-10x.
• Most services use H.264 + H.265 + AV1 in tiers - H.264 universal, AV1 for the most-watched content where transcoding cost amortizes.

Per-title encoding (Netflix's innovation)
Different content compresses at different rates. A still-frame interview compresses far smaller than an action sequence at the same quality. Per-title encoding analyzes each video's complexity and produces a custom ladder, saving 20-30% of bandwidth.

Encoding chunks
Source video is split into 2-10 second chunks. Each chunk is independently encoded by a different worker. Embarrassingly parallel - 1 hour of video transcodes in 1 minute on 60 parallel workers.

Storage bloat
A 1-hour 4K source = 13 GB. With a 7-rung ladder, total encoded outputs = ~25 GB (the lower rungs are tiny). Aggressive lifecycle: low-watch tail content can drop to 3 rungs (mobile-friendly only) after a year, saving storage.

Adaptive bitrate (ABR) means the player switches between encoding ladder rungs as network conditions change. The two protocols are HLS (Apple) and DASH (everyone else).

Manifest structure
The manifest lists all available rungs and the URLs of their segment files. Player downloads the manifest, picks a starting rung based on initial bandwidth measurement, requests segments, monitors download time, switches rungs up or down.

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=240p
240p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3000000,RESOLUTION=720p
720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=6000000,RESOLUTION=1080p
1080p.m3u8

Segment design
Segments are typically 2-10 seconds long. Shorter = faster quality switching, more manifest churn. Longer = smoother playback, slower adaptation.

• 6-second segments are a common compromise.
• Live often uses 2-second segments to minimize end-to-end latency.

Switching algorithm
Player measures recent download throughput. If bandwidth > current rung × 1.5, step up. If bandwidth < current rung, step down (to avoid stalling).

• Naive throughput-based ABR is sensitive to network jitter.
• Buffer-based ABR (BOLA, MPC) considers buffer fullness - drop quality if buffer is shallow even if throughput briefly recovers.
• ML-based ABR (Pensieve) trains on network traces; outperforms heuristic ABR by 10-15%.

Initial rung selection
The first segment is the slowest to download because we don't know bandwidth yet. Start low (480p) for fast first-frame, ramp up after 1-2 segments. Trade-off: low first-frame quality vs slow startup.

Pre-fetching and CDN warming
Player fetches the next 1-2 segments speculatively. The CDN must serve them with low TTFB - a slow CDN miss kills perceived quality. Aggressive edge caching is essential.

Video delivery is a CDN problem. The transcoding pipeline produces outputs once; the CDN delivers them billions of times. CDN cost is the dominant operational expense.

Cache hit rate matters everything
At 99% hit rate, origin egress = 1% of total. Origin storage costs are bounded by content volume. CDN costs scale with viewers.

At 95% hit rate, origin egress = 5%. 5x the storage egress cost. Cache hit rate is a critical KPI.

Cache hierarchy

• Edge POPs (close to users): hot content. Tiny capacity per POP, lots of POPs.
• Mid-tier (regional): medium-warm content. Used as cache fill for edge.
• Origin (object storage): everything. Expensive to read; minimize hits.

Hot content is short
80% of views go to <1% of content. The hot working set fits in edge cache. Long-tail content takes a small bandwidth share but lives mostly in origin.

Cache key design
Each segment is keyed by (video_id, ladder_rung, segment_id). Identical content → same key globally. Personalized content (ads inserted) breaks caching - design ad insertion to be client-side or done at origin once per ad.

CDN strategy: build vs buy

• Buy: Akamai, Cloudflare, Fastly. Easy to start, expensive at exabyte-scale.
• Build: Netflix Open Connect, YouTube Edge Cache. Custom hardware in ISP networks. Saves 50-70% of CDN cost at scale.
• Hybrid: own POPs in primary markets + commercial CDN for long-tail geographies.

Multi-CDN
Many services use 2-3 CDN providers and route based on cost / performance / capacity. Multi-CDN offers redundancy (one CDN going down doesn't kill playback) and pricing leverage.

The interview signal
CDN cost is the dominant operational concern in video. Candidates who design without acknowledging CDN economics miss the central trade-off.

Live streaming shares the storage and CDN layers but the playback path is different.

Ingest
Broadcasters connect to a regional ingest endpoint via RTMP, SRT, or WebRTC. Ingest gateway authenticates, forwards to live transcoder.

Real-time transcoding
Live transcoder must keep up with input. 1 hour of live = 1 hour of transcode (real-time, not batch). GPU-accelerated; same encoding ladder concept as VOD but with tighter latency.

Segment generation
Live transcoder writes new segments every 2-6 seconds. Each segment is uploaded to a segment store (object storage with very short retention) and added to the live manifest.

Manifest update
Live manifests are periodically refreshed - players poll for the latest segment list. Usual cadence: every 2-3 seconds.

Latency sources (60 seconds total in standard live)

• Encoding: 5-10 seconds buffer for encoder lookahead.
• Segment buildup: 6 seconds (one segment).
• CDN cache fill: 1-3 seconds.
• Player buffer: 30-60 seconds (stability vs freshness trade-off).
• Total: 30-60 seconds end-to-end is typical.

Low-latency live (LL-HLS, DASH-LL, WebRTC)

• LL-HLS uses sub-second segments + chunked transfer encoding. End-to-end latency ~2-5 seconds.
• WebRTC: <1 second, but doesn't go through CDN (peer-to-peer / SFU). Different infrastructure.
• Trade-off: lower latency = more sensitive to network glitches = more rebuffers.

Live stream lifecycle

• Pre-stream: broadcaster authenticates, gets ingest endpoint URL.
• During: real-time pipeline runs, segments accumulate.
• End: stream is finalized, segments rolled into a VOD asset, transcoded again at higher quality (offline mode), made available as VOD.

Concurrent broadcasters at scale (Twitch)
At 100K+ concurrent broadcasters, the live transcoder pool dominates compute cost. Many broadcasters have very few viewers - cheap-encode for them; reserve high-quality encode for popular streams. Dynamic allocation based on viewer count.

Exabyte-scale storage at petabyte cost requires tiering.

Hot tier (last 30 days)
SSD-backed object storage. Read-optimized. ~$0.025/GB/month. Holds: recently uploaded videos, currently popular content.

Warm tier (30 days - 1 year)
HDD-backed object storage. ~$0.008/GB/month. Holds: most videos. Reads still serve in milliseconds.

Cold tier (>1 year)
Glacier / nearline. ~$0.001/GB/month. Reads take minutes. Holds: long-tail content with very low watch rate.

Re-tiering
Background process: track watch rate per video over rolling 30-day window. Move videos with watch rate < threshold to colder tiers. Move videos with sudden watch spike to hotter tiers (e.g., a 5-year-old clip goes viral).

Source vs ladder retention
Source video is the most expensive (4K = 13 GB/hour). Outputs are cheaper. Some services delete the source after 1 year and re-transcode from the highest-rung output if needed (loses some quality but saves PB).

Replication strategy

• Hot: 2-region replication for HA + DR.
• Warm: single-region with erasure coding (saves 30-50% over 3x replication).
• Cold: single-region. Restore time is part of the SLA.

Lifecycle automation
S3 / equivalent lifecycle policies. Encode the rules (after N days move to warmer/colder). Don't write your own lifecycle service - cloud providers do this well.

The economics
At YouTube-scale (1+ exabyte hot, 100s of EB warm/cold), storage tiering saves $100M+/year vs all-hot. This is one of the highest-leverage operational decisions in the system.

The video pipeline is necessary but not sufficient. Discovery is what keeps users engaged.

Search index
Title, description, tags, transcript indexed in ElasticSearch / equivalent. Updated on upload. Includes engagement signals (views, like ratio) for ranking.

Recommendations pipeline
Two-tower neural network common today. Inputs: user history, video metadata, contextual features. Outputs: top-K candidate videos. Re-ranked by online model with freshness, diversity, calibration.

Watch graph
User → video edges with weights (watch duration, engagement). Used both for collaborative filtering and for content discovery. Stored in graph DB or as sparse matrix in object storage.

Trending detection
Real-time: video gets X views in Y minutes → flag as trending. Needs streaming pipeline (Flink, Spark Streaming) consuming watch events.

Personalization vs editorial
Pure personalization creates filter bubbles. Most platforms blend personalized recommendations with editorial picks (trending, recommended-for-you mixed with what's hot globally).

Cold-start (new content)
A just-uploaded video has no watch signals. Bootstrap with: creator's prior history, content metadata (transcript, visual features from CV models), thumbnail click-through rate. After 1-10K views, use real engagement.

Cold-start (new users)
Newly registered. Show globally trending + a wide content sample. Lock in preferences quickly via implicit signals.

The interview signal
Designing the playback pipeline correctly without the discovery layer means designing a successful upload tool, not a successful product. Always note discovery as a critical adjacent system.