Hard · Media and streaming
Live streaming
Streamers broadcast live and thousands watch together, with a chat on the side. Everyone wants the same piece of video at the same second, so nothing can be saved for later. When a big streamer goes live, the whole audience arrives at once. Practise this hard system design problem for free with a Google or GitHub account.
The brief
Streamers broadcast live and thousands watch together, with a chat on the side. Everyone wants the same piece of video at the same second, so nothing can be saved for later. When a big streamer goes live, the whole audience arrives at once.
Think of a live platform like Twitch.
Users, scale and requirements
- Who uses it
- 1 million viewers a month, thousands live at once
- Audience in plain numbers
- 1,000,000 people per month
- Normal peak
- ≈ 3,200 requests per second
- Request mix
- Live segment: 70%; Message: 20%; Read: 6%; Sign-in: 2%; Write: 2%
- Speed for 99 of 100 requests
- Live segment: ≈ 1,500 ms; Message: ≈ 600 ms; Read: ≈ 400 ms; Sign-in: ≈ 1,000 ms; Write: ≈ 400 ms
- Successful requests
- 99.8%
- Data safety
- Best effort: small losses are tolerated.
- Monthly budget
- ≈ $270,000
- Data at rest
- ≈ 20 TB
- Where users are
- North America: 50%; Europe: 30%; Asia Pacific: 20%
- Home region
- North America
- Waiting deadlines
- Live segment: 900 seconds; Message: 900 seconds; Read: 900 seconds; Sign-in: 900 seconds; Write: 3,600 seconds
≈ marks an approximation, just as in the game. Waiting deadlines measure the age of queued work and are separate from interactive response time.
- Each live piece is 1 MB and covers 2 seconds.
- One live request stands for a 2-second piece of the broadcast, about 1 MB (approx.).
Questions that shape the design
How far behind real time may the video be?
About 3 to 5 seconds. Viewers tolerate that, but not freezes.
Is chat as important as the video?
Almost. Chat messages must arrive in a second or two, and a huge chat is mostly many people reading the same few messages.
What if a streamer is raided by another channel?
The audience can grow ten times in minutes. It will happen on the busiest night.
Do viewers need to sign in?
To chat, yes. To watch, no.
The traffic day
Consumer traffic, busiest in the evening. The clock covers 00:00 to 24:00 in the home region. Events add pressure on top of that normal traffic.
20:00 · Synchronized rush
Many callers send requests together instead of spreading them over time.
Traffic for Sign-in, Message, Read, Sync change reaches 10 times the scheduled load; rises over 10 seconds, holds for 60 seconds and eases over 300 seconds. Other request types keep their scheduled load.
20:30 · Flash crowd
A scheduled launch brings a sudden rush of requests.
Traffic reaches 8 times the scheduled load; rises over 30 seconds, holds for 900 seconds and eases over 1,800 seconds.
21:30 · Region outage
A whole cloud region goes down.
Duration: ≈ 1,200 seconds.
What a strong design has to handle
- Everyone wants the same piece of video at the same second. What does that mean for caching?
- Distribute live segments within a bounded delay and isolate chat pressure.
- A 5-second cache is long enough to merge thousands of identical requests and short enough to keep the stream live. A longer cache would make the video lag.
- Collapsing misses protects the origin but makes everyone wait if the origin is slow. That is the right trade for live video, because they all wait together.
Make a prediction, run a full traffic day, then compare the result. The in-game editorial opens after a finished day and includes measured reference designs.
Concepts to practise
Related problems
- Hard
Group chat and presence
Teams talk in channels with many people. Everyone sees messages in the same order, sees who is online and who is typing, and gets notified when mentioned. At nine every morning thousands of workplaces start at once, and one announcement can reach a channel of tens of thousands.
- Medium
Feature flag service
Companies turn features on and off for groups of users without redeploying. Millions of apps ask "is this on for me?", and the answer must be instant. A wrong flag value can take down an entire customer, so changes must roll out safely and be reversible in seconds.
- Easy
Photo sharing
You are building a photo app. People post pictures and everyone else scrolls through them. Almost all the traffic is people looking at photos, and the photos take up far more room than anything else you store.
- Easy
Podcast hosting
You host shows for creators. Listeners press play and download big audio files, and creators upload a new episode now and then. The audio is the bill: every episode downloaded costs bandwidth.