Easy · Developer platforms
Status page
When a big service has a problem, millions of people open its status page to see what is happening. The page must stay up exactly when everything else is down, and it must not share a home with the thing it reports on. A few staff update it by hand. Practise this easy system design problem for free with a Google or GitHub account.
The brief
When a big service has a problem, millions of people open its status page to see what is happening. The page must stay up exactly when everything else is down, and it must not share a home with the thing it reports on. A few staff update it by hand.
Think of a public status page.
Users, scale and requirements
- Who uses it
- 5 million visitors in a bad week (approx.)
- Audience in plain numbers
- 5,000,000 users or devices
- Normal peak
- ≈ 800 requests per second
- Request mix
- Read: 99%; Write: 0.5%; Sign-in: 0.5%
- Speed for 99 of 100 requests
- Read: ≈ 400 ms; Write: ≈ 400 ms; Sign-in: ≈ 1,000 ms
- Successful requests
- 99.9%
- Data safety
- At most 1 minute of writes may be lost.
- Monthly budget
- ≈ $3,500
- Data at rest
- ≈ 0.01 TB
- Where users are
- North America: 50%; Europe: 30%; Asia Pacific: 20%
- Home region
- North America
- Waiting deadlines
- Read: 900 seconds; Write: 3,600 seconds; Sign-in: 900 seconds
≈ marks an approximation, just as in the game. Waiting deadlines measure the age of queued work and are separate from interactive response time.
- The whole page is a few kilobytes of text (approx.).
- During a big outage reads can be 5 times the usual (approx.).
- All numbers are rough approximations.
Questions that shape the design
How fresh must the page be?
About 30 seconds behind is fine. Staff updates do not need to appear instantly.
Can it live next to the main service?
It should not. If the main region fails, the status page has to keep working.
How many people write to it?
A handful of staff. Almost all traffic is reading.
The traffic day
Fairly steady traffic throughout the day. The clock covers 00:00 to 24:00 in the home region. Events add pressure on top of that normal traffic.
14:00 · Flash crowd
A scheduled launch brings a sudden rush of requests.
Traffic reaches 5 times the scheduled load; rises over 30 seconds, holds for 900 seconds and eases over 1,800 seconds.
14:20 · Zone outage
An availability zone loses power or network access.
Duration: ≈ 3,600 seconds.
What a strong design has to handle
- Everyone reads the same page. How many of those reads must reach your own servers?
- A 30 second copy is slightly stale. Anything shorter costs more origin traffic with little gain.
- Hosting away from the main service means a second set of accounts to look after, but it survives the failure you are reporting.
Make a prediction, run a full traffic day, then compare the result. The in-game editorial opens after a finished day and includes measured reference designs.
Concepts to practise
Related problems
- Hard
Metrics platform
Thousands of services send a stream of measurements: CPU, errors, latency. You store them, draw the dashboards and fire the alerts. The platform watching everyone else must never go down first, and one customer creating millions of new series can ruin the bill.
- Hard
Maps and routing
You run a maps app with places, routes and live traffic. Map pictures never change much, but route lookups, place search and traffic from moving phones do. Soon a whole new continent will start using it. Keep maps fast everywhere without paying to serve the same picture twice.
- Medium
Feature flag service
Companies turn features on and off for groups of users without redeploying. Millions of apps ask "is this on for me?", and the answer must be instant. A wrong flag value can take down an entire customer, so changes must roll out safely and be reversible in seconds.
- Hard
Logs platform
Build a shared log service for 2,000 teams. Collectors send compressed chunks, and on-call engineers find logs by tenant, service and time before filtering their words. A wide search or a runaway label must not stop other teams from investigating an outage.