Hard · Commerce and retail
Black Friday marketplace
Once a year a marketplace puts thousands of deals live at the same minute. Millions browse and search, a few doorbuster items get hammered, and a smaller crowd checks out. A payment must never be taken twice, and the day must survive the card processor having a bad hour. Practise this hard system design problem for free with a Google or GitHub account.
The brief
Once a year a marketplace puts thousands of deals live at the same minute. Millions browse and search, a few doorbuster items get hammered, and a smaller crowd checks out. A payment must never be taken twice, and the day must survive the card processor having a bad hour.
Think of the biggest sale day of an online marketplace like Amazon.
Users, scale and requirements
- Who uses it
- 30 million shoppers on the day (approx.)
- Audience in plain numbers
- 30,000,000 users or devices
- Normal peak
- ≈ 20,000 requests per second
- Request mix
- Read: 55%; Search: 15%; Write: 15%; Payment: 2.5%; Sign-in: 10%; Notification: 2.5%
- Speed for 99 of 100 requests
- Read: ≈ 500 ms; Search: ≈ 700 ms; Write: ≈ 500 ms; Payment: ≈ 4,000 ms; Sign-in: ≈ 1,000 ms
- Successful requests
- 99.9%
- Data safety
- No acknowledged write may be lost.
- Monthly budget
- ≈ $44,000
- Data at rest
- ≈ 20 TB
- Where users are
- North America: 60%; Europe: 30%; Asia Pacific: 10%
- Home region
- North America
- Waiting deadlines
- Read: 900 seconds; Search: 900 seconds; Write: 3,600 seconds; Payment: 3,600 seconds; Sign-in: 900 seconds; Notification: 900 seconds
- Required proofs
- Managed encryption keys; Audit trail
≈ marks an approximation, just as in the game. Waiting deadlines measure the age of queued work and are separate from interactive response time.
- Use the peak and mix to estimate payment and search rates separately.
- The card processor is paid from revenue and is not in this budget.
- All numbers are rough approximations.
- Notification speed measures provider acceptance. Device receipt is outside this traffic model; check expiry, provider outcome and retries in the correctness trace before claiming delivery guarantees.
Questions that shape the design
Can search results and product pages be a little old?
Yes, by a minute. A changed price is checked again at checkout.
Can a shopper be charged twice?
Never. Phones retry when the network is slow, so every payment needs one unique key.
What if the card processor is down?
Orders must still be accepted and paid when it returns. A shopper should see "order received", not an error.
Do all items sell equally?
No. A handful of doorbusters take a large share of the taps and the stock counts.
The traffic day
Consumer traffic, busiest in the evening. The clock covers 00:00 to 24:00 in the home region. Events add pressure on top of that normal traffic.
06:00 · Flash crowd
A scheduled launch brings a sudden rush of requests.
Traffic reaches 3 times the scheduled load; rises over 30 seconds, holds for 900 seconds and eases over 1,800 seconds.
06:10 · Hot key
One item gets a huge share of traffic: a celebrity profile, one product, one merchant.
Affected share: 30%. Duration: ≈ 3,600 seconds.
13:00 · Duplicate delivery
The queue redelivers messages. Without duplicate protection, side effects may be applied twice.
Affected share: 1%. Duration: ≈ 3,600 seconds.
19:30 · Dependency outage
An outside service fails calls during an outage.
Failed calls: 100%. Duration: ≈ 1,200 seconds.
What a strong design has to handle
- Four different jobs share one site: browsing, searching, the cart and paying. Do they need the same kind of storage?
- Protect hot inventory and keep checkout recoverable during payment failure.
- A cache and CDN make browsing cheap and fast, but a price or a stock count can be a minute old.
- A queue in front of payments keeps the site up when the processor is slow, but the final "paid" message arrives later.
Make a prediction, run a full traffic day, then compare the result. The in-game editorial opens after a finished day and includes measured reference designs.
Concepts to practise
Related problems
- Easy
Lab results portal
Labs send results to a portal, patients get a notice that a result is ready and open it. When the notice goes to thousands of people at the same minute, they all open the portal at once. Nobody should get the same notice twice.
- Medium
Rate limiter for a public API
Developers call your public API from scripts and apps. A few misbehaving or malicious clients can crowd out everyone else. Build limits that are fair, fast and cheap, and that survive a retry storm.
- Medium
Notification service
Every other team asks you to send messages: order shipped, new login, a sale starting. You send them by push, email and text through outside providers who have their own limits. Send each message once, and never block the team that asked.
- Hard
Scheduled job service
Customers register jobs: "call my web address every night at 02:00" or "every hour on the hour". Aim to fire within 5 seconds; work still waiting after 60 seconds counts as failed. Retry safely after a crash. One external effect requires the customer endpoint to honor a stable job occurrence key. Half the world picks the top of the hour, so a huge crowd of jobs comes due in the same second.