A System Designer guide
Chat system walkthrough
Design message identity, durable acknowledgement and reconnect behavior before treating chat as a socket-count problem.
This walkthrough uses the One-to-one chat scenario. Try an attempt first if you want to make your own choices before reading a possible approach.
Requirements and scale
People send short messages to each other from their phones. A message must arrive once, in order, even when the phone is on a bad connection and sends it again. If the other person is offline, the message waits and a notification goes out.
- Who uses it
- 10 million people a month
- Audience in plain numbers
- 10,000,000 people per month
- Normal peak
- ≈ 20,000 requests per second
- Request mix
- Message: 55%; Sync change: 15%; Read: 10%; Analytics event: 10%; Sign-in: 2%; Notification: 8%
- Speed for 99 of 100 requests
- Message: ≈ 300 ms; Sync change: ≈ 400 ms; Read: ≈ 400 ms; Sign-in: ≈ 1,000 ms
- Successful requests
- 99.9%
- Data safety
- No acknowledged write may be lost.
- Monthly budget
- ≈ $60,000
- Data at rest
- ≈ 30 TB
- Where users are
- North America: 30%; Europe: 30%; Asia Pacific: 40%
- Home region
- North America
- Waiting deadlines
- Message: 900 seconds; Sync change: 900 seconds; Read: 900 seconds; Analytics event: 3,600 seconds; Sign-in: 900 seconds; Notification: 900 seconds
- Fresh reads
- People must see their own changes. At most 0.1% of reads may be stale in the model.
- Personal reads
- 95% of reads are private; a shared public copy cannot answer them.
≈ marks an approximation, just as in the game. Waiting deadlines measure the age of queued work and are separate from interactive response time.
- A message is about 1 KB (approx.).
- A delivery receipt ("delivered", "read") counts as an event.
- Notification speed measures provider acceptance. Device receipt is outside this traffic model; check expiry, provider outcome and retries in the correctness trace before claiming delivery guarantees.
Estimates to work through
- Messages per second at peak
- ≈ 11,000 msg/s. 20,000 x 0.55 (approx.)
- Messages stored per day
- ≈ 400,000,000 messages. about 40 per person per day x 10 million (approx.)
- Storage added per year
- ≈ 146 TB. 400 million x 1 KB x 365 = about 146 TB (approx.)
Define one message
A phone can lose a reply after the server saved a message. When it retries, the server must recognise the same intended message. Give it a stable identity on the device before the first send. Keep that identity across reconnects and retries.
State what sent, delivered and read mean. Durable storage is a different promise from arrival on another phone. Decide where ordering applies, usually within a conversation, and how clients detect a missing sequence position after reconnecting. A timestamp by itself does not settle simultaneous events on different devices.
A design that holds
Connection gateways keep sockets open for online recipients. Chat servers validate membership, deduplicate a message and append it to durable conversation storage before acknowledging it. Store the message identity and its outcome together so a crash does not leave an unprotected gap between a duplicate check and a write.
Deliver live messages through the recipient’s connection. Use a cursor to fetch missed messages after reconnecting. Offline notifications ask the recipient to check for messages; they are a separate path with separate provider limits. A notification accepted by a provider is not proof that the phone received it.
The trade-offs
Waiting for durable copies before acknowledging adds latency but protects acknowledged messages after failure. Holding sockets consumes memory even when no message is moving. Polling is simpler in some clients but repeatedly asks the server whether anything changed.
Deduplication needs a retention policy. If its records disappear before an old client retries, a repeated message can look new. Per-conversation ordering avoids imposing a global order on unrelated conversations, but reconnect and gap repair still need a protocol.
What breaks it
Lose a gateway, then let many devices reconnect together. Check spare connection capacity, history reads and duplicate sends at the same time. Next trace a crash after storage but before acknowledgement. A smooth traffic run cannot prove that this race preserves message identity.
Work through Live sockets and presence, Exactly once, on purpose and Ordering and time. Then try One-to-one chat. Explain one recovery trace in words and one capacity failure using the measured day.
The day that tests the design
Consumer traffic, busiest in the evening. The clock covers 00:00 to 24:00 in the home region. Events add pressure on top of that normal traffic.
08:00 · Synchronized rush
Many callers send requests together instead of spreading them over time.
Traffic for Sign-in, Message, Read, Sync change reaches 10 times the scheduled load; rises over 10 seconds, holds for 60 seconds and eases over 300 seconds. Other request types keep their scheduled load.
14:00 · Zone outage
An availability zone loses power or network access.
Duration: ≈ 3,600 seconds.
19:00 · Duplicate delivery
The queue redelivers messages. Without duplicate protection, side effects may be applied twice.
Affected share: 1%. Duration: ≈ 3,600 seconds.
Play this problem. Keep one baseline, predict a change and compare the next full day. The in-game editorial has calibrated reference designs after you finish a traffic day.