Easy · AI, ads and search
Semantic search
A help center lets people type a question in plain words and finds the right articles even when the words differ. Each question is turned into a list of numbers (a vector) and matched to similar articles. Many people ask the same few questions. Practise this easy system design problem for free with a Google or GitHub account.
The brief
A help center lets people type a question in plain words and finds the right articles even when the words differ. Each question is turned into a list of numbers (a vector) and matched to similar articles. Many people ask the same few questions.
Think of "search by meaning" on a help center.
Users, scale and requirements
- Who uses it
- 3 million people a month (approx.)
- Audience in plain numbers
- 3,000,000 people per month
- Normal peak
- ≈ 400 requests per second
- Request mix
- Search: 85%; Read: 12%; Write: 3%
- Speed for 99 of 100 requests
- Search: ≈ 1,000 ms; Read: ≈ 800 ms; Write: ≈ 800 ms
- Successful requests
- 99.9%
- Data safety
- At most 1 minute of writes may be lost.
- Monthly budget
- ≈ $2,800
- Data at rest
- ≈ 0.05 TB
- Where users are
- North America: 60%; Europe: 40%
- Home region
- North America
- Waiting deadlines
- Search: 900 seconds; Read: 900 seconds; Write: 3,600 seconds
≈ marks an approximation, just as in the game. Waiting deadlines measure the age of queued work and are separate from interactive response time.
- About 50,000 articles, each stored as one vector (approx.).
- Making a vector takes about 30 ms on a small model server (approx.).
- All numbers are rough approximations.
Questions that shape the design
How fast must an answer come back?
Under about a second. A person is waiting at the search box.
Do many people ask the same thing?
Yes. About half of all questions repeat within a day.
How often do articles change?
A few times a day. New articles are searchable within minutes.
The traffic day
Traffic concentrated in working hours. The clock covers 00:00 to 24:00 in the home region. Events add pressure on top of that normal traffic.
10:00 · Viral spike
A post blows up. Traffic climbs to several times normal within minutes.
Traffic reaches 4 times the scheduled load; rises over 600 seconds, holds for 3,600 seconds and eases over 3,600 seconds.
14:00 · Bad deploy
A new version ships with a bug and some requests fail.
Lasts until detected and repaired.
What a strong design has to handle
- Half of the questions repeat. Do you need to turn each into a vector again?
- A longer cache time saves model work, but a changed article may not show for an hour.
- A bigger model finds better matches and costs more per question. A small one is usually enough for a help center.
Make a prediction, run a full traffic day, then compare the result. The in-game editorial opens after a finished day and includes measured reference designs.
Concepts to practise
Related problems
- Medium
Recommendations
Every page shows a row of things the visitor might like. A ranking model builds each row, but the model is slow and costly. The row must appear instantly, even when the model is having a bad day.
- Hard
Ad serving
Each page view asks you to pick the best ad in about a quarter of a second: check who is allowed to see what, score the candidates with a model, run a quick auction and answer. Every impression and click must be counted exactly because money depends on it.
- Hard
AI chatbot serving
Millions of people ask your assistant questions. Each reply takes a GPU several seconds, and GPUs are expensive and sometimes simply unavailable. Keep replies flowing, protect your wallet from abuse, and have a plan for when the GPUs run out.
- Medium
Search autocomplete
As people type, they expect suggestions after every letter, in the blink of an eye. Most people type the same popular beginnings. When news breaks, one phrase suddenly dominates.