The fastest way to stall out in a recommendation system design interview is to reach for a model in the first two minutes. The prompt is usually one line, something like “design the home feed for a short-video app” or “recommend restaurants for a food delivery service,” and a lot of candidates answer by naming an algorithm before anyone has agreed on what the system optimizes or how large it is. The first five minutes set up the next forty.
The prompt lands in one of a handful of shapes:
- “Design the For You feed for a short-video app like TikTok.”
- “Design YouTube’s video recommendations.”
- “Recommend restaurants to a user on a food delivery app.”
- “Build the product recommendations for an e-commerce home page.”
- “Design Spotify’s Discover Weekly playlist.”
Nail down the objective before you draw a single box
Recommending the next video, the next product, and the next person to follow are three different problems with different labels and different ways to fail. So the opening move is to ask what you’re actually building. What are we recommending, and onto what surface: an endless feed, a shelf of ten items, a single next-up slot? What does the business want more of: watch time, purchases, retention, revenue per session? How many users and items, and at what request rate? What’s the latency budget from request to rendered list?
Get rough numbers on the table, because they drive every later decision. A catalog of a few thousand items is a ranking problem you can brute-force. A catalog of a billion videos or half a billion products is a retrieval problem first, and no amount of clever ranking saves you if you try to score the whole catalog per request. Say the numbers out loud: hundreds of millions of daily users, a catalog in the hundreds of millions to billions, tens of thousands of requests per second at peak, a budget of maybe 100 to 200 milliseconds for the whole recommendation path. Now the architecture has constraints to satisfy instead of floating in the abstract.
The funnel is how you serve a billion items in 100 milliseconds
No production recommender scores its entire catalog for every request. They stage the work into a funnel that gets narrower and more expensive at each step. Retrieval, also called candidate generation, pulls a few hundred to a few thousand plausible items out of the full catalog using cheap methods. Ranking scores those candidates with a heavier model and rich features. A final re-ranking or policy layer reorders the top of the list for diversity, freshness, and business rules before it goes on screen. YouTube described exactly this split in their 2016 candidate-generation-plus-ranking paper, and the shape has held up across TikTok, Instagram Reels, Pinterest, and most large feeds since.
| Stage | Catalog size in to out | Typical model | Feature scope | Latency budget |
|---|---|---|---|---|
| Retrieval (candidate generation) | ~1B items to ~1,000 | Two-tower plus ANN index, collaborative filtering, trending sources | User and item embeddings, no cross features | ~1 to 10 ms |
| Ranking | ~1,000 to ~100-500 | Gradient-boosted trees, or deep nets (Wide & Deep, DCN, DLRM), often multi-task | User, item, context, and user × item cross features | ~10 to 50 ms |
| Re-ranking / policy | ~100-500 to ~10-20 shown | Rules plus a light model | Diversity, freshness, business and safety rules | ~1 to 5 ms |
Draw this funnel early. It answers the scale question before the interviewer has to ask it, and it gives you a natural place to go deep when they say “pick a stage and design it.”
Retrieval: two towers and approximate nearest neighbors
The workhorse for candidate generation is the two-tower model. One tower encodes the user, their history and demographics and context, into an embedding; the other encodes an item into an embedding in the same space. Training pulls the vectors of items a user engaged with close together and pushes everything else apart, usually with in-batch negatives so every other item in the training batch acts as a cheap negative example.
The reason this architecture wins at serving time is the asymmetry. Item embeddings don’t depend on the user, so you compute all of them offline, refresh them on a schedule, and load them into an approximate nearest neighbor index like FAISS or ScaNN. When a request comes in, only the user tower runs online to produce one query vector, and the ANN index returns the top few hundred items in a couple of milliseconds even against a catalog of hundreds of millions. That’s the point the interviewer wants you to explain: precompute the expensive half, keep the online half tiny.
One two-tower model is rarely the whole story. Real systems blend several retrieval sources and merge the results: an embedding retriever for personalized taste, a collaborative-filtering source for “people like you also watched,” a source for accounts or creators the user follows, and a trending or fresh-content source so new items get a shot. Saying you’d run multiple sources in parallel and dedupe them signals you’ve seen a production system rather than a diagram.
Ranking: where the cross features and multiple objectives live
Retrieval deliberately throws away the interaction between a specific user and a specific item because it has to score in bulk. Ranking gets that back. Now you have a few hundred candidates instead of a billion, so you can afford a model that takes explicit user × item cross features: has this user watched this creator before, how many of this restaurant’s dishes match the user’s past orders, how long since the user last saw this item. Gradient-boosted trees are still a strong and common baseline here. The large platforms mostly run deep networks built for this, Wide & Deep and DCN from Google, DLRM from Meta, DeepFM, all designed to learn feature interactions that trees approximate more slowly.
The part that separates a good answer from a great one is admitting you’re not optimizing a single number. Optimize click-through rate alone and you build a clickbait machine. Production rankers are multi-task: they predict several things at once, P(click), P(watch past 30 seconds), P(like), P(share), P(hide or report), and combine them into one score with tuned weights that encode what the product actually values. When the interviewer asks how you’d stop the feed from filling up with outrage bait, this is the answer: the negative-signal heads and their weighting, plus the re-ranking layer that caps how much of any one kind of content a session can show.
Cold start and the feedback loop nobody plans for
A new item has no interaction history, so an embedding retriever trained on engagement can’t reach it, and it never gets shown, so it never earns any history. You break the loop with content features (the item’s text, thumbnail, category, audio) so a fresh item has a reasonable embedding on day one, plus an exploration budget that deliberately shows new or uncertain items to gather signal. A new user is the same problem on the other axis: fall back to popularity and trending, use whatever onboarding signals you have such as declared interests, device, and locale, and personalize fast as the first few interactions arrive.
The subtler trap is that the model trains on data the model itself generated. It only sees engagement on items it chose to show, and users click higher positions more regardless of relevance. That’s position bias and feedback-loop bias, and if you ignore them the system slowly narrows into whatever it already believed. Strong answers mention logging the propensity, the probability that an item was shown, so you can reweight during training and evaluation, and keeping some exploration through epsilon-greedy or Thompson sampling so the model keeps learning about items it’s unsure of.
Metrics: offline numbers lie, so design the A/B test
You’ll get asked how you know it works. Offline, retrieval is measured with recall@k, meaning whether the items the user actually engaged with showed up in the retrieved set, and ranking with AUC or NDCG on held-out logs. Report those, but say the quiet part too: offline gains often don’t survive contact with real traffic, because the logs were collected under the old model and can’t tell you how users react to recommendations they never saw. The number that decides a launch is an online A/B test on the real objective, watch time or retention or revenue, with guardrail metrics watching for damage the main metric hides, things like report rate, catalog diversity, session length, and complaints from creators whose reach changed.
Interviewers care less about how many model architectures you can name than about whether you scoped the objective, built a funnel that makes the scale tractable, designed one component in real depth, and could say clearly how the system fails and how you’d catch it. Fifteen minutes spent getting the objective and the funnel right buys you somewhere to stand for everything after.
