A single WhatsApp front-end server can hold somewhere around two million open connections at once. That one number is most of the interview. The service pushes on the order of 100 billion messages a day to more than two billion people, and almost none of the hard part is storing those messages. It’s knowing which server a recipient’s phone is attached to at this instant, and getting a few hundred bytes onto that phone before anyone notices a lag.
Interviewers like WhatsApp because the requirements sound like a weekend project and the mechanics are anything but. Send text, get a checkmark, done. Then come the follow-ups: what happens when the recipient’s phone is off for two days, when a group has 900 members, when the server holding a live connection dies with a message half-delivered.
Cut the scope before you draw a single box
Say out loud what you’re building and what you’re setting aside. In scope: one-to-one text, group text, delivery and read receipts, and online and last-seen status, including delivery to phones that were offline. Out of scope, but worth naming so the interviewer knows you know: media, which rides a separate upload-and-CDN path rather than the message channel; voice and video calls, which are a different real-time stack; and end-to-end encryption, which shapes what the server can and can’t do but still leaves it routing ciphertext it can’t read. Cutting media out early is worth real points, because plenty of candidates burn ten minutes designing a blob store nobody asked about.
Then do the arithmetic, because it tells you what to optimize. Two billion users, call it half connected at peak, so roughly a billion live connections. A hundred billion messages a day averages a bit over a million per second, and real traffic spikes maybe 3x around evenings and holidays, so plan for three to four million messages per second. A text message is tiny, a hundred bytes or so. That combination is the whole shape of the problem: trivial bandwidth, enormous connection count, and routing as the bottleneck. If you delete each message once it’s delivered, which is what WhatsApp historically did, your durable store only has to hold the undelivered backlog, not years of history.
The connection layer is where the design lives
Every phone keeps a long-lived TCP connection open to a gateway server. Browsers use a WebSocket; the mobile apps run a compact custom protocol over TCP. WhatsApp built these gateways on Erlang for a specific reason: each connection becomes a cheap, isolated lightweight process, and one box can babysit a couple million of them without falling over. The gateway holds the session in memory and nothing more precious than that.
Routing is the actual puzzle. When A sends to B, the gateway holding A has to find the gateway holding B. You keep a session registry that maps each user id to the gateway currently serving them. On connect, a gateway claims ownership by writing that mapping to a fast shared store; on disconnect, it clears it. To deliver, A’s gateway looks up B, opens or reuses an internal connection to B’s gateway, and hands the message across, and B’s gateway writes it down B’s socket. Losing the registry in a crash is fine, because clients reconnect and rewrite their entries within seconds.
Store, forward, then forget
Follow one message end to end and the checkmarks explain themselves. A’s client sends; A’s gateway writes the message to an offline mailbox keyed by the recipient and immediately returns the single check, meaning the server has it. The gateway looks up B. If B is online, it forwards, B’s device acknowledges, and the server marks the message delivered, the second check, and can drop its stored copy. If B is offline, the message waits in the mailbox until B reconnects, at which point the gateway drains the mailbox in order, delivers, collects acks, and deletes. The blue read receipt is a separate tiny message B’s client sends back when the chat is actually opened, which is why read receipts arrive after delivery and can be switched off on their own.
What the backend keeps, and where
Different data has wildly different access patterns, so it lands in different stores. Naming the right tool for each, and saying why, is most of what a system-design grader is listening for.
| Data | Storage choice | Access pattern it has to serve |
|---|---|---|
| Session registry (user to gateway) | In-memory table backed by a Redis-style store | Millions of tiny lookups per second on the delivery path; safe to lose in a crash since clients reconnect |
| Undelivered messages (offline mailbox) | Wide-column store such as Cassandra or HBase | Write-heavy, keyed by recipient, drained in order and deleted once delivery is confirmed |
| Users, contacts, and group membership | Sharded relational store (sharded MySQL or Postgres) | Strongly consistent reads, low write rate, read on every group send so it gets cached |
| Presence and last-seen | In-memory with a short TTL (Redis) | Cheap, self-expiring, tolerant of being a few seconds stale |
| Media (images, video, documents) | Object store plus CDN | Large payloads uploaded once and fetched by URL, off the message channel entirely |
Groups turn one message into hundreds
A group send is a fanout. One inbound message from the sender becomes N-1 outbound copies, one per other member, each either forwarded to that member’s gateway or dropped into their mailbox. For a fifty-person group this direct fanout is fine and runs inline. Group size caps, 256 for years and later raised past a thousand, exist partly to bound exactly this cost. Once groups get large, you don’t want a single message tying up the sender’s gateway while it resolves a thousand members and writes a thousand mailboxes, so you push that work onto dedicated fanout workers behind a queue like Kafka: the gateway publishes once, and the workers do the expansion. Membership lives in the metadata store and gets cached hard, since you read it on every single group message.
Encryption complicates groups more than it complicates one-to-one chats. Under the Signal protocol the server never sees plaintext, and a naive group message would be encrypted separately for each recipient. WhatsApp uses a sender-key scheme to cut that down, but the point for an interview is that group fanout also has to distribute encrypted session keys, and the server stays a blind courier the whole way through.
Presence you can actually afford
The naive version of online status broadcasts a notification to all your contacts every time you connect or drop. At two billion users that’s a self-inflicted denial of service. The cheap version: the client sends a heartbeat every few seconds, which refreshes a TTL key in Redis; miss two heartbeats and you’re marked offline with a last-seen timestamp. You only compute and push presence for chats a user currently has open, on demand, rather than telling every contact about every state change. Privacy settings then gate whether last-seen is even visible, a product decision that happens to also cut load.
Ordering, duplicates, and the gateway that dies mid-delivery
Ordering is per-conversation, not global. Each message carries a monotonic sequence number within its chat, enough for the client to order messages and notice a gap, and you never need a global clock across the whole system. That restraint is the answer to the trap question about globally ordering billions of messages a second. You don’t. You order within a conversation and nowhere else.
The network drops packets, so you retry, and retries mean a message can show up twice. Every message gets a client-generated id, and the recipient dedupes on it. Acks are the spine of the whole thing: the sender retries until the server acks, and the server retries delivery until the device acks. That’s at-least-once delivery with client-side dedup, which is the right tradeoff for a mobile network you can’t trust.
Which sets up the question interviewers actually enjoy asking. A few of the ones that separate a clean design from a shaky one:
- The gateway holding B crashes after it received the message but before B’s device acked it. What happens to the message?
- B has two phones and a laptop. How does one message reach all three and stay in sync?
- A sends, immediately loses signal, and the app retries the send. Does B end up seeing it twice?
The first one is the payoff of everything above. Because the message stays in the mailbox until a delivery ack comes back, a dead gateway loses nothing: B reconnects to some other gateway, that gateway drains the mailbox again, and B’s client throws away the duplicate by id. Multi-device is the same trick aimed at a fan of destinations, each device carrying its own mailbox cursor. And the retry-into-a-duplicate case is exactly why the id and the dedup exist in the first place.
This design holds up under two billion users for one reason: it treats the network as hostile and server memory as disposable. A gateway can vanish and the worst case is a reconnect plus a duplicate the client silently drops. Being able to say that, and show why it’s true at every hop, is what the WhatsApp question is really checking.
Keep sharpening your system design:
