The hardest part of a chat system shows up the moment the sender’s phone displays one gray checkmark while the recipient’s phone shows nothing at all. The message left one device and never reached the other, or reached it twice, or landed after a reply that logically came later. When an interviewer asks you to design WhatsApp or Slack, that gap is what’s being graded: do you know where those failures come from, and can you design so they don’t happen.
Draw the “Chat Service” box quickly and get past it. The parts that earn the offer are how a message travels from one connection to another, how you promise it arrived, how you keep a conversation in order, and how you know who is online without writing to your database a million times a second.
The connection layer is most of the problem
Chat is a write-heavy, real-time system where a message has to reach the right device in a couple hundred milliseconds and never silently vanish. Plain HTTP request/response doesn’t fit, because the server has to push to the client on its own, at any moment, without waiting for the client to ask. So the connection is a WebSocket: one long-lived TCP connection per device, over which either side can send a frame whenever it wants.
Each gateway server holds a large number of these open sockets, often in the tens or hundreds of thousands. That creates the routing problem that trips up most candidates. User A’s socket lives on gateway 3. User B’s socket lives on gateway 7. When A sends a message to B, gateway 3 has no idea where B is connected. You need a registry: a fast store, usually Redis, mapping each user to the gateway and connection currently holding their socket. Gateway 3 looks up B, finds gateway 7, forwards the message there over an internal channel, and gateway 7 writes it to B’s socket.
A question that sounds simple and isn’t: a user has the app open on their phone and their laptop, then sends a message from the phone. Walk through every hop. Answering it forces you to admit that one account maps to several live connections, so the registry points a user at a set of connections, not one. Every other device that same user owns should also receive the message, which is how the laptop stays in sync. And the recipient’s account fans out to all of their devices too. One logical send becomes many socket writes.
What “delivered” has to mean
Delivery has three possible promises. At-most-once sends and forgets, and loses messages whenever the network hiccups. Exactly-once sounds ideal and is expensive enough that almost nobody actually builds it. Chat systems pick at-least-once and strip the duplicates on the client. A message may arrive more than once, so the receiver needs a way to recognize a repeat and drop it.
That recognition depends on the client generating an ID before it ever sends. If the phone retries because it never heard back, the retry carries the same client-side ID, and the server or the receiving device discards the second copy. The server also stamps its own canonical ID, which it uses for storage and ordering.
{
"client_msg_id": "b1e...c9", // made on the sender's device, used for dedup
"conversation_id": "conv_42",
"sender_id": "user_a",
"body": "on my way",
"sent_at": 1720000000000 // client clock, for display only, never for ordering
}
The ack chain is where the checkmarks come from. The sender’s device keeps the message pending until the server confirms it wrote the message down; that first checkmark means stored, not yet delivered. When the recipient’s device receives it, a delivery ack travels back for the second checkmark. When the recipient opens the conversation, a read ack follows. Each of those transitions is a separate small message going the opposite direction from the chat itself.
The order of those steps matters more than it looks. Persist before you ack the sender. If the server says “sent” and then crashes before the write lands, the sender believes the message is safe while it has actually vanished. Durable first, acknowledge second, every time.
Keeping a conversation in order
You cannot put a total order on every message in the whole system, and you don’t need to. Nobody cares whether a message in your family group happened before or after a stranger’s message in an unrelated channel. Order only has to hold within a single conversation.
The clean approach is a server-assigned sequence per conversation, or a time-sortable ID such as a Snowflake: a 64-bit number built from a timestamp, a machine ID, and a per-generator counter, which climbs monotonically from any one generator. Store messages keyed by conversation and that ID. Everyone who loads the conversation reads the same stored order, so everyone sees the same thing.
The trap the interviewer is hunting for is ordering by client timestamp. Device clocks are wrong, drift apart, and can be set by hand. If two phones disagree by three seconds, their messages interleave differently for different viewers. A common follow-up: two people reply in the same channel at the same instant, so what order does everyone see, and why is it identical for all of them? The answer is that one point assigns the sequence, whether that’s a single sequencer per conversation or the database’s own clustering order on the message ID. Because the order is written down exactly once, every reader inherits the same tie-break.
Presence costs more than the messages do
Presence looks trivial and quietly generates more write traffic than the messages themselves. Every connect, disconnect, and heartbeat is a write, and a person fires off far more heartbeats than messages in a day. So presence lives in its own service with its own store, kept off the message path.
The usual mechanism is a heartbeat against a short TTL. On connect, set a key that expires in about 30 seconds, and have the client ping every 10 to refresh it:
SET presence:user_b 1 EX 30 # on connect
# client heartbeat every ~10s:
EXPIRE presence:user_b 30 # refresh the window
# no heartbeat for 30s -> key expires -> user shows offline
“Last seen 2 minutes ago” is an approximation, and you should keep it one. Write last_seen on disconnect and occasionally while connected, never on every heartbeat. The other scaling question is who gets told when someone’s dot turns green. Pushing every presence change to everyone who lists that person as a contact means a popular account sets off a storm each time their network flickers. The fix is to subscribe to presence only for the conversations a user currently has open, so you compute and push presence for a handful of people at a time instead of thousands.
Group chat and the fan-out decision
For a small group, fan-out on write is fine: when a message arrives, push it to each member’s socket in a loop. Two hundred members is two hundred socket writes and takes milliseconds. For a 50,000-member channel, that same loop turns into a stampede on every single message, and most of those members aren’t even looking at the screen.
Large rooms flip to fan-out on read. Members pull with a cursor, asking for messages after the last sequence they have seen, and the server stops pushing to everyone. Slack works this way for big channels. The design most people land on is a hybrid with a size threshold, and above it messages go onto a queue partitioned by conversation, with workers handling delivery so a burst in one giant channel doesn’t stall everything else.
| Group size | Delivery model | How messages reach members | Why this model, not the other |
|---|---|---|---|
| Under ~200 members | Fan-out on write (push) | Server pushes to each online member’s WebSocket in a loop; offline members get a push notification | A few hundred writes per message is cheap and delivers instantly |
| ~200 to ~1,000 members | Hybrid (queue then push) | Publish to a queue partitioned by conversation_id; workers push to online members | Decouples a busy room from the sender’s request so one active group can’t stall others |
| Over ~1,000 members (large channels) | Fan-out on read (pull) | Clients poll with a cursor (“messages after seq X”); no per-message push to everyone | Pushing every message to thousands of mostly-idle members wastes the connection layer |
The thresholds aren’t sacred numbers, and a good candidate says so out loud. What the interviewer wants is the reasoning behind them: push gives the lowest latency and costs one write per recipient, pull costs almost nothing to send but adds a poll on every reader, and the crossover sits wherever recipient count times message rate stops being cheap.
When the recipient is offline
A message to an offline user still gets persisted the same way; being offline changes delivery, not storage. On reconnect, the device asks what it missed since its last known sequence and pulls the backlog, and then those pending messages flip to delivered and their acks travel back to the senders. A push notification through APNs or FCM wakes the app in the meantime. Because each device tracks its own last-delivered cursor, the phone and the laptop can sit at different points in the same conversation and each one catches up on its own.
Candidates who do well spend their forty-five minutes on the ack chain and the ordering guarantee, not on adding more boxes to the diagram. A gray checkmark is a promise the system made to a person, and the whole design exists to keep it. If you can explain how that promise survives a dropped connection, a duplicate retry, and two people typing at the same millisecond, you have answered the question. For a longer end-to-end version, the Hello Interview WhatsApp breakdown covers the same ground in more depth.
Keep sharpening your system design:
