Run five cache servers and pick one with hash(key) % 5, and everything works until the day you add a sixth. Switch to % 6 and almost every key now maps to a different server. Your hit rate collapses at the exact moment you were adding capacity, the databases behind the cache take the full brunt of the miss storm, and whoever is on call gets to explain why scaling up caused an outage.
Consistent hashing is the standard answer to that failure, and it shows up in system design rounds any time you shard state across a changing set of machines: a cache tier, a database ring, session affinity, a request router. The idea is small. The reasons it is built the way it is are what interviewers actually want to hear.
The modulo trap
With N servers, plain hash(key) % N spreads keys evenly, which is why people reach for it first. The catch is that N sits inside the formula. Change N and you change the output for nearly every key. Add one server to a cluster of nine and roughly 90% of keys relocate. For a cache that means a near-total miss rate until things warm back up, and for a shard map it means moving almost all your data to add 10% capacity.
What you want instead: when a server joins or leaves, only the keys that belonged to that server should move. Everything else stays put. Consistent hashing gets close to that. Adding an Nth node moves about 1/N of the keys, not all of them.
The ring, and the one property that matters
Map the output of your hash function onto a circle. If the hash produces a 32-bit value, picture the range 0 to 2^32 – 1 bent around so the top wraps back to zero. Hash each server’s identifier, its IP or name, and place it at that point on the ring. To find where a key lives, hash the key, then walk clockwise until you hit the first server. That server owns the key.
Now remove a server. Only the keys sitting between it and its predecessor on the ring need a new home, and they slide to the next server clockwise. Every other key is untouched, because its clockwise walk still lands in the same place. Add a server and the mirror image happens: it inserts itself at one point and takes over just the slice of keys between it and the previous node. One node’s worth of keys move. That is the whole property, and it is the sentence you should be able to say cleanly at a whiteboard.
Why the plain ring balances badly
There is a problem the clean version hides. With only a handful of servers hashed onto the ring, their positions are random, so the arcs between them come out wildly uneven. One server might own a third of the ring while another owns a sliver. Hashing does not distribute a few points evenly, it distributes many points evenly, and five is not many.
The fix is virtual nodes. Instead of placing each physical server at one point, place it at many: hash server-A#1, server-A#2, on up to server-A#200, and scatter all of them around the ring. Each physical machine now owns a couple hundred small arcs spread across the circle instead of one big contiguous chunk, and the law of large numbers evens the load out. A useful side effect shows up when a node dies: its share does not dump onto a single unlucky neighbor, it spreads across all the nodes that happened to sit clockwise of its many virtual points. Mismatched hardware falls out for free too, since a box with twice the memory can simply be given twice as many virtual nodes.
What Cassandra, DynamoDB, and Discord actually do
Cassandra is the textbook production example. It calls virtual nodes “vnodes” and assigns each node a set of tokens on the ring. For years the default was 256 tokens per node; Cassandra 4.0 dropped that to 16 and paired it with an allocation algorithm that places tokens to keep replicas balanced, because 256 random tokens per node turned out to hurt availability and slow down repair. That tradeoff (more vnodes means smoother distribution but more ring metadata and heavier operations) is a real design axis, not a settled number.
Amazon’s original Dynamo paper is where much of this entered mainstream practice, using consistent hashing with virtual nodes across the ring. The managed DynamoDB service hides the ring entirely: it splits a partition automatically once it passes about 10 GB and shuffles throughput around with adaptive capacity. You still cannot escape the underlying physics. A single hot partition key concentrates traffic on one partition, and the standard remedy is to shard the key in the application by suffixing it with a small random value, which is consistent hashing’s uneven-arc problem reappearing one level up.
Discord routes chat traffic with consistent hashing in its Elixir stack, mapping each guild to the node responsible for it so any process in the cluster can find the right home for a given room. Same pattern as the cache example, applied to stateful sessions instead of cached values.
Bounded loads, and the hot key the ring cannot fix
Virtual nodes even out keyspace, not traffic. If one key is a celebrity and takes a million reads a second, consistent hashing faithfully sends all million to whichever node owns that point, and that node falls over while its neighbors idle. A uniform spread of keys says nothing about a uniform spread of requests.
Google published “consistent hashing with bounded loads” in 2017 for exactly this. You set a capacity per node as (1 + ε) times the average load, and when you place a key you walk clockwise to the first node that is not already at capacity rather than the first node period. It keeps the low-churn property of the ring while capping how overloaded any single node can get. This is not a museum piece: the HAProxy load balancer implements it, and in 2025 the KubeAI project reached for the same algorithm to route LLM inference, since a prompt with a long shared prefix wants to land on the replica that already has that prefix cached in GPU memory, yet no single replica can be allowed to absorb every request for a popular prefix.
Rendezvous hashing, the alternative worth naming
If someone asks whether the ring is the only way, the answer is no, and naming the alternative scores points. Rendezvous hashing, also called highest random weight, computes hash(key, server) for every server and sends the key to whichever server scores highest. Adding or removing a server only changes the assignment for keys whose top-scoring server was the one that changed, so it has the same minimal-movement property with no ring to maintain. The cost is an O(N) scan per lookup instead of an O(log N) binary search over sorted ring positions, which is fine for tens of servers and painful for tens of thousands. It is what you want when N is small and you would rather skip the vnode bookkeeping.
| Key-placement strategy | Keys that move when one node joins or leaves (cluster of N) | Load balance with only a few nodes | Lookup cost per key | Seen in production |
|---|---|---|---|---|
Modulo, hash(key) % N |
Nearly all keys, about (N-1)/N | Even while N is fixed | O(1) | Naive sharding of a static cluster |
| Consistent hashing, plain ring | About 1/N of keys | Uneven; large arcs with few nodes | O(log N) | Rarely used on its own |
| Consistent hashing with virtual nodes | About 1/N of keys | Even, and tunable by vnode count | O(log(V·N)) | Cassandra, Dynamo, Discord |
| Consistent hashing with bounded loads | About 1/N, plus spillover to the next open node | Even, with a hard per-node cap | O(log N) amortized | HAProxy, KubeAI LLM routing |
| Rendezvous (highest random weight) | Only keys whose top-scoring node changed, about 1/N | Even | O(N) | Small, mostly static clusters |
What the interviewer is really probing
Reaching for consistent hashing at the right moment matters more than reciting the mechanics. The trigger is a set of nodes that changes over time plus data that has to stick to specific nodes. If the data is stateless, a plain load balancer is the better answer and consistent hashing is over-engineering. Expect the follow-ups to go straight at the weak spots:
- “You have five cache nodes and add a sixth. Roughly what fraction of keys move, and why is it not all of them?”
- “With only five servers on the ring, one is getting hammered. What is wrong, and how do you fix it?”
- “A single key is now 40% of your traffic. Does consistent hashing help? What does?”
- “How does a request router learn the current ring as nodes join and leave? Who owns that membership?”
That last one separates people who read a blog post from people who have run this. The ring only works if every client agrees on the current membership, so something has to propagate it: a gossip protocol like Cassandra’s, a coordination service like ZooKeeper or etcd, or a config service the routers poll. When that membership view goes stale or splits, two clients disagree about who owns a key, and you get the split-brain writes and cache inconsistency the tidy diagram never shows. Get the ring right and you have solved key placement. Getting everyone to agree on the ring is the part that keeps you up at night.
Keep sharpening your system design:
