system design

How to design a URL shortener that survives real traffic

The entire service is one lookup. A request arrives for tor.li/9aX2bQ, you turn those characters back into the original URL, and you return a redirect. Storage is small, the logic fits on an index card, and that is precisely why interviewers keep asking it. The question isn’t whether you can build it. It’s whether you catch the three or four decisions that separate a weekend toy from something that holds a billion links and tens of thousands of redirects a second.

The read-to-write ratio decides the architecture

Traffic shape drives every later choice, so pin it down first. Reads beat writes by a wide margin, commonly around 100 to 1 and sometimes closer to 1000 to 1. A link gets created once and clicked for months. Assume 100 million new links a month and you get roughly 40 writes a second on average, a few hundred at peak. The write path can be modest. The read path is the one that hurts. At 100:1 you are serving low thousands of redirects a second on average, tens of thousands at the busy end for a large service, and each redirect sits in the critical path of a page load. Three hundred milliseconds on a redirect is something people feel.

Turning a number into six characters

The short code is the product, so spend real time here. There are three sensible ways to mint one, and an interviewer will push on the failure mode of whichever you choose.

The cleanest is a global counter plus Base62. Hand out a unique integer (a Redis INCR, or a ticket server that gives each app server a range of IDs to burn through), then encode it into the 62 URL-safe characters.

ALPHABET = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"

def encode(n):
    if n == 0:
        return ALPHABET[0]
    out = []
    while n:
        n, r = divmod(n, 62)
        out.append(ALPHABET[r])
    return "".join(reversed(out))

Seven characters give you 62^7, about 3.5 trillion codes, so you won’t run out for a very long time. Because the counter never repeats, you skip collision checks entirely, which is the whole appeal. The catch is that sequential integers produce sequential-looking codes, and anyone can walk your namespace by counting up. If some links are meant to be private, that leaks them. The fix is to scramble the integer before encoding, with a keyed permutation or a bit shuffle, so the output looks random while staying unique.

The second option is to hash the long URL, take the first 43 or so bits of an MD5 or SHA-256, and Base62 that. The same URL always lands on the same code, which gives you free deduplication. The price is collisions: two different URLs whose hash prefixes coincide. Every write becomes read, check, then write, guarded by a unique constraint on the code column and a retry when two servers race for the same slot. Under load those retries add tail latency exactly when you don’t want it.

The third is a key generation service that builds a large batch of random codes offline and hands them out from memory, marking each used as it goes. Writes never do a collision check, and the codes aren’t guessable. What you’re buying, and paying for, is a stateful service that has to stay available. If it crashes with codes checked out but not yet persisted, you lose those codes, so you track a used flag durably and accept that a crash wastes a few thousand out of a 3.5-trillion pool. That’s a rounding error.

Short-code generation strategy How a code is produced Collision handling Main tradeoff
Global counter + Base62 Increment a shared integer, encode it in the 62 URL-safe characters None required; the counter never repeats Codes are sequential and enumerable unless you scramble the integer first
Hash of the long URL Base62 of the first ~43 bits of an MD5 or SHA-256 digest Read-check-write with a unique constraint and a retry on races Same URL reuses one code, but hot writes contend and retries add tail latency
Pre-generated keys (key generation service) Random codes built offline and handed out from memory None at write time; each code is marked used as it is issued Requires a stateful, highly available service you must run and back up

301 or 302, and why interviewers care

The redirect status code is the question that catches people who memorized a diagram. A 301 says permanent, so browsers and proxies cache it and stop asking your server after the first hit. Wonderful for load. Ruinous if you want click counts, because the repeat visits never reach you. A 302 says temporary, the client comes back every time, and you keep both control and your analytics at the cost of every click touching the service. A shortener whose business is click data leans toward 302, or a 301 with a deliberately short cache lifetime, for that exact reason. Answer “301 for performance” without naming the analytics you just gave away and you’ve shown you’re reciting rather than reasoning.

Where the mappings live

The row is trivial: short code, long URL, owner, created timestamp, maybe an expiry. No joins, no multi-row transactions, one clean primary key. That is a key-value or wide-column store’s home turf. A single Postgres instance carries the first few hundred million rows without complaint, and more real shorteners run on ordinary relational databases than the diagrams suggest. Past that point a store built for horizontal sharding (DynamoDB, Cassandra, or a sharded MySQL) starts to pay off, partitioned on the short code itself so a redirect goes straight to one shard. The code is the only thing a redirect ever knows, so it is the natural shard key.

The cache that carries the load

Clicks follow a Zipfian curve: a small slice of links attracts most of the traffic. That is the ideal case for a cache. Put Redis or Memcached in front keyed on the short code and a large share of redirects never reach the database. Cache-aside works cleanly because a mapping is immutable once created, a code points at one URL forever, so there is no invalidation fight. Expiring links are the lone wrinkle, handled by a short TTL and a check on read. When someone raises the thundering herd, the answer is that immutable data barely has one. A miss just refills from the store, and a cold cache after a deploy is a short blip rather than a cascade, because nothing has to be recomputed.

Custom aliases, expiry, and abuse

Custom aliases like tor.li/black-friday change the write story. The user picks the code, so you can’t pre-assign it, and you need a uniqueness check with a clean error when it’s taken. Keep custom and generated codes in one namespace so nobody can claim a code you might later issue. Expiry is either a sweeper job or a lazy check that treats a stale row as a 404. Rate limiting belongs on the create endpoint, not the redirect, since abuse looks like a script minting spam links by the thousand.

Counting clicks without slowing the redirect

Analytics is where the design earns its keep, because you can’t do a synchronous database write on every redirect and still be fast. Push each click onto a queue (Kafka, Kinesis, or an append-only log) and let a downstream consumer aggregate. The redirect returns the moment the event is enqueued; the counting runs out of band. For live totals, keep an approximate counter in Redis and reconcile against the exact numbers from the batch job. This is also the real answer to “what happens when a link goes viral”: the redirect path is cache plus enqueue, both of which scale flat, and the numbers trail reality by seconds instead of collapsing.

The follow-up that actually separates a senior candidate is about trust, not scale. Someone will shorten a link that points at a phishing page or a malware drop, and your domain wears the reputation damage. So the redirect path grows a safety layer: a bloom filter of known-bad destinations in front, an asynchronous scan against something like Google Safe Browsing after creation, and a way to disable a code without deleting the row so its analytics survive. Get handed that question and you’re no longer designing a lookup table. You’re designing something with a blast radius.

newsletter

What's actually being asked right now

Interview patterns & comp trends, straight to your inbox.

No spam. Unsubscribe anytime.

newsletter

What's actually being asked right now

Interview patterns & comp trends, straight to your inbox.

No spam. Unsubscribe anytime.

1972 Soviet postage stamp commemorating the Mars 2 probe

worth a read

Mars For The Rest of Us — a weekly-or-more deep dive on the technical side of Mars exploration: rocket propulsion, microbiology, mission architecture, and everything in between. Written by Maciej Ceglowski.

Read it on Substack →
Scroll to Top