URL Shortener: Low-Level Design Deep Dive

A URL shortener converts a long URL into a short alias (e.g., bit.ly/abc123) that redirects to the original. Building one requires designing for high read throughput (billions of redirects per day), low latency (< 10ms per redirect), and URL uniqueness at scale. It is a canonical system design interview question because it touches URL encoding, distributed ID generation, caching, and analytics.

ID Generation and Encoding

The short code (e.g., abc123) must be unique. Common approaches: (1) Hash of the long URL: MD5 or SHA256 the URL, take the first 7 characters. Problem: hash collisions require retry logic; different users can get the same short code for the same URL (usually acceptable). (2) Auto-increment ID: generate a sequential ID (database sequence, distributed ID generator) and base62-encode it. A 7-character base62 string (62^7 ≈ 3.5 trillion combinations) accommodates any realistic scale. This approach is simpler, collision-free, and predictable. (3) Random string: generate a random 7-character base62 string, check for uniqueness in the database. Acceptable but requires a uniqueness check round-trip per creation.

Base62 Encoding

Base62 uses [0-9A-Za-z] — 62 characters, URL-safe without percent-encoding. To encode an integer ID: repeatedly divide by 62, use the remainder as the index into the character set, prepend to the result. To decode: treat the string as a base-62 number and convert to integer. A database auto-increment ID of 1,000,000 encodes to “4c92” (4 characters) — growing to 7 characters at ID 3.5 trillion. This means short codes grow naturally with usage, and you can reverse-engineer the sequential ID from the short code (a potential information leak — use random IDs if sequential exposure is a concern).

Database Schema

Minimal schema: CREATE TABLE urls (id BIGINT PRIMARY KEY, short_code VARCHAR(8) UNIQUE, long_url TEXT NOT NULL, created_at TIMESTAMP, user_id BIGINT, expires_at TIMESTAMP). Index: PRIMARY KEY on id (for ID lookup), UNIQUE on short_code (for redirect lookup), INDEX on (user_id, created_at) (for user dashboard). Read pattern: 99% of queries are “given short_code, return long_url” — a point lookup on short_code. This is extremely cache-friendly: one Redis GET per redirect, with the full URL as the value.

Caching Strategy

Redirect lookups dominate the read workload. Cache short_code → long_url in Redis with a TTL matching the URL’s expiration (or a default TTL of 24 hours for non-expiring URLs). Cache hit rate for a URL shortener is extremely high — popular short codes are accessed millions of times per day. Cache miss: read from the database, populate the cache. Cache invalidation: when a URL is deleted or updated, delete the cache key. For read throughput: a single Redis node handles 100k+ GET operations per second — adequate for most deployments. Sharded Redis handles billions of requests per day.

Redirect Response Code

Two HTTP status codes for redirects: 301 (Permanent Redirect) and 302 (Found / Temporary Redirect). 301: the client caches the redirect and future requests go directly to the long URL without hitting the shortener. Reduces shortener load but loses analytics — subsequent requests bypass the shortener entirely. 302: the client does not cache; every request goes through the shortener, enabling click tracking. Most URL shorteners use 302 for analytics tracking. If analytics are not needed and minimal server load is the goal, 301 is appropriate.

Analytics

Tracking click analytics (count, referrer, user agent, geo) per redirect adds latency to the critical path if done synchronously. Use async analytics: on each redirect, push the click event to a Kafka topic or a Redis list. An analytics consumer processes events in batches, aggregating counts by short_code, date, country, device. This keeps redirect latency at < 5ms (cache hit + Redis push) while enabling rich analytics from the asynchronous pipeline. Store aggregated analytics in a time-series store (InfluxDB, BigQuery) rather than the primary database — analytics queries are heavy aggregations that should not compete with redirect lookups.