Low Level Design: URL Shortener System

By /

A URL shortener converts a long URL into a compact alias and redirects visitors to the original. Services like bit.ly, TinyURL, and t.co handle billions of redirects daily. The system looks simple but has interesting design decisions around ID generation, collision handling, caching, analytics, and abuse prevention.

Core Operations

The API has two primary operations:

  • Shorten: POST /shorten {url: "https://..."} → returns a short code (e.g., aB3xZ9)
  • Redirect: GET /{code} → HTTP redirect to the original URL

The system is heavily read-skewed. A URL might be shortened once but redirected millions of times. Design should optimize the redirect path.

ID Generation Strategies

Four approaches to generate the short code:

  • Auto-increment + base62 encode: DB auto-increments an integer ID; encode it in base62. Predictable, no collisions, but exposes sequential IDs (enumerable).
  • Hash-based (MD5/SHA256 truncated): hash the long URL, take the first 7 chars. Fast, deterministic (same URL → same code), but collision-prone at scale — must check and handle.
  • Random (nanoid/UUID truncated): generate a cryptographically random 7-char base62 string. Not enumerable, low collision probability, but still must check uniqueness before insert.
  • Snowflake ID + base62: distributed unique ID (timestamp + worker ID + sequence) encoded as base62. Sortable, no coordination needed, no collisions.

For interviews: auto-increment + base62 is the cleanest starting point. Discuss snowflake for multi-region deployments.

Base62 Encoding

Base62 uses characters 0-9a-zA-Z (62 symbols). A 7-character code gives 62^7 ≈ 3.5 trillion combinations — sufficient for any realistic URL shortener. Encoding a numeric ID:

CHARSET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"

def encode(n):
    result = []
    while n:
        result.append(CHARSET[n % 62])
        n //= 62
    return "".join(reversed(result)).zfill(7)

def decode(s):
    return sum(CHARSET.index(c) * (62**i) for i, c in enumerate(reversed(s)))

Collision Handling

For hash-based approaches, collisions (two different URLs hashing to the same code) must be handled:

  • Generate code from hash of URL.
  • Check DB: if code doesn’t exist, insert. Done.
  • If code exists and original_url matches, return existing code (deduplication).
  • If code exists but original_url differs, append a salt (url + ":1", url + ":2") and re-hash. Retry up to N times.
  • If all retries collide, fall back to random code generation.

Redirect Types: 301 vs 302

The HTTP redirect status code is a product decision:

  • 301 Permanent: browser caches the redirect. Subsequent clicks go directly to the destination without hitting your server. Reduces load but loses click analytics.
  • 302 Temporary: browser never caches. Every click hits your server, enabling accurate analytics counting. Higher server load but full visibility.

Most analytics-focused shorteners use 302. If analytics aren’t needed, 301 saves significant server load for viral links.

Storage Schema

urls (
  short_code   VARCHAR(16) PRIMARY KEY,
  original_url TEXT NOT NULL,
  user_id      BIGINT,           -- null for anonymous
  created_at   TIMESTAMPTZ,
  expires_at   TIMESTAMPTZ,      -- null = no expiry
  click_count  BIGINT DEFAULT 0  -- approximate; precise count in analytics table
)

CREATE INDEX ON urls (user_id);   -- for "list my links" queries

Redirect Caching

Every redirect hitting the DB is wasteful. Cache the mapping in Redis:

-- On shorten: SET short:{code} {long_url} EX {ttl_seconds}
-- On redirect: GET short:{code}
--   hit: redirect immediately, log click async
--   miss: query DB, populate cache, redirect

Set cache TTL to match expires_at. For non-expiring URLs, use a long TTL (e.g., 24 hours) with cache-aside invalidation on URL deletion. A small Redis instance can absorb millions of redirects per second with sub-millisecond latency.

Custom Slugs and URL Expiration

Custom slugs: let users specify a vanity path (e.g., yourdomain.com/my-campaign). Validate: alphanumeric + hyphens, max 50 chars, not in a reserved words blocklist (api, admin, health, etc.). Check uniqueness before insert; return a conflict error if taken.

URL expiration: store expires_at in DB and set matching Redis TTL. On redirect, check expiry even on cache hit (cache TTL should not exceed DB expiry). Run a periodic cleanup job (DELETE FROM urls WHERE expires_at < NOW()) to reclaim DB space. Optionally return 410 Gone instead of 404 for expired URLs to signal intentional removal.

Click Analytics Pipeline

Counting clicks synchronously on every redirect creates write bottlenecks. Use an async pipeline instead:

  • On redirect, fire-and-forget: publish a click event to a Kafka topic or SQS queue containing {short_code, timestamp, user_agent, referrer, ip, geo}.
  • A stream processor (Flink, Spark Streaming, or a simple consumer) aggregates events and batch-upserts counts into a click_stats table.
  • For real-time dashboards, maintain a Redis counter per short code: INCR clicks:{code}.

Separate the hot redirect path (reads Redis) from the analytics path (writes Kafka) entirely. The redirect latency is unaffected by analytics volume.

Abuse Prevention

URL shorteners are abused for phishing and malware distribution. Mitigations:

  • Blocklist check on creation: compare against Google Safe Browsing API or a local blocklist of known malicious domains.
  • Rate limiting: limit shortening requests per IP (e.g., 10/minute for anonymous, 100/minute for authenticated users) using a Redis sliding window counter.
  • Interstitial page: for suspicious URLs, show a "you are leaving our site" warning page before redirecting.
  • Automated scanning: periodically re-scan stored URLs against updated threat feeds; deactivate malicious ones retroactively.

Horizontal Scaling

The system is read-heavy (redirects) with moderate writes (shortening). Scale accordingly:

  • Redirect path: stateless app servers behind a load balancer; Redis cluster for cache; read replicas for DB fallback.
  • Write path: rate-limited; a single primary DB handles shortening load at most scales. For global deployments, use a distributed ID generator (Snowflake) to avoid coordination.
  • Analytics path: decoupled via Kafka; scale consumers independently from the redirect path.
  • CDN: for 301 redirects, a CDN can serve the redirect response itself — no origin hit for cached short codes.
Scroll to Top