System Design: Content Delivery Network (CDN) — Edge Caching, Routing, and Cache Invalidation

What is a CDN and Why Does It Matter?

A CDN is a geographically distributed network of edge servers that cache and serve content close to end users. Without a CDN, every request travels to the origin server — a user in Singapore fetching content from a New York server experiences 200-300ms RTT. With a CDN, the user gets content from a Singapore edge node in < 10ms. Beyond latency: CDNs absorb traffic load (origin serves only cache misses), provide DDoS protection (massive distributed capacity), and reduce bandwidth costs (edge serves cached content cheaply). Cloudflare has ~300 PoPs globally; Fastly has 60+; AWS CloudFront has 400+. This question is asked at Cloudflare, Akamai, Netflix, and any company with significant static asset serving.

Cache Architecture and Hit Ratio Optimization

Two-tier CDN: edge nodes (L1) close to users + regional PoPs (L2) between edge and origin. Cache miss at L1 → check L2 → if L2 miss → fetch from origin. This reduces origin load and gives L2 nodes a warmer cache (they serve multiple L1 nodes). Cache key: (hostname, URL path, normalized query string, Vary headers). Vary header: if the response varies by Accept-Encoding (gzip vs br) or Accept-Language, each variant is cached separately. Cache-Control headers drive caching behavior: max-age=31536000 + immutable: cache forever (use for hashed static assets like bundle.abc123.js). max-age=0, must-revalidate: always revalidate with origin (for HTML pages). s-maxage: CDN-specific TTL, overrides max-age for CDNs while browsers use max-age. no-store: never cache (for sensitive personalized content). Hit ratio targets: static assets (JS, CSS, images) should achieve > 99% hit ratio. API responses: 30-70% depending on personalization. HTML pages: 50-80% if not personalized. Hit ratio drives cost — each cache miss generates an expensive origin request.

Request Routing

How does a user request reach the nearest edge node? DNS-based routing: the CDN’s authoritative DNS server returns different IP addresses based on the resolver’s location. The resolver closest to the user (typically the ISP’s DNS) gets an IP for the nearest PoP. TTL of 30-60 seconds allows fast failover. Anycast: the CDN announces the same IP prefix from all PoPs via BGP. Internet routers direct traffic to the nearest PoP topologically (not geographically). More robust than DNS routing for DDoS: traffic is absorbed by the nearest PoP automatically. Health checks: each PoP continuously health-checks its neighbors and the origin. On PoP failure, DNS routing removes the failed PoP from responses within one TTL cycle. GeoDNS and Anycast are often combined: Anycast handles routing within a region, GeoDNS handles continent-level routing. Load balancing within a PoP: multiple edge servers per PoP. Consistent hashing distributes requests for the same URL to the same server (maximizing cache hits within the PoP). Layer 7 load balancer (Nginx, HAProxy) in front of edge servers.

Cache Invalidation

Cache invalidation is “one of the two hard things in computer science.” Approaches: TTL-based expiry: simplest. Content expires and is fetched fresh from origin on the next request. No active invalidation needed. Risk: stale content for the duration of the TTL. URL versioning (cache-busting): change the URL when content changes (bundle.abc123.js → bundle.def456.js). The old URL stays cached (fine — it’s no longer referenced). The new URL is a cold miss on first request. Eliminates the need for active invalidation for static assets. Purge API: CDNs expose an API to immediately invalidate cached content by URL, URL pattern, or tag. Cloudflare Cache Purge, Fastly Instant Purge. Purge propagates to all edge nodes within seconds. Use for: urgent content removals (GDPR deletion requests, incorrect information). Surrogate keys (cache tags): tag a cached response with logical keys (e.g., product:12345). On a product update: purge all cached pages tagged with product:12345 in one API call. Fastly calls these “surrogate keys”; Cloudflare calls them “cache tags”. Stale-while-revalidate: serve stale content immediately while fetching fresh content in the background. User sees no latency; content updates within one request cycle.

Origin Shield and Security

Origin shield: an intermediate caching layer between edge nodes and origin. All L1 cache misses go to the origin shield (a single regional node) rather than directly to origin. If the shield has the content: serves it. If not: fetches from origin once, caches, serves to all requesting edges. Reduces origin request rate by 10-100x for popular content. DDoS protection: CDN absorbs volumetric attacks at the edge (layer 3/4: UDP floods, SYN floods). Cloudflare’s network capacity (197 Tbps) exceeds the largest recorded DDoS attacks. Layer 7 DDoS: rate limiting per IP, per ASN, per country. Challenge pages (JS challenge, CAPTCHA) for suspicious traffic. Bot detection: behavioral analysis (mouse movements, timing) to distinguish humans from bots. TLS termination at the edge: CDN terminates TLS at the nearest PoP, reducing TLS handshake RTT for the user. Uses session resumption and OCSP stapling to minimize handshake overhead. The CDN-to-origin connection can use a private network backbone (faster, more reliable than public internet).

{“@context”:”https://schema.org”,”@type”:”FAQPage”,”mainEntity”:[{“@type”:”Question”,”name”:”What is a two-tier CDN architecture and how does it reduce origin load?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”A two-tier CDN has edge nodes (L1) close to users and regional PoPs (L2) between the edges and the origin. A cache miss at L1 checks L2 before going to origin. L2 nodes serve many L1 nodes, so popular content is usually already in L2 cache. This means the origin only serves content when it is missing from both tiers, dramatically reducing origin request volume — often by 10-100x compared to single-tier CDN.”}},{“@type”:”Question”,”name”:”How do surrogate keys (cache tags) enable efficient bulk cache invalidation?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Each cached response is tagged with logical identifiers (e.g., product:12345, category:electronics). The CDN stores a tag → URL mapping. On a product update, a single API call purges all pages tagged with product:12345 — product pages, category listings, search results — in one operation. Without surrogate keys, you would need to enumerate every URL individually, which is impractical for content that appears across many pages.”}},{“@type”:”Question”,”name”:”What is stale-while-revalidate and when should you use it?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”stale-while-revalidate allows the CDN to serve a cached (stale) response immediately while fetching a fresh copy from origin in the background. The user sees no latency; the cache updates within one request cycle. Use it for content that changes infrequently but where you want near-real-time updates without the latency penalty of synchronous revalidation. Typical usage: Cache-Control: max-age=60, stale-while-revalidate=3600.”}},{“@type”:”Question”,”name”:”How does anycast routing work in a CDN context?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”The CDN advertises the same IP prefix from all of its Points of Presence (PoPs) via BGP. When a user sends a packet to that IP, internet routers automatically route it to the topologically nearest PoP based on BGP path selection. No DNS lookup is required for routing — the routing decision is made at the network layer. Anycast also provides inherent DDoS resilience: attack traffic is spread across all PoPs rather than concentrating on one.”}},{“@type”:”Question”,”name”:”What is an origin shield and when should you use it?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”An origin shield is a designated intermediate caching node between all CDN edge nodes and the origin server. All cache misses from all edge nodes go to the shield instead of directly to origin. If the shield has the content, it serves all edges from its cache. If not, it fetches from origin once and serves all requesting edges. This collapses the thundering herd problem: instead of 200 edge nodes all independently fetching the same uncached content from origin, only one request reaches origin.”}}]}