Thumbnail Service Low-Level Design: On-Demand Generation, URL-Based Params, and Cache Hierarchy

URL Scheme and Parameter Design

A thumbnail service exposes a clean HTTP API for on-demand image transformation. The URL encodes all transformation parameters:

/thumbnail?url=s3://bucket/originals/photo.jpg&w=300&h=200&fit=cover&format=webp

Parameters:

url — source image location (S3 key or signed URL)
w, h — target width and height in pixels
fit — resize mode (cover, contain, fill, inside, outside)
format — output format (webp, avif, jpeg, png)
q — quality (1–100, default 80)

Critically, dimension values must be whitelisted to a set of allowed sizes (e.g., 100, 200, 300, 400, 600, 800, 1200). Accepting arbitrary dimensions creates a DoS vector: an attacker can generate millions of unique URLs with slightly different dimensions, each requiring a separate cache entry and generation pass.

Fit Modes

Different use cases require different resize behaviors:

cover: resize to fill the target dimensions, cropping excess. Output is always exactly w×h. Used for thumbnails in grids where consistent size is required.
contain: resize to fit within target dimensions, preserving aspect ratio. Adds letterbox/pillarbox padding to fill. Used when full image must be visible.
fill: stretch to exactly w×h ignoring aspect ratio. Rarely used.
inside: resize so image fits within w×h, no upscaling. Output may be smaller than requested.
outside: resize so image covers w×h, no downscaling. Output may be larger than requested.

Format Selection and Modern Formats

Output format has a significant impact on file size and browser compatibility:

WebP: ~30% smaller than JPEG at equivalent visual quality. Supported in all modern browsers (Chrome, Firefox, Safari 14+, Edge).
AVIF: ~50% smaller than JPEG, even better than WebP. Supported in Chrome 85+, Firefox 93+, Safari 16+. Encoding is slower.
JPEG: fallback for older browsers and non-photo content. Still required for Safari < 14.
PNG: lossless, for images requiring transparency without WebP support.

Best practice: use the Accept header to detect browser capabilities and serve WebP/AVIF when supported, JPEG otherwise. This can be handled at the CDN layer (Cloudflare Image Resizing, AWS CloudFront Functions) or in the service itself.

Cache Key Design

The cache key must be a deterministic, collision-free identifier for a specific (source, parameters) combination. Compute a hash of the normalized parameters:

cache_key = SHA256(canonical_url + ":" + w + ":" + h + ":" + fit + ":" + format + ":" + q)

Normalize before hashing: sort query parameters alphabetically, lowercase format strings, clamp quality to allowed range. The S3 output path becomes /thumbnails/{first2_hex}/{cache_key}.{format} — the two-character prefix distributes keys across S3 partitions.

Generation Flow

Request flow on a cache miss:

CDN edge: cache miss → forward request to origin.
Origin service: compute cache key → check Redis (L1 cache) for existence flag.
If Redis hit: redirect to S3 URL (already generated).
If Redis miss: check S3 for existing file at /thumbnails/{cache_key}.{format}.
If S3 miss: download source image from S3, process (resize + format convert), upload result to S3, set Redis existence flag with long TTL.
Return generated image with long-lived cache headers.

Image processing uses libvips (Node.js: Sharp, Python: pyvips) — significantly faster than PIL/Pillow for resize operations due to streaming pipeline architecture and demand-driven processing.

CDN Edge Caching

Generated thumbnails are immutable — a given (source, params) combination always produces the same output. Use content-addressed URLs and set aggressive cache headers:

Cache-Control: public, max-age=31536000, immutable

One year TTL. The CDN caches the thumbnail at edge for effectively forever. When source images change (user re-uploads), a new source URL is generated (new S3 key with UUID), which produces a new cache key — old thumbnails naturally expire.

Security: Signed URLs

An unsigned thumbnail service is vulnerable to abuse: hotlinking (embedding your service's URLs in external sites), scraping (systematic generation of all permutations), and resource exhaustion. Protect with signed URLs:

/thumbnail?url=...&w=300&h=200&fit=cover&format=webp&sig=HMAC(secret, params)

The service validates the HMAC signature before processing. Signatures can be scoped to a specific domain, user session, or expiry time. For public content, use a shared secret; for user-uploaded private content, sign per-request on the server side.

Lazy vs Eager Generation

Lazy (on-demand): generate thumbnails only when first requested. Lower storage cost (only generate what's actually needed), but first-request latency is high. Good for long-tail content.
Eager (pre-generate): generate all standard thumbnails immediately after upload. Zero first-request latency, but wastes compute and storage on content that may never be viewed. Good for high-traffic content (product images, featured articles).

Hybrid: eager generation for a small set of canonical sizes (e.g., 300×200 cover WebP) at upload time; lazy generation for all other size/format combinations.

Resource Limits

Max source image size: reject source images above a limit (e.g., 50MB, 10000×10000 pixels) to prevent memory exhaustion during decode.
Max output dimensions: cap w and h at a maximum (e.g., 4000px) regardless of requested values.
Concurrency limits: image processing is CPU-intensive; limit concurrent generation jobs per pod/instance and queue excess requests.

Summary

URL-parameter API with whitelisted dimension values to prevent DoS.
Deterministic cache key via SHA256 of normalized parameters.
Generation pipeline: CDN miss → Redis check → S3 check → libvips process → S3 store.
WebP/AVIF for 30–50% size reduction; JPEG fallback for old browsers.
Immutable CDN cache with 1-year TTL and content-addressed URLs.
HMAC-signed URLs to prevent hotlinking and abuse.
Hybrid eager/lazy generation; resource limits on source and output dimensions.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “How is a thumbnail URL designed to be cache-friendly?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “A cache-friendly thumbnail URL encodes all transformation parameters (width, height, format, quality) as a stable path or query string derived deterministically from the source asset's content hash and the requested dimensions, so the same logical thumbnail always maps to the same URL and CDN cache key. Avoid including timestamps or session tokens in the URL, as these create cache misses for every request even when the image and parameters have not changed.”
}
},
{
“@type”: “Question”,
“name”: “How does on-demand thumbnail generation work with CDN caching?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “On a CDN cache miss the request is forwarded to an origin thumbnail service that fetches the source image from object storage, resizes and encodes it, streams the result to the client, and sets a long Cache-Control max-age (e.g., 1 year for content-addressed URLs). Subsequent requests for the same URL are served entirely from the CDN edge without touching the origin, so the origin only bears the cost of first-time generation per unique (image, dimension, format) triple.”
}
},
{
“@type”: “Question”,
“name”: “How are thumbnail generation requests protected against DoS via unique parameter combinations?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Thumbnail URLs are cryptographically signed with an HMAC over the source image key and the allowed parameter set, and the origin service verifies the signature before processing; this prevents attackers from requesting arbitrary dimension combinations that would exhaust CPU and storage resources. Alternatively, the service restricts dimensions to a discrete allowlist (e.g., 100, 200, 400, 800 px) so that the total number of valid cache entries per source image is bounded.”
}
},
{
“@type”: “Question”,
“name”: “How does WebP format reduce thumbnail file size?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “WebP uses a block-based predictive coding scheme derived from VP8 for lossy compression and a combination of LZ77, Huffman coding, and color cache for lossless compression, achieving 25–35% smaller file sizes than JPEG at equivalent perceptual quality (SSIM) for photographic thumbnails. For thumbnails with transparency (product images on white backgrounds), WebP lossless replaces PNG with similar quality and 20–30% smaller payloads, reducing both egress costs and Time to First Byte on image-heavy pages.”
}
}
]
}