Image Optimization Service Low-Level Design: On-the-Fly Transforms, Format Selection, and Perceptual Quality

URL Scheme

An image optimization service accepts transform parameters via URL and returns a processed image. A clean URL scheme makes this easy to integrate:

/img/{source_path}?w=800&h=600&q=85&fmt=auto&fit=cover

Parameters: w (width), h (height), q (quality 1-100), fmt (output format), fit (resize mode). The source_path maps to an object key in your S3 source bucket.

Format Negotiation

When fmt=auto, the service inspects the Accept request header to determine the best format the client supports:

Accept: image/avif present → serve AVIF
Accept: image/webp present → serve WebP
Fallback → JPEG for photos, PNG for images with transparency

This negotiation happens at the service level, not in client code. CDN must be configured to vary cache on the Accept header (or use separate cache keys per format) to avoid serving WebP to browsers that requested JPEG.

Format File Size Comparison

For equivalent perceptual quality on a typical photo:

JPEG: ~100 KB
WebP: ~70 KB (30% smaller)
AVIF: ~50 KB (50% smaller)

AVIF offers the best compression but encoding is CPU-intensive. Cache processed results aggressively — encoding once and serving many times is the correct tradeoff.

Quality and Perceptual Optimization

Fixed quality (q=85) is the simple approach: works well for JPEG and WebP. A more sophisticated approach targets a perceptual quality score:

Encode at decreasing quality levels, measure SSIM (Structural Similarity Index) against the original after each pass.
Stop when SSIM drops below threshold (e.g., 0.95).
Result: complex images get higher quality setting automatically; simple images get lower, saving bytes.

Tools: libvips for fast image processing, sharp (Node.js wrapper), or Pillow (Python). For AVIF, libavif via cavif.

Resize Operations

fit=cover: Scale and crop to exactly w×h. Fills the frame, may clip edges. Most common for thumbnails.
fit=contain: Scale to fit within w×h, preserving aspect ratio. Adds padding if needed.
fit=scale-down: Like contain, but never upscales. Returns original dimensions if smaller than target.

Never upscale raster images. Upscaling wastes bandwidth and looks worse than the original. Enforce this with fit=scale-down as the safe default.

Image Processing Pipeline

Parse and validate URL parameters.
Check signature (HMAC validation — see below).
Check processed image cache in S3 (keyed by hash of source path + params).
Cache hit: redirect to cached URL or stream directly.
Cache miss: fetch source image from S3.
Decode source → apply resize → convert format → encode.
Store processed result in S3 under /processed/{param_hash}/{source_key}.
Return processed image with appropriate cache headers.

Signed URLs

Without signing, anyone can request arbitrary transforms: ?w=9999&h=9999 to DoS your encoding fleet. Sign URLs with HMAC-SHA256 over the path and parameters:

sig = HMAC-SHA256(secret, "/img/photo.jpg?w=800&h=600&q=85")
URL = /img/photo.jpg?w=800&h=600&q=85&sig=abc123def456

Service validates signature before processing. Invalid or missing signature returns 403. Signing key rotates periodically; support two valid keys during rotation window.

Lazy vs Eager Processing

Lazy (on-demand): Generate on first request, cache result. Simple to operate; cold requests are slow.
Eager (pre-generation): At upload time, generate a defined set of common sizes (thumbnail, medium, large, retina variants). Eliminates first-request latency for known sizes.

In practice: eager-generate the critical sizes (hero image, thumbnail), lazy-generate everything else.

Special Cases

SVG: Vector format — pass through directly without processing. No rasterization needed.
Animated GIF: Convert to WebP (animated) or MP4 for significant size reduction. MP4 is not an image format but browsers handle <video autoplay muted loop> well.
Max resolution limits: Reject requests where w > 4096 or h > 4096. Prevents decompression bomb attacks on large source images.
EXIF stripping: Remove EXIF metadata before serving (privacy: GPS coordinates, device info).

CDN Caching

Processed images are served behind a CDN with long-lived cache headers. Cache key must include all transform parameters and the negotiated format. Use Vary: Accept header so CDN maintains separate cache entries per format. Once a processed image is cached at the edge, subsequent requests never hit the processing service.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “How does content negotiation select the optimal image format?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “The optimizer inspects the client's Accept header; if it includes image/avif or image/webp, the service encodes and returns the best supported format in priority order, falling back to the original format (JPEG/PNG) for browsers that advertise neither. The Vary: Accept response header ensures CDN caches store format variants separately per client capability.”
}
},
{
“@type”: “Question”,
“name”: “How are signed URLs used to prevent image optimization abuse?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “The application server generates a signed URL by appending an HMAC of the path and transformation parameters (width, height, quality) using a secret key, so the optimizer can verify any incoming request has been explicitly authorized before performing CPU-intensive transcoding. Without a valid signature the optimizer returns 403, preventing attackers from enumerating arbitrary resize parameters and exhausting compute resources.”
}
},
{
“@type”: “Question”,
“name”: “How does stale-while-revalidate work for optimized images?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “The CDN or cache layer serves the cached (stale) image immediately to the user while asynchronously dispatching a background revalidation request to the optimizer, so the user never waits for re-encoding even when the cached entry has expired. The next user after the background refresh completes receives the freshly optimized version without any added latency.”
}
},
{
“@type”: “Question”,
“name”: “What is the file size improvement from WebP vs JPEG at equivalent quality?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Google's WebP codec typically achieves 25–35% smaller file sizes than JPEG at perceptually equivalent quality levels, owing to its use of block prediction, adaptive quantization, and an arithmetic entropy coder instead of JPEG's Huffman coding. AVIF pushes further still — often 40–50% over JPEG — by leveraging the AV1 intra-frame codec, though encoding is significantly more CPU-intensive.”
}
}
]
}