Low Level Design: Reverse Proxy – Tech Interview Dot Org

Core Functions

A reverse proxy sits between clients and upstream servers, providing: TLS termination, request routing, upstream connection pooling, response caching, compression, rate limiting, and DDoS mitigation.

TLS Termination

The proxy decrypts incoming TLS and forwards plain HTTP to upstreams. Certificates rotate without downtime via SNI — multiple certs loaded simultaneously; the correct one selected per hostname at handshake time.

HTTP/2 to HTTP/1.1 Translation

Client HTTP/2 streams are multiplexed over a single connection. The proxy fans these out to upstream HTTP/1.1 connections drawn from a per-upstream pool, bridging protocol versions transparently.

Connection Pool

Per-upstream pool:
  max_connections = 100
  idle_timeout    = 60s
  health_check    = TCP connect every 10s on idle connections

Connection reuse eliminates per-request TCP and TLS handshake overhead to upstreams.

Request Buffering

The proxy buffers the complete request body before forwarding to the upstream. This decouples slow clients from upstream workers, preventing slow-loris-style resource exhaustion.

Response Streaming

Large responses are streamed directly to the client without full buffering at the proxy, reducing Time to First Byte (TTFB) and memory pressure.

Compression

Gzip and Brotli compression applied when the client sends Accept-Encoding and the response content type is compressible (text/html, application/json, etc.). Skipped for already-compressed formats (images, video).

DDoS Mitigation

Rate limit by IP + User-Agent combination.
Redirect suspicious traffic to a CAPTCHA challenge page.
Block known bad IPs via a continuously updated IP blocklist.

Header Manipulation

The proxy can add, remove, or rewrite both request headers (before forwarding) and response headers (before returning to client). Common uses: inject X-Request-ID, strip internal headers, set Strict-Transport-Security.

Web Application Firewall

Requests are inspected for SQLi, XSS, and other OWASP Top 10 patterns before forwarding. Matched requests are blocked or flagged for logging.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “What is the difference between a reverse proxy and a load balancer?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “A load balancer's primary function is distributing traffic across backend instances. A reverse proxy is broader: it also handles TLS termination, request/response header manipulation, caching, compression, WAF inspection, and DDoS mitigation — and may include load balancing as one of its features.”
}
},
{
“@type”: “Question”,
“name”: “How does a reverse proxy handle HTTP/2 and HTTP/1.1 backends?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “The reverse proxy accepts multiplexed HTTP/2 streams from clients over a single connection and fans them out to upstream HTTP/1.1 connections drawn from a per-upstream connection pool. This bridges the two protocol versions transparently and reuses upstream connections to avoid repeated TCP and TLS handshake overhead.”
}
},
{
“@type”: “Question”,
“name”: “Why does a reverse proxy buffer request bodies?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Buffering the complete request body before forwarding decouples slow clients from upstream workers. Without buffering, a slow client sending a large upload would hold an upstream connection open for the entire duration, enabling slow-loris-style resource exhaustion attacks.”
}
},
{
“@type”: “Question”,
“name”: “How does a reverse proxy mitigate DDoS attacks?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “A reverse proxy can rate-limit requests by IP and User-Agent, redirect suspicious traffic to a CAPTCHA challenge, and block known malicious IPs via a continuously updated blocklist. Combined with WAF inspection for application-layer attacks (SQLi, XSS), these layers absorb most volumetric and application-layer DDoS traffic before it reaches upstream services.”
}
}
]
}