# Low Level Design: Request Routing Service

Source: https://www.techinterview.org/post/3233470928/lld-request-routing/
Updated: 2026-06-30 · techinterview.org

## What Is a Request Routing Service?

A request routing service directs incoming HTTP requests to the appropriate upstream based on rules such as path, headers, and traffic weights. It enables canary deployments, A/B testing, and blue-green promotions without application changes.

## Route Types

- **Exact match** — `/api/v1/users` matches only that path

- **Prefix match** — `/api/v1/` matches any path starting with that prefix

- **Regex match** — `/api/v[0-9]+/users` for version-agnostic routing

Priority order: exact > prefix > regex. Within the same type, higher `priority` value wins.

## Routing Rule Table


```
routing_rules (
  id             BIGSERIAL PRIMARY KEY,
  match_type     TEXT NOT NULL,         -- exact, prefix, regex
  pattern        TEXT NOT NULL,
  method         TEXT NOT NULL,         -- GET, POST, *, etc.
  headers_match  JSONB,                 -- e.g. {X-Canary: true}
  weight         INT DEFAULT 100,       -- 0-100, used for weighted split
  upstream       TEXT NOT NULL,
  priority       INT DEFAULT 0
)
```


## Header-Based Routing

Match on arbitrary request headers to direct traffic to a specific upstream. Example: requests with `X-Canary: true` route to the canary upstream; all others route to stable. Useful for internal testing and gradual feature rollout.


```
Rule: headers_match = {X-Canary: true} -> upstream = canary
Default rule: upstream = stable
```


## Weight-Based Traffic Splitting

Assign integer weights to competing rules for the same pattern. Use weighted random selection at runtime:


```
canary rule:  weight=5  → 5% of traffic
stable rule:  weight=95 → 95% of traffic

Selection: rand(0,100) < 5 → canary, else stable
```

Weights are evaluated after header rules so explicit header overrides take precedence.

## Sticky Routing

Use consistent hashing on `session_id` or `user_id` to ensure the same user always reaches the same upstream during a split. This prevents a user from seeing inconsistent behavior when they bounce between canary and stable across requests.


```
hash(user_id) % 100 < canary_weight → canary
else → stable
```


## Traffic Mirroring

Shadow-copy requests to a test upstream asynchronously. The original request is proxied to the primary upstream and the response returned to the client immediately. The mirrored copy is sent to the shadow upstream in a fire-and-forget goroutine/thread. Shadow responses are discarded — useful for validating new service behavior under real traffic.

## Route Evaluation Algorithm

On each request:

- Check exact match rules — O(1) via hash map

- Check prefix rules — use a trie for efficient longest-prefix match

- Check regex rules — iterate sorted by priority, short-circuit on first match

- Apply header conditions on matched rules

- Apply weighted selection if multiple rules remain

## Blue-Green Promotion


```
Phase 1: blue=100, green=0    (green deployed, no traffic)
Phase 2: blue=50,  green=50   (validation)
Phase 3: blue=0,   green=100  (full cutover)
Rollback: set blue=100, green=0 instantly
```

Because weight changes are propagated via Redis pub/sub (see below), rollback takes effect in under 1 second across all gateway instances.

## Upstream Health Integration

The router only selects upstreams that are currently marked healthy by the health check service. If all weighted upstreams for a rule are unhealthy, return `503 Service Unavailable` rather than routing to a known-bad instance.

## Route Change Propagation


```
Admin API → DB write → Redis pub/sub publish
  → all gateway instances subscribe → in-memory reload (<1s)
```

No rolling restart required. Propagation latency is typically under 100ms on a local network.

## Summary

A request routing service enables safe, gradual deployments through exact/prefix/regex matching, header overrides, weighted splits, and consistent hashing for stickiness. Hot-reloadable rules via Redis pub/sub make blue-green promotion and rollback near-instant.

See also: [Scale AI Interview Guide 2026: Data Infrastructure, RLHF Pipelines, and ML Engineering](https://www.techinterview.org/post/3233460275/scale-ai-interview-guide-2026-data-infrastructure-rlhf-pipelines-and-ml-engineering/)

See also: [Anthropic Interview Guide 2026: Process, Questions, and AI Safety](https://www.techinterview.org/?p=3233460282)

See also: [Netflix Interview Guide 2026: Streaming Architecture, Recommendation Systems, and Engineering Excellence](https://www.techinterview.org/post/3233460274/netflix-interview-guide-2026-streaming-architecture-recommendation-systems-and-engineering-excellence/)