Request Router Low-Level Design: Path Matching, Weighted Routing, and Traffic Splitting

What a Request Router Does

A request router sits between clients and backend services, inspecting each incoming HTTP request and forwarding it to the correct upstream based on path, headers, weights, or hash-based rules. Getting this right requires efficient matching, deterministic traffic splitting, and zero-downtime config updates.

Path Matching with a Radix Trie

Storing routes in a radix trie (compressed prefix tree) gives O(k) lookup where k is the path length, regardless of how many routes are registered. A path like /api/v1/users/{id} is stored with the segment {id} marked as a parameter capture node. When /api/v1/users/123 arrives, the trie traversal extracts id=123 as a path parameter.

Wildcard and catch-all nodes (*, **) live at lower priority in the trie. More specific static segments always match before parameter nodes, and parameter nodes match before wildcards.

Route Priority

Specificity wins. When multiple routes could match a request, the router applies this ordering:

Exact static match: /api/v1/users/me
Parameterized match: /api/v1/users/{id}
Prefix wildcard: /api/v1/**

This ensures /api/v1/users/me is never accidentally routed as a user lookup for ID “me”.

Weighted Round-Robin to Multiple Upstreams

Each route can define a set of upstream targets with weights:

upstreams:
  - url: https://stable.svc
    weight: 9
  - url: https://canary.svc
    weight: 1

The router keeps an atomic counter per route. On each request: target = upstreams[counter % sum_of_weights], increment counter. The weight array is expanded — weight 9 and 1 becomes a 10-slot array with 9 stable entries and 1 canary entry. Atomic increment ensures thread safety without locks.

Header-Based Routing

Routes can match on request headers before falling through to path-based routing:

X-Beta-User: true → canary backend
User-Agent contains Mobile → mobile-optimized backend
Accept-Language: fr → French-locale service

Header rules are evaluated in declaration order. The first matching rule wins. Header routing is evaluated before weighted selection, allowing targeted overrides.

Sticky Routing for Session Affinity

When upstream state is not fully replicated, sticky routing pins a client to the same upstream. The router hashes session_id or user_id from the cookie or header using a consistent hash ring. The same key always maps to the same upstream node. When an upstream is removed, only the keys it owned are redistributed — minimal disruption.

Traffic Splitting for A/B and Canary

Deterministic user assignment: bucket = crc32(user_id) % 100. Users with bucket < 10 go to the canary, the rest to stable. This is sticky across requests without storing state — the same user_id always hashes to the same bucket. Traffic split percentages can be changed in config and take effect on the next request after hot reload.

Upstream Health Checking

A background goroutine (or thread) probes each upstream every 5 seconds with a lightweight HTTP GET to a /healthz endpoint. On failure, the upstream is marked unhealthy and removed from the routing pool atomically. After N consecutive successful health checks (e.g., 3), the upstream is re-added. The routing table is a pointer swap — reads never block on health check writes.

Hot Config Reload

The router watches its config source (file inotify event or polling a config API) for changes. On change:

Parse and validate the new config in a separate data structure
Build the new radix trie and upstream pools
Atomically swap the active routing table pointer
In-flight requests continue using the old table until they complete

No restart required. Zero dropped connections.

Per-Route Timeouts and Middleware Chain

Different upstreams have different SLAs. A fast internal API might have a 200ms timeout; a slow data export endpoint gets 30 seconds. Timeouts are configured per route and enforced via context cancellation.

Each request passes through a middleware chain before reaching the proxy step:

auth → rate_limiter → route_match → upstream_select → proxy → response

Middleware failures short-circuit the chain and return the appropriate HTTP error code.

Per-Route Metrics

The router emits labeled metrics for every route:

router_requests_total{route="...", status="2xx"}
router_latency_seconds{route="...", quantile="0.99"}
router_upstream_errors_total{route="...", upstream="..."}

Route name labels enable per-endpoint dashboards and SLA alerting without a separate observability service.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “How does a radix trie match URL paths efficiently?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “A radix trie (compressed prefix tree) stores route segments as shared edge prefixes so the router traverses only one node per path segment rather than scanning all registered routes linearly; parameter segments (e.g., :id) and wildcards are stored as special node types that match any value at their position. Lookup time is O(k) where k is the number of path segments, independent of the total number of registered routes.”
}
},
{
“@type”: “Question”,
“name”: “How does weighted round-robin implement traffic splitting?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Each backend is assigned an integer weight; the router maintains a current-weight counter per backend and on each selection increments all counters by their weight, picks the backend with the highest current-weight, then decrements that backend's counter by the sum of all weights — this is the Nginx smooth weighted round-robin algorithm, which distributes load evenly without bursts. A weight of 10 vs 90 across two backends routes approximately 10% and 90% of traffic respectively.”
}
},
{
“@type”: “Question”,
“name”: “How is hot config reload implemented without dropping requests?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “The router stores its routing table behind an atomic pointer (e.g., a sync/atomic value or RCU-style structure); on receiving a SIGHUP or a config-change notification the router parses and validates the new config in a goroutine or background thread, then atomically swaps the pointer so new requests immediately use the new table while in-flight requests complete against the old one. Zero connections are dropped because no listener socket is closed during the swap.”
}
},
{
“@type”: “Question”,
“name”: “How does consistent hashing provide sticky routing?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Backends are mapped to many virtual nodes on a hash ring by hashing their addresses with multiple seeds; a request's key (session ID, user ID, or IP) is hashed to a point on the ring and routed to the first virtual node clockwise from that point, which maps back to a real backend. When a backend is added or removed only the keys that hash to its ring segment are remapped — O(K/N) keys — rather than rehashing the entire keyspace as with modulo hashing.”
}
}
]
}