A/B Test Assignment Service Low-Level Design: Consistent Hashing, Experiment Namespaces, and Exposure Logging

A/B Test Assignment Service: Low-Level Design

An A/B test assignment service determines which experiment variant a user sees and guarantees that assignment is stable, consistent, and auditable. The core requirements are: the same user always gets the same variant, experiments do not interfere with each other, and every assignment is logged for metric analysis.

Assignment Algorithm

Variant assignment uses a deterministic hash function:

bucket = hash(user_id + experiment_id) mod 100  // bucket in [0, 99]

Each variant in the experiment owns a range of buckets. For a 50/50 split: control owns buckets 0–49, treatment owns 50–99. For a 10% test: treatment owns 0–9, control owns 10–99. Because the hash is deterministic, the same user always maps to the same bucket for a given experiment — as long as the experiment config does not change.

Recommended hash function: MurmurHash3 or FNV-1a. Avoid MD5/SHA — they are slower with no benefit for this use case.

Experiment Schema

experiment_id — unique identifier (UUID or slug)
name — human-readable label
status — draft / running / paused / concluded
variants — list of {name, traffic_pct}; percentages must sum to 100 within the experiment's allocated traffic
traffic_pct — percentage of all eligible users entering the experiment (e.g., 10% of users see this experiment at all)
start_date / end_date — scheduling boundaries
targeting_rules — conditions for eligibility (logged-in only, country, app version, etc.)
layer_id — the namespace/layer this experiment belongs to

Namespace and Layer Isolation

Experiments are organized into layers (also called namespaces) to control overlap:

Same layer — experiments are mutually exclusive. A user can only be assigned to one experiment per layer. Useful when experiments modify the same UI component and would conflict if combined.
Different layers — experiments are independent. A user can be in one experiment from layer A and one from layer B simultaneously. Useful for testing orthogonal features (e.g., checkout flow in layer 1, recommendation algorithm in layer 2).

Within a layer, the hash bucket space is partitioned between experiments. Experiment A owns buckets 0–19, experiment B owns 20–39, buckets 40–99 are unallocated (users see default experience).

Traffic Allocation

Two levels of traffic control:

Experiment traffic — what percentage of all eligible users enter the experiment at all (e.g., 10%)
Variant split — within the experiment, how traffic is divided between variants (e.g., 50% control, 50% treatment)

The sum of all experiment traffic allocations within a layer must not exceed 100%. A validation check enforces this at experiment creation time.

Holdout Groups

A permanent holdout is a fixed 1% of users who never see any new features regardless of experiments. They serve as a long-term control group to measure the cumulative impact of all shipped features over months. Holdout users are excluded before any experiment assignment logic runs.

Exposure Logging

An exposure event is logged when a user is first assigned to a variant — not on every request:

Fields: user_id, experiment_id, variant, timestamp, client_context (platform, app version, country)
Deduplication: Redis set tracks user_id:experiment_id pairs seen in the last 24 hours to prevent duplicate exposure logs
Logging is asynchronous — written to Kafka, then consumed by the metrics pipeline
Only log exposure when the user actually encounters the experimental treatment, not at assignment time if the feature is gated behind a condition

Targeting Rules

Before hashing, the service evaluates eligibility rules:

Authentication state — only logged-in users
Country — experiment only runs in US and CA
App version — minimum version 4.2.0 required for a mobile feature
Custom segments — user is in “power user” cohort

Rules are evaluated against user context passed in the assignment request. Ineligible users receive the default experience and are not logged.

Assignment Caching

Computing assignments for all active experiments on every request is cheap but not free. Assignments are cached in Redis per user with a 5-minute TTL:

Key: assignments:{user_id}
Value: JSON map of {experiment_id: variant_name}
TTL: 300s

On experiment config change, the cache for affected users is invalidated. For high-throughput services, the SDK pre-fetches and caches assignments client-side.

SDK Design

The client SDK (mobile, web, backend) fetches the full experiment config once at startup and evaluates assignments locally:

Config is fetched from a CDN-backed endpoint and cached with a 5-minute stale-while-revalidate TTL
Assignment hash computation runs in-process — no network call per assignment
Exposure events are batched and sent asynchronously to avoid blocking the request path
Fallback: if config fetch fails, all experiments return the control variant

Summary

The assignment service combines deterministic hashing for stable variant selection, layer-based mutual exclusion for experiment isolation, targeting rules for eligibility, async exposure logging for metric analysis, and client-side SDK evaluation for low-latency assignment — forming the foundation of a reliable experimentation platform.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “How does consistent hashing ensure stable user-to-variant assignment?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “The assignment function computes hash(experiment_id + user_id) modulo 100 to produce a deterministic bucket in [0, 100), which is then mapped to a variant by comparing against cumulative allocation thresholds (e.g., 0–49 = control, 50–99 = treatment). Because the hash is purely a function of its inputs and requires no stored state, the same user always lands in the same bucket regardless of which server handles the request.”
}
},
{
“@type”: “Question”,
“name”: “What is experiment namespace isolation and why is it needed?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Namespace isolation partitions the user population into non-overlapping slices (namespaces), each of which can host one experiment at a time, preventing two experiments from simultaneously modifying the same user's experience and confounding each other's results. Within a namespace, experiments use orthogonal hash functions or sub-bucketing so that users assigned to one experiment's treatment are independent of assignments in other layers.”
}
},
{
“@type”: “Question”,
“name”: “How is exposure logging deduplicated?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Exposure events are written to a stream (e.g., Kafka) with a composite key of (experiment_id, user_id, variant); a deduplication operator in the stream processor (using a Bloom filter or RocksDB-backed set keyed on this composite) drops duplicate exposures within a sliding window, ensuring each user is counted at most once per experiment in the metrics computation. Idempotency keys on the downstream sink table provide a second dedup layer.”
}
},
{
“@type”: “Question”,
“name”: “How does a holdout group interact with the A/B assignment system?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “A holdout group is carved out before experiment assignment by reserving a fixed bucket range (e.g., hash % 100 < 5) that is excluded from all experiments and remains on the baseline product experience. The A/B assignment system checks membership in the holdout namespace first and short-circuits further assignment, ensuring holdout users never receive any treatment and serve as a clean long-term control for measuring cumulative product impact.”
}
}
]
}