Low Level Design: Ranking System – Tech Interview Dot Org

What Is a Ranking System?

A ranking system assigns ordinal positions to entities (users, products, documents, players) based on one or more scoring signals. Unlike a simple leaderboard, a ranking system may apply composite scoring, decay functions, tie-breaking rules, and personalization. It is foundational to search engines, recommendation feeds, matchmaking, and competitive gaming.

Data Model

Scores are multi-dimensional. A score record captures the raw signals before normalization:

CREATE TABLE entity_scores (
    entity_id    BIGINT NOT NULL,
    namespace    VARCHAR(64) NOT NULL,       -- e.g. 'search', 'game', 'product'
    signal       VARCHAR(64) NOT NULL,       -- e.g. 'clicks', 'wins', 'revenue'
    raw_value    DOUBLE PRECISION NOT NULL,
    recorded_at  TIMESTAMP NOT NULL,
    PRIMARY KEY (entity_id, namespace, signal, recorded_at)
);

CREATE TABLE composite_ranks (
    entity_id    BIGINT NOT NULL,
    namespace    VARCHAR(64) NOT NULL,
    composite    DOUBLE PRECISION NOT NULL,
    rank_pos     INT NOT NULL,
    computed_at  TIMESTAMP NOT NULL,
    PRIMARY KEY (entity_id, namespace)
);

Redis stores the live ranking surface:

ZADD rank:search 0.9821 entity:77
ZADD rank:search 0.8734 entity:42
ZREVRANK rank:search entity:77   -- real-time position

Core Algorithm and Workflow

Composite Score Computation

A weighted linear combination is the simplest approach:

composite = w1 * normalize(signal1) + w2 * normalize(signal2) + ... + wN * normalize(signalN)

Signals are normalized to [0,1] using min-max scaling within the namespace. Weights are configured per namespace and updated via an admin API without code deploys.

Decay Functions

For recency-sensitive rankings (news, trending), apply exponential decay:

decayed = raw_value * exp(-lambda * hours_since_event)

Lambda controls the half-life. A batch job recomputes decayed scores hourly and pushes updates to Redis.

Rank Assignment

Batch scorer reads entity signals from the data warehouse (Spark job, hourly or daily).
Computes composite scores, resolves ties by secondary key (e.g., entity_id ascending for determinism).
Writes composite scores and rank positions to composite_ranks.
Bulk-loads Redis sorted sets via ZADD pipeline.

Real-Time Rank Updates

For latency-sensitive signals (live game score, auction bid), a streaming pipeline (Kafka + Flink) recomputes the affected entity's composite score on each event and issues a targeted ZADD update to Redis.

Failure Handling and Consistency

Stale ranks: If the batch job fails, Redis retains the last successful ranking. An SLA on batch freshness (e.g., ranks no older than 2 hours) triggers an alert and fallback to the previous snapshot.

Partial updates: Bulk Redis loads use pipelining inside a Lua script or a Redis transaction (MULTI/EXEC) per shard to prevent partially-updated sorted sets from serving traffic. A swap-key pattern (write to rank:search:next, then RENAME) ensures atomic promotion.

Signal outage: If a signal source goes offline, the scorer uses the last known value with a staleness flag. Downstream API responses annotate ranks as approximate when signals are stale beyond a threshold.

Scalability Considerations

Namespace isolation: Each ranking namespace (search, products, players) lives in its own Redis keyspace and can be sharded independently to separate Redis clusters based on traffic profile.

Approximate top-K: For very large entity sets (>100M), maintaining a full sorted set is expensive. Use a two-tier approach: a sample-based sketch (Count-Min Sketch or reservoir sampling) identifies top-K candidates; only those are promoted to the full sorted set for precise ranking.

Batch vs. streaming tradeoff: Batch scoring is cheaper and simpler but introduces lag. Streaming scoring is lower latency but operationally complex. A lambda architecture runs both and merges results, with streaming scores overriding batch for recently-updated entities.

Caching read results: Top-N rank pages are cached in a CDN or application cache with short TTLs (5-30 seconds). This absorbs read spikes without additional Redis load.

Summary

A ranking system extends a simple leaderboard with multi-signal composite scoring, decay functions, and configurable weights. Batch pipelines handle the bulk of computation; streaming layers provide low-latency updates for hot signals. Redis sorted sets serve the ranking surface, refreshed atomically via swap-key patterns. Namespace sharding and approximate top-K structures keep the system scalable to hundreds of millions of entities.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “What is a ranking system in system design?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “A ranking system is a service that assigns an ordered position to items — such as search results, job postings, or social feed content — based on a scoring function. It combines signals like relevance, recency, engagement, and personalization to produce a ranked list served to users in low-latency queries.”
}
},
{
“@type”: “Question”,
“name”: “How does a ranking system differ from a recommendation system?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “A recommendation system generates a candidate set of items a user might like, while a ranking system orders a given candidate set by predicted relevance or utility. In practice they are often pipelined: a retrieval stage produces candidates, and a ranking stage applies a heavier scoring model (e.g., a gradient-boosted tree or neural network) to reorder them before serving.”
}
},
{
“@type”: “Question”,
“name”: “How do you design a ranking system that handles billions of items?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “For billion-scale ranking, a multi-stage funnel is used. An approximate nearest-neighbor (ANN) index (e.g., Faiss or ScaNN) narrows billions of items to thousands of candidates in milliseconds. A lightweight scorer reduces candidates to hundreds, and a full ranking model (with features fetched from a feature store) scores the final set. Results are cached by query signature to reduce latency.”
}
},
{
“@type”: “Question”,
“name”: “What are common interview questions about ranking system design?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Interviewers at Google, LinkedIn, and Meta commonly ask: How would you design LinkedIn’s job ranking system? How do you incorporate real-time engagement signals into ranking? How do you A/B test a new ranking model without degrading user experience? How do you prevent position bias from corrupting your training data?”
}
}
]
}