A real-time leaderboard ranks users or entities by a score that updates continuously. The core challenge is serving low-latency rank queries at scale while accepting high-throughput score updates. The primary data structure is Redis Sorted Sets, which provide O(log N) insert and rank lookup and are the standard building block for leaderboards in gaming, e-commerce, and social applications.
Redis Sorted Set
Redis Sorted Sets store members with an associated floating-point score. Key operations: ZADD leaderboard {score} {user_id} (upsert with new score, O(log N)), ZRANK leaderboard {user_id} (rank in ascending order, 0-indexed, O(log N)), ZREVRANK leaderboard {user_id} (rank in descending order — typically what leaderboards show, O(log N)), ZREVRANGE leaderboard 0 9 WITHSCORES (top 10 with scores, O(log N + K) where K=10), and ZINCRBY leaderboard {delta} {user_id} (increment score atomically, O(log N)). All operations are atomic; no application-level locking is needed for concurrent score updates.
Score Update Architecture
Score updates flow from game events through an event queue (Kafka) to a score aggregation service. The aggregation service consumes events, applies scoring rules (kill = +100 points, assist = +25 points, death = -10 points), and calls ZINCRBY on Redis. This decouples event ingestion rate from Redis update rate. For extremely high update rates (millions of events per second), coalesce events in memory: accumulate score deltas for 100ms, then apply one ZINCRBY per user per interval. This reduces Redis write amplification at the cost of 100ms of leaderboard staleness.
Global vs Partitioned Leaderboards
A global leaderboard across millions of users requires a single sorted set with millions of members. Redis sorted sets handle tens of millions of members efficiently; memory usage is ~100 bytes per member. For 10 million users: ~1GB of Redis memory — reasonable for a single instance. Partitioned leaderboards (weekly, by region, by friend group) use separate sorted sets: leaderboard:weekly:2024-W15, leaderboard:region:NA. Expire weekly leaderboards with TTL after the week ends (retain for N weeks for historical display). Friend leaderboards: maintain a sorted set per user containing only their friends — updated on each friend’s score change (fan-out on write).
Rank Queries Near the User
Users want to see their own rank and the players just above and below them, not just the global top 10. ZREVRANK returns the user’s rank (O(log N)). To show the 5 players above and below: ZREVRANGE leaderboard (rank-5) (rank+5) WITHSCORES — a single O(log N + 10) call. Handle edge cases: users near rank 1 (cannot show 5 players above) and users near the bottom. For very large leaderboards (100M+ users), approximate rank is acceptable: maintain coarser rank buckets (top 1%, top 5%, top 10%) rather than exact positions beyond rank 10,000.
Persistence and Recovery
Redis is in-memory; a restart loses the leaderboard unless persistence is configured. Options: RDB snapshots (periodic full dump, may lose recent updates on crash), AOF persistence (log every write operation, near-zero data loss, higher disk I/O), or hybrid (RDB for fast restart, AOF for durability). For leaderboards where scores can be recomputed from an event log, losing the Redis state is recoverable: replay events from Kafka to rebuild the sorted set. Set Redis maxmemory and eviction policy to noeviction (return error when memory is full rather than evicting data) — evicting leaderboard members silently corrupts rankings.
Tie-Breaking
Redis sorted sets break ties arbitrarily (lexicographic by member name). For leaderboards where tie-breaking should favor earlier achievers (the first player to reach 1000 points ranks higher than a later player with the same score), encode the tie-breaking criterion in the score. For time-based tie-breaking: score = points * 1e12 + (MAX_TIMESTAMP – achievement_timestamp). This ensures higher points always rank above lower points, and for equal points, earlier achievers rank higher. The timestamp component must be scaled so it never exceeds the magnitude of a one-point difference (1e12 provides space for 1e12 nanoseconds ≈ 16 minutes of tie-break precision).
Serving Top-N with Caching
The top-10 (or top-100) leaderboard is read by every user but updates continuously. Cache the top-N result with a short TTL (5-30 seconds): cache_key = “leaderboard:top100”, TTL = 10s. On a read, check the cache first; on miss, execute ZREVRANGE leaderboard 0 99 WITHSCORES and populate the cache. With 100,000 concurrent users viewing the leaderboard, the cache serves 99%+ of requests from memory with single-digit millisecond latency. The 5-30 second staleness is acceptable for display — the leaderboard is still “real-time” from the user perspective while dramatically reducing Redis load.