A live leaderboard ranks users by score in real time — ranking millions of players by score in a game, tracking top contributors in a collaborative platform, or displaying real-time standings during a live event. The core challenges are: maintaining sorted order under high-frequency score updates, efficiently querying rank, and delivering updates to users watching the leaderboard without polling.
Redis Sorted Set as the Core Data Structure
Redis sorted sets (ZSETs) are purpose-built for leaderboards. Each member has a score; the set maintains sorted order with O(log n) updates and rank queries. Key operations: ZADD leaderboard {score} {user_id} — update score (or add new member). ZREVRANK leaderboard {user_id} — get 0-based rank (highest score = rank 0). ZSCORE leaderboard {user_id} — get score. ZREVRANGE leaderboard 0 99 WITHSCORES — get top 100 users with scores. ZRANGEBYSCORE with pagination: ZREVRANGEBYSCORE leaderboard +inf -inf WITHSCORES LIMIT {offset} {count}. Score update: for a game where scores only increase, use ZINCRBY leaderboard {delta} {user_id} rather than ZADD — ZINCRBY is atomic and returns the new score. At 10M users, a Redis sorted set with 8-byte user IDs and 8-byte scores uses ~300MB — fits comfortably in memory.
Partitioned Leaderboards
A single global sorted set works for up to ~100M members in Redis. For larger scale or multiple leaderboard types: partition by time window (daily, weekly, all-time are separate sorted sets). Time-windowed leaderboards reset on a schedule — daily leaderboard key: leaderboard:{YYYY-MM-DD}; EXPIRE set at end of day. Partitioned global leaderboard: shard users across N sorted sets by user_id mod N. ZADD goes to leaderboard:{user_id % N}. Getting global rank: query rank within the shard + count of users in higher-scoring shards (requires scanning all shards — expensive). Better: maintain a global sorted set for the leaderboard and use partitioning only if the single set exceeds Redis memory limits. Friend leaderboards: a user-specific sorted set containing only the user’s friends (ZADD friend_lb:{user_id} {score} {friend_id}); updated when any friend’s score changes. This requires fan-out on score update to all friends’ leaderboards — expensive for users with many friends.
Score Update Pipeline
At high update rates (millions of score events per second in a popular game), direct Redis writes from the application may saturate Redis. Batch score updates: buffer score increments in memory (or a local Redis instance) for 1-5 seconds, then batch-apply as ZINCRBY operations. This trades sub-second leaderboard freshness for throughput. Kafka pipeline: score events → Kafka → score aggregation consumer → batch ZINCRBY to Redis every 5 seconds. The consumer accumulates increments per user_id and applies them in bulk. For competitive games where exact real-time scoring matters (tournament finals): write-through directly to Redis — accept the throughput limitation and scale Redis vertically or use Redis Cluster.
Rank Approximation for Very Large Leaderboards
Exact rank in a 100M-user leaderboard requires ZREVRANK — O(log n), fast in Redis. But displaying a user’s exact rank on every page load means a Redis query per active user per page view. Optimization: cache individual user ranks with a short TTL (30-60 seconds). The rank display shows “approximately #4,523,122” — exact rank staleness is acceptable. Score distribution sampling: precompute score percentiles (top 1%, top 10%) using ZCOUNT leaderboard {min_score} +inf. Display “You’re in the top 5%” rather than an exact rank — eliminates per-user rank queries. Approximate rank: maintain a score histogram (score buckets → count of users). User rank ≈ sum of counts in all higher buckets. Update histogram asynchronously every minute. O(1) rank approximation without per-user Redis queries.
Real-Time Updates to Watching Users
Users watching the leaderboard need updates without constant polling. Server-Sent Events (SSE): the client opens a persistent HTTP connection; the server pushes leaderboard update events. For a top-100 leaderboard: broadcast updates whenever the top-100 changes (a score update causes a rank change in the top 100). A change detector compares the top-100 snapshot every second — if changed, push the new top-100 to all SSE connections. WebSockets: bidirectional but unnecessary for leaderboard — SSE is sufficient since updates flow server→client only. Polling with ETag: the client polls /leaderboard?since={timestamp}; the server returns 304 Not Modified if unchanged. Simpler than SSE, slightly higher latency. At 10,000 concurrent leaderboard viewers: SSE fan-out from a single server handles ~10,000 connections with minimal overhead. At 100,000+ viewers: use a pub/sub system (Redis Pub/Sub, Kafka) to distribute updates across multiple SSE servers.