Requirements
- Generate a personalized feed of posts ranked by relevance/recency for each user
- Supports follow relationships: see posts from users you follow
- 500M users, 10M DAU, 100K new posts/minute
- Feed load time <200ms; new posts appear in feed within 5 seconds
- Mixed content: posts, images, videos, reposts
Two Approaches: Fan-out on Write vs Fan-out on Read
Fan-out on Write (push model): when a user publishes a post, immediately write it to the feed of all followers. Feed reads are fast (pre-built). Write amplification: if a user has 1M followers, 1 post = 1M feed writes. Good for users with few followers. Bad for celebrities (Justin Bieber problem).
Fan-out on Read (pull model): when a user opens their feed, fetch recent posts from each followee and merge/rank them in real time. No write amplification. Feed generation is slow (O(followees) fetches per request). Good for power users with many followees.
Hybrid (recommended): fan-out on write for normal users ( 10K followers). On feed load: merge pre-built feed (from write fan-out) + real-time fetch of recent celebrity posts.
Data Model
Post(post_id UUID, author_id, content TEXT, media_urls[], post_type ENUM(TEXT,IMAGE,VIDEO,REPOST),
repost_of UUID, created_at, like_count, comment_count, share_count)
Follow(follower_id, followee_id, created_at)
-- Index: (follower_id, created_at DESC) for "who I follow"
-- Index: (followee_id, follower_count) for follower count
FeedItem(user_id, post_id, score FLOAT, post_created_at, source ENUM(FOLLOW,RECOMMEND))
-- Stored in Redis sorted set: key=feed:{user_id}, member=post_id, score=ranking_score
Feed Generation Service
On new post published by user U:
1. Publish to Kafka topic: new_posts
2. Fan-out workers consume from Kafka:
a. Fetch follower IDs of U (from Follow table, paginated)
b. For each follower F (if follower count of U <= 10K):
ZADD feed:{F} {score} {post_id}
ZREMRANGEBYRANK feed:{F} 0 -501 # keep top 500 only
3. Score = recency_score + engagement_score + personalization_score
The feed sorted set in Redis stores post_ids ordered by score. On feed read, ZREVRANGE feed:{user_id} 0 49 fetches the top-50 post_ids in O(log n + k). Then a batch DB lookup fetches post content for those IDs.
Ranking Score
score = 0.4 * recency + 0.3 * engagement_velocity + 0.2 * relationship_weight + 0.1 * content_type_pref recency = 1 / (1 + hours_since_post) # decays over time engagement_velocity = (likes + comments*2 + shares*3) / hours_since_post relationship_weight = close_friend ? 1.5 : 1.0 # based on interaction history content_type_pref = user_affinity_for_content_type # from ML model
Pagination and Cursor
Cursor-based pagination on the feed: the client sends last_seen_score on each request. The server fetches items with score < last_seen_score. This avoids the "shifting page" problem (offset pagination breaks when new items are inserted). Store cursor in the client; the server is stateless.
New Post Delivery (5s SLA)
Fan-out workers process Kafka messages in near-real-time. For a user with 100K followers: fan-out = 100K ZADD operations. At 1ms per Redis operation, this takes ~100 seconds sequentially. Solution: parallel fan-out workers, each handling a shard of followers. With 100 parallel workers, 100K followers → 1K per worker → ~1 second total. Kafka partition by user_id for ordering; fan-out workers scale horizontally.
Key Design Decisions
- Hybrid fan-out: write for normal users, read for celebrities — eliminates write amplification spikes
- Redis sorted set for feed — O(log n) insert, O(log n + k) range read, automatic score-based ranking
- Kafka fan-out workers — async, scalable, fan-out doesn’t block the post API
- Cursor pagination — correct behavior for real-time feeds with frequent inserts
- Score-based ranking — decoupled from storage; score can be recomputed by a re-ranking job
{“@context”:”https://schema.org”,”@type”:”FAQPage”,”mainEntity”:[{“@type”:”Question”,”name”:”What is fan-out on write vs fan-out on read for news feeds?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Fan-out on write (push model): when User A posts, immediately write the post to the feed of all A's followers. Feed reads are O(1) — the feed is pre-built. Write amplification: one post → N writes where N = follower count. A user with 1M followers causes 1M Redis writes per post. Fan-out on read (pull model): when User B opens their feed, fetch recent posts from each of B's followees in real time and merge/rank them. No write amplification. Read is expensive: O(followees) DB queries per feed load, real-time merging. Hybrid (standard production approach): fan-out on write for users with < 10K followers; fan-out on read for celebrities (> 10K followers). On feed load, merge the pre-built feed (from write fan-out of normal followees) with a real-time fetch of celebrity posts. This eliminates write amplification spikes while keeping read latency low.”}},{“@type”:”Question”,”name”:”How do you implement a feed with Redis sorted sets?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Store each user's feed as a Redis sorted set: key=feed:{user_id}, member=post_id, score=ranking_score. Insert: ZADD feed:{user_id} {score} {post_id}. Trim to keep only the top 500: ZREMRANGEBYRANK feed:{user_id} 0 -501 (removes everything below the top 500 by score). Read: ZREVRANGE feed:{user_id} 0 49 returns the top 50 post_ids by score in O(log n + k). Then batch-fetch post content from DB: SELECT * FROM posts WHERE post_id IN (…). At 500M users, memory for feed sorted sets: 500M * 500 posts * ~50 bytes = ~12.5TB. In practice, only store feeds for active users (accessed in last 30 days). Evict inactive feeds with Redis TTL or LRU policy.”}},{“@type”:”Question”,”name”:”How do you rank posts in a content feed?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Feed ranking uses a scoring function combining multiple signals: (1) Recency — newer posts score higher; exponential decay: score *= exp(-lambda * hours_since_post). (2) Engagement velocity — likes + comments*2 + shares*3 per hour since posting. Viral posts with high engagement velocity score high even if slightly older. (3) Author relationship — posts from close friends (frequent interactions) score higher than casual follows. (4) Content affinity — match post type/topic to the user's historical engagement patterns (ML model). Score = w1*recency + w2*engagement + w3*relationship + w4*affinity. Weights are tuned via A/B testing. The score is computed at fan-out time and stored as the sorted set score. A periodic re-ranking job can recompute scores for posts in active feeds as engagement data accumulates.”}},{“@type”:”Question”,”name”:”How does cursor-based pagination work for infinite scroll feeds?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Offset pagination (LIMIT 20 OFFSET 40) breaks for feeds: if 5 new posts are inserted, the 41st post at query time was the 36th post when the user started scrolling. Items appear twice or are skipped. Cursor pagination: the client sends the score (or timestamp) of the last seen item. The server fetches items with score strictly less than last_seen_score: ZREVRANGEBYSCORE feed:{user_id} {last_seen_score} -inf LIMIT 0 20. Since the cursor is a score value, not an offset, new inserts above the cursor don't affect pagination below it. The cursor is opaque to the client (encode as base64 if needed). For the first page: use +inf as the cursor. Store cursor in client state; server is stateless.”}},{“@type”:”Question”,”name”:”How do you deliver new posts to feeds within 5 seconds?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Pipeline: post created → Kafka topic new_posts → fan-out worker pool. Workers consume from Kafka and execute ZADD for each follower. Latency is dominated by: Kafka produce latency (~5ms), Kafka consume latency (~10ms), fan-out time. For a user with 100K followers: 100K ZADD operations at ~0.5ms each = 50 seconds sequentially. Solution: parallel fan-out. Partition followers across N worker threads/goroutines. With 100 parallel workers: 1000 ZADDs each = ~0.5 seconds per worker = ~0.5s total. Kafka partitioning: partition by author_id so all posts from the same author go to the same partition (in-order per author). Workers are stateless and scale horizontally. At 100K posts/minute with 100K average followers: 100K * 100K = 10B ZADDs/minute — requires O(1000) Redis nodes and O(100) worker pods.”}}]}
Meta system design is the canonical news feed interview topic. See common questions for Meta interview: news feed and content ranking system design.
Twitter system design covers feed generation and ranking. Review patterns for Twitter interview: content feed and timeline system design.
Snap system design covers content feeds and story delivery. See design patterns for Snap interview: content feed and story system design.
See also: Anthropic Interview Guide 2026: Process, Questions, and AI Safety