Core Entities and Fan-Out Strategies
Post: post_id, author_id, content, media_urls (array), created_at, like_count, comment_count, share_count. Follow: follower_id, followee_id, created_at. FeedItem: feed_id, user_id, post_id, score (ranking), inserted_at. (Materialized per-user feed.) Two fan-out strategies: Fan-out on write (push): when a user posts, immediately insert the post into the feed of all followers. Fast reads (pre-computed feed), slow writes (N followers = N inserts). Best for users with few followers. Fan-out on read (pull): when a user loads their feed, query the posts of all accounts they follow, sort by time. Slow reads, fast writes. Best for high-follower accounts (celebrities). Hybrid: fan-out on write for users with < 10K followers, fan-out on read (merge at read time) for celebrities. This is Instagram/Twitter's approach.
Feed Generation
class FeedService:
MAX_FEED_SIZE = 1000 # max items stored per user in Redis
def get_feed(self, user_id: int, page: int = 0,
page_size: int = 20) -> list[dict]:
# 1. Get pre-computed feed from Redis (sorted set by score)
start = page * page_size
end = start + page_size - 1
post_ids = self.redis.zrevrange(
f"feed:{user_id}", start, end, withscores=True
)
if len(post_ids) list:
# Pull model: join follows with posts
followees = self.db.query(
"SELECT followee_id FROM follows WHERE follower_id = %s", user_id
)
followee_ids = [f.followee_id for f in followees]
if not followee_ids: return []
posts = self.db.query(
"SELECT post_id FROM posts WHERE author_id = ANY(%s) "
"ORDER BY created_at DESC LIMIT %s OFFSET %s",
followee_ids, page_size, page * page_size
)
return [p.post_id for p in posts]
def fan_out_post(self, author_id: int, post_id: int,
score: float):
# Fan-out on write for users with small follower counts
followers = self.db.query(
"SELECT follower_id FROM follows WHERE followee_id = %s "
"AND follower_id NOT IN (SELECT id FROM users WHERE follower_count > 10000)",
author_id
)
pipe = self.redis.pipeline()
for f in followers:
pipe.zadd(f"feed:{f.follower_id}", {post_id: score})
pipe.zremrangebyrank( # cap feed size
f"feed:{f.follower_id}", 0, -(self.MAX_FEED_SIZE + 1)
)
pipe.execute()Feed Ranking
Chronological sort (simplest): score = created_at timestamp. Sort descending. Engagement-based ranking: score = created_at + engagement_bonus. engagement_bonus = log(1 + likes) * W_like + log(1 + comments) * W_comment + log(1 + shares) * W_share. Weights tuned to platform goals. Personalized ranking: ML model scores each post for a user based on: author relationship strength (how often user interacts with this author), content affinity (topics user historically engages with), freshness decay (exponential decay by age), media type preference. For scale: pre-compute ranking scores when posts are created (fan-out writes the score into the sorted set). Re-rank on read for the top 50 posts if personalization signals need real-time data.
Feed Caching and Invalidation
Feed stored in Redis sorted set: key = feed:{user_id}, score = ranking_score, member = post_id. Max 1000 items per feed (ZREMRANGEBYRANK on each write). Cache individual posts: key = post:{post_id}, TTL 1h. Invalidation: post deletion requires iterating all follower feeds to remove the post – expensive for high-follower accounts. Mitigation: use a tombstone approach – mark post as deleted in DB; on read, filter out tombstoned posts from the feed without clearing the sorted set. Like count updates: increment a Redis counter on each like; batch-flush to DB every 60 seconds. This avoids a DB write per like under high engagement.
{“@context”:”https://schema.org”,”@type”:”FAQPage”,”mainEntity”:[{“@type”:”Question”,”name”:”What is the difference between fan-out on write and fan-out on read for social feeds?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Fan-out on write (push model): when a user posts, immediately write the post to every follower's feed. Reads are fast (pre-computed), writes are slow (O(followers) per post). Bad for celebrities with millions of followers. Fan-out on read (pull model): when a user loads their feed, query recent posts from all accounts they follow and merge/sort. Reads are slow (O(followees) queries), writes are instant. Bad for users following thousands of accounts. Hybrid (used by Instagram, Twitter): fan-out on write for regular users, fan-out on read for celebrities (> 10K followers). At read time, merge pre-computed feed with live celebrity posts.”}},{“@type”:”Question”,”name”:”How do you store and serve user feeds efficiently at scale?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Store each user's feed as a Redis sorted set: key=feed:{user_id}, member=post_id, score=ranking_score. Cap at 1000 items per feed (ZREMRANGEBYRANK on each write). When reading: ZREVRANGE to get top N post IDs by score, then batch-fetch post content from a Redis post cache (key=post:{post_id}, TTL 1h). Two-level cache: feed sorted set (post order) and post content cache (post data). Database is only hit on cache miss. For new users with empty feeds: fall back to pull model query from DB.”}},{“@type”:”Question”,”name”:”How do you implement feed ranking beyond chronological order?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Engagement-weighted score: score = created_at_unix + (log(1+likes)*W_l + log(1+comments)*W_c + log(1+shares)*W_s). Weights are tuned empirically (comments are worth more than likes). Log-scale dampening prevents viral posts from permanently dominating. For personalized ranking: add a personalization bonus based on author relationship strength (how often the viewer has interacted with this author) and content affinity (ML-predicted probability the viewer engages with this content type). Scores are computed at fan-out time and stored in the sorted set. Re-rank top 50 posts at read time using real-time personalization signals if needed.”}},{“@type”:”Question”,”name”:”How do you handle post deletion from feeds at scale?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Hard deletion from all follower feeds (removing from N Redis sorted sets) is too expensive for high-follower accounts – O(followers) operations. Use tombstones instead: mark the post as deleted in the DB and cache (post:{post_id} -> {deleted: true}). On feed read, filter out tombstoned posts from the returned list. The post_id may remain in Redis sorted sets, but is filtered at read time. Periodic cleanup job: scan Redis feeds and remove tombstoned post IDs. This trades slight memory overhead for fast deletes. Alternatively, store a user-specific deleted_posts set and exclude on read.”}},{“@type”:”Question”,”name”:”How do you handle like counts without a database write per like?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Use a Redis counter: INCR like_count:{post_id} on each like. Decrement on unlike. Read the counter from Redis for display – no DB hit. Periodically (every 60 seconds), flush Redis counters to the DB in batch: UPDATE posts SET like_count = %s WHERE post_id = %s for all posts with dirty counters. Mark counters as clean after flush. If Redis fails and restarts, rebuild counters from the DB. This approach handles viral posts with thousands of likes per second without DB write amplification. Apply the same pattern to comment counts and view counts.”}}]}
Meta’s News Feed is the canonical social feed system design topic. See common questions for Meta interview: social feed and News Feed system design.
Twitter system design interviews cover feed fan-out and ranking. Review patterns for Twitter/X interview: feed and timeline system design.
LinkedIn system design covers professional feed ranking and personalization. See patterns for LinkedIn interview: professional feed system design.