Pinterest is a visual discovery platform with 450+ million monthly active users who save, organize, and discover ideas through images (pins). Designing Pinterest combines image processing, visual search (finding similar images by content), recommendation engines, and a unique social-graph structure (boards, followers, interests). This guide covers the key architectural components for a system design interview.
Data Model: Pins, Boards, and Users
Core entities: (1) Pin — an image with metadata: pin_id, image_url, title, description, source_url (external link), creator_id, board_id, created_at, save_count, click_count. (2) Board — a collection of pins organized by theme: board_id, user_id, name, description, category, pin_count, follower_count, is_secret. (3) User — user_id, username, interests (inferred), followers, following, boards. (4) Save — when a user saves a pin to their board: save_id, user_id, pin_id, board_id, created_at. This is the primary engagement signal. The interest graph: unlike Twitter (who you follow) or Facebook (who you know), Pinterest is organized around interests. A user follows boards (topics), not just people. The recommendation engine models user interests based on saved pins, clicked pins, and followed boards. Storage: pin metadata in PostgreSQL (sharded by pin_id). Images in S3. The pin-user-board relationships (saves, follows) in a graph store or wide-column database. Elasticsearch for pin search (text + visual features).
Image Processing Pipeline
When a pin is created or an image is uploaded: (1) Store the original image in S3. (2) Generate multiple resized versions: thumbnail (150px), medium (600px), and full-size (original). Serve via CDN with format negotiation (WebP/AVIF for modern browsers). (3) Extract visual features — a deep learning model (convolutional neural network) processes the image and produces a feature embedding vector (e.g., 2048-dimensional vector). This embedding captures the visual content: colors, objects, style, composition. Store the embedding in a vector database (Pinecone, Milvus, or custom FAISS-based service). (4) Object detection and classification — ML identifies objects in the image: “red dress,” “modern kitchen,” “chocolate cake.” These labels enrich the pin metadata for search and recommendation. (5) Text extraction (OCR) — extract any text visible in the image (recipe instructions, product names, quotes). Add to searchable metadata. (6) Content moderation — ML scans for policy violations (nudity, violence, spam). Flagged images are queued for human review. (7) Duplicate detection — compute a perceptual hash (pHash). Compare against existing pins to detect duplicates and near-duplicates. Prevent the same image from flooding the platform.
Visual Search: Find Similar Images
Pinterest Lens lets users take a photo and find visually similar pins. Architecture: (1) The user takes a photo or selects a region of an existing pin. (2) The image is processed by the same CNN that generates embeddings. (3) Approximate nearest neighbor (ANN) search finds the K most similar embeddings in the vector database. With 10+ billion pins, exact nearest-neighbor search is too slow. ANN algorithms (HNSW, IVF-PQ in FAISS) return approximate results in milliseconds. (4) The results are ranked by visual similarity, relevance to the user interests, and pin quality (save count, freshness). (5) Display results with “Shop the Look” (link to buy identified products). The embedding model is trained on Pinterest data: pins that are frequently saved together are embedded closer. This captures both visual similarity and semantic relevance (a blue couch and blue cushions are related even though they look different). Crop search: the user draws a box around a specific object in a pin (a lamp in a room photo). The cropped region is embedded and searched separately, returning pins of similar lamps. This is object-level visual search, powered by object detection + per-object embeddings.
Home Feed Generation
The Pinterest home feed is entirely recommendation-driven (unlike Twitter where you see posts from accounts you follow). The feed shows pins the user is predicted to find interesting, regardless of who posted them. Pipeline: (1) Candidate generation — produce thousands of candidate pins from: pins similar to recently saved pins (embedding similarity), pins popular in the user interest categories, pins from followed boards and users, and trending pins in the user region. (2) Ranking — an ML model scores each candidate. Features: pin quality (save rate, click-through rate), visual appeal (learned from engagement data), relevance to user interests (embedding distance), freshness (new pins get a boost), and diversity (avoid showing 10 kitchen pins in a row). (3) Blending — mix candidates from different sources. Ensure a balanced feed: some from interests, some new discoveries, some trending. (4) Deduplication — remove pins the user has already seen (maintain a seen set per user). The feed is generated in batches (pre-compute the next 100 pins when the user is near the end of the current batch). Each batch request to the recommendation service takes 50-100ms.
Search
Pinterest search combines text search and visual search. Text search: Elasticsearch indexes pin titles, descriptions, board names, and ML-generated labels (object detection output). Query processing: spell correction, synonym expansion, and intent classification (is the user looking for ideas, products, or tutorials?). Results are ranked by text relevance (BM25) + engagement (save rate, click rate) + personalization (user interest match). Visual search: when a user searches “modern living room,” the system also retrieves pins with similar visual embeddings to the user previously saved “modern living room” pins. This personalizes results beyond text matching. Guided search: after a query, Pinterest shows refinement chips: “modern living room” -> “small space” | “minimalist” | “with fireplace” | “Scandinavian.” These are generated from common query refinements learned from search logs. Shopping: pins linked to products show prices and availability. Product pins are indexed with structured attributes (price, brand, availability). Search results blend organic pins with shoppable pins when commercial intent is detected.