Search Indexer Low-Level Design: Document Ingestion, Index Building, and Incremental Updates
Search Indexer Overview A search indexer transforms raw documents into a data structure that supports fast full-text queries. The pipeline […]
Search Indexer Overview A search indexer transforms raw documents into a data structure that supports fast full-text queries. The pipeline […]
Spam Detection System Overview Spam detection combines fast rule-based velocity checks, text classification, and graph-based account analysis. No single signal
Abuse Detection System Overview An abuse detection system protects users and the platform from account takeover, credential stuffing, scraping, fake
Search Ranking System Overview A search ranking system determines the order in which retrieved documents are presented to users. The
Personalization Engine Overview A personalization engine ranks content by predicted user interest rather than global popularity. The core idea: represent
What a Session Store Does A session store maintains server-side state for authenticated users across stateless HTTP requests. The client
Two-Token Architecture Modern authentication uses two token types with different properties: Access token: Short-lived (15 minutes), self-contained JWT. Resource servers
What an Access Log Service Provides Every HTTP request to a web service generates a log record. At scale, this
Why PII Scrubbing Is Hard PII (Personally Identifiable Information) appears in many forms: structured fields like email addresses and SSNs,
What a Consensus Log Provides A consensus log is a fault-tolerant, ordered sequence of records that all nodes in a
What an Orchestrator Does A data pipeline orchestrator schedules and executes directed acyclic graphs (DAGs) of computational tasks. Each DAG
What a Request Router Does A request router sits between clients and backend services, inspecting each incoming HTTP request and
What a Response Cache Does A response cache sits in front of upstream services and stores HTTP responses so that
The Distributed Scheduler Problem A scheduler that runs on a single node is a single point of failure. A scheduler
Coupon Schema A coupon service manages promotional codes that apply discounts to orders. The core schema: CREATE TABLE coupons (