# Outbox Pattern Low-Level Design Source: https://www.techinterview.org/post/3233468057/lld-outbox-pattern/ Updated: 2026-04-18 · techinterview.org ## Outbox Pattern — Low-Level Design The Transactional Outbox pattern ensures that database writes and message/event publishing happen atomically, without two-phase commit. It solves the dual-write problem: how do you write to both a database and a message broker (Kafka, SQS) without risking one succeeding and the other failing? This pattern is asked at Stripe, Uber, and any company doing event-driven microservices. ## The Dual-Write Problem ``` -- Naive approach (broken): def create_order(user_id, items): order = db.insert(Order, {...}) # Step 1: write to DB kafka.publish('order.created', order) # Step 2: publish event return order -- Failure modes: -- Step 1 succeeds, step 2 fails → DB has order but no event published -- → downstream services never process the order -- Step 1 fails, step 2 succeeds → event published for non-existent order -- → downstream services process a ghost order -- Two-phase commit (2PC) is the "correct" solution but requires -- distributed transaction support across DB + broker — complex, slow, rarely available. ``` ## Outbox Pattern: Write to Same DB Transaction ``` -- The outbox table lives in the same database as the business data. -- Writing both in one transaction makes them atomic. CREATE TABLE OutboxEvent ( id BIGSERIAL PRIMARY KEY, aggregate_type TEXT NOT NULL, -- 'Order', 'Payment' aggregate_id BIGINT NOT NULL, event_type TEXT NOT NULL, -- 'order.created' payload JSONB NOT NULL, status TEXT DEFAULT 'pending', -- pending, published created_at TIMESTAMPTZ DEFAULT NOW(), published_at TIMESTAMPTZ ); def create_order(user_id, items): with db.transaction(): # Step 1: create the order order = db.insert(Order, {'user_id': user_id, 'status': 'pending'}) # Step 2: write the event to the outbox — same transaction db.insert(OutboxEvent, { 'aggregate_type': 'Order', 'aggregate_id': order.id, 'event_type': 'order.created', 'payload': {'order_id': order.id, 'user_id': user_id, 'items': items}, }) # If the transaction commits: both order AND outbox event exist # If the transaction rolls back: neither exists return order ``` ## Outbox Relay Worker ``` def relay_outbox_events(): """Runs continuously; picks up pending outbox events and publishes them.""" while True: # Claim a batch atomically events = db.execute(""" UPDATE OutboxEvent SET status = 'publishing' WHERE id IN ( SELECT id FROM OutboxEvent WHERE status = 'pending' ORDER BY id ASC LIMIT 100 FOR UPDATE SKIP LOCKED ) RETURNING * """) for event in events: try: kafka.publish( topic=event.event_type, key=str(event.aggregate_id), value=event.payload, headers={'event_id': str(event.id)}, # for deduplication ) db.execute(""" UPDATE OutboxEvent SET status='published', published_at=NOW() WHERE id=%(id)s """, {'id': event.id}) except Exception: # Reset to pending for retry db.execute("UPDATE OutboxEvent SET status='pending' WHERE id=%(id)s", {'id': event.id}) if not events: time.sleep(0.1) ``` ## Change Data Capture Alternative ``` -- Instead of a relay worker polling the OutboxEvent table, -- use PostgreSQL logical replication (Debezium) to stream changes. -- Debezium reads PostgreSQL WAL (write-ahead log) and publishes -- every INSERT into OutboxEvent directly to Kafka. -- Advantages over relay worker: -- Near real-time (milliseconds vs. polling interval) -- No additional DB load from polling queries -- Guaranteed ordering (WAL is ordered) -- Disadvantages: -- Requires Debezium/Kafka Connect setup -- More infrastructure to manage -- WAL retention must be configured (replication slot lag) -- For most teams: start with a polling relay worker. -- Migrate to CDC when sub-second latency is required. ``` ## Outbox Cleanup ``` -- Published events accumulate. Clean up after retention window. -- Run nightly: DELETE FROM OutboxEvent WHERE status = 'published' AND published_at < NOW() - INTERVAL '7 days'; -- Or use partitioning: partition by week, drop old partitions. -- Stale 'publishing' events (relay worker crashed): UPDATE OutboxEvent SET status='pending' WHERE status='publishing' AND created_at < NOW() - INTERVAL '5 minutes'; ``` ## Key Interview Points - **Same-DB transaction is the key insight:** The outbox works because both the business record and the event record live in the same database. One ACID transaction writes both atomically. The relay worker publishes to Kafka separately — if it fails, the event stays in the outbox and is retried. - **Events may be published more than once:** The relay worker publishes at-least-once. Consumers must be idempotent (use the event.id as a deduplication key). The outbox pattern guarantees no events are lost; it does not guarantee exactly-once delivery. - **Order preservation:** Processing outbox events in id (insertion) order preserves causality. A payment.captured event will always be published after the corresponding order.created event. - **FOR UPDATE SKIP LOCKED enables multiple relay workers:** Run multiple relay workers in parallel; each claims a disjoint batch. Add workers to increase throughput. Outbox pattern and payment event reliability design is discussed in [Stripe system design interview questions](https://www.techinterview.org/post/3233460268/stripe-interview-guide-2026-process-bug-bash-round-and-payment-systems/). Outbox pattern and reliable event delivery design is covered in [Uber system design interview preparation](https://www.techinterview.org/post/3233460273/uber-interview-guide-2026-dispatch-systems-geospatial-algorithms-and-marketplace-engineering/). Outbox pattern and order processing reliability is discussed in [Shopify system design interview guide](https://www.techinterview.org/post/3233459924/shopify-interview/). See also: [Scale AI Interview Guide 2026: Data Infrastructure, RLHF Pipelines, and ML Engineering](https://www.techinterview.org/post/3233460275/scale-ai-interview-guide-2026-data-infrastructure-rlhf-pipelines-and-ml-engineering/)