Outbox Pattern — Low-Level Design
The Transactional Outbox pattern ensures that database writes and message/event publishing happen atomically, without two-phase commit. It solves the dual-write problem: how do you write to both a database and a message broker (Kafka, SQS) without risking one succeeding and the other failing? This pattern is asked at Stripe, Uber, and any company doing event-driven microservices.
The Dual-Write Problem
-- Naive approach (broken):
def create_order(user_id, items):
order = db.insert(Order, {...}) # Step 1: write to DB
kafka.publish('order.created', order) # Step 2: publish event
return order
-- Failure modes:
-- Step 1 succeeds, step 2 fails → DB has order but no event published
-- → downstream services never process the order
-- Step 1 fails, step 2 succeeds → event published for non-existent order
-- → downstream services process a ghost order
-- Two-phase commit (2PC) is the "correct" solution but requires
-- distributed transaction support across DB + broker — complex, slow, rarely available.
Outbox Pattern: Write to Same DB Transaction
-- The outbox table lives in the same database as the business data.
-- Writing both in one transaction makes them atomic.
CREATE TABLE OutboxEvent (
id BIGSERIAL PRIMARY KEY,
aggregate_type TEXT NOT NULL, -- 'Order', 'Payment'
aggregate_id BIGINT NOT NULL,
event_type TEXT NOT NULL, -- 'order.created'
payload JSONB NOT NULL,
status TEXT DEFAULT 'pending', -- pending, published
created_at TIMESTAMPTZ DEFAULT NOW(),
published_at TIMESTAMPTZ
);
def create_order(user_id, items):
with db.transaction():
# Step 1: create the order
order = db.insert(Order, {'user_id': user_id, 'status': 'pending'})
# Step 2: write the event to the outbox — same transaction
db.insert(OutboxEvent, {
'aggregate_type': 'Order',
'aggregate_id': order.id,
'event_type': 'order.created',
'payload': {'order_id': order.id, 'user_id': user_id, 'items': items},
})
# If the transaction commits: both order AND outbox event exist
# If the transaction rolls back: neither exists
return order
Outbox Relay Worker
def relay_outbox_events():
"""Runs continuously; picks up pending outbox events and publishes them."""
while True:
# Claim a batch atomically
events = db.execute("""
UPDATE OutboxEvent
SET status = 'publishing'
WHERE id IN (
SELECT id FROM OutboxEvent
WHERE status = 'pending'
ORDER BY id ASC
LIMIT 100
FOR UPDATE SKIP LOCKED
)
RETURNING *
""")
for event in events:
try:
kafka.publish(
topic=event.event_type,
key=str(event.aggregate_id),
value=event.payload,
headers={'event_id': str(event.id)}, # for deduplication
)
db.execute("""
UPDATE OutboxEvent
SET status='published', published_at=NOW()
WHERE id=%(id)s
""", {'id': event.id})
except Exception:
# Reset to pending for retry
db.execute("UPDATE OutboxEvent SET status='pending' WHERE id=%(id)s",
{'id': event.id})
if not events:
time.sleep(0.1)
Change Data Capture Alternative
-- Instead of a relay worker polling the OutboxEvent table,
-- use PostgreSQL logical replication (Debezium) to stream changes.
-- Debezium reads PostgreSQL WAL (write-ahead log) and publishes
-- every INSERT into OutboxEvent directly to Kafka.
-- Advantages over relay worker:
-- Near real-time (milliseconds vs. polling interval)
-- No additional DB load from polling queries
-- Guaranteed ordering (WAL is ordered)
-- Disadvantages:
-- Requires Debezium/Kafka Connect setup
-- More infrastructure to manage
-- WAL retention must be configured (replication slot lag)
-- For most teams: start with a polling relay worker.
-- Migrate to CDC when sub-second latency is required.
Outbox Cleanup
-- Published events accumulate. Clean up after retention window.
-- Run nightly:
DELETE FROM OutboxEvent
WHERE status = 'published'
AND published_at < NOW() - INTERVAL '7 days';
-- Or use partitioning: partition by week, drop old partitions.
-- Stale 'publishing' events (relay worker crashed):
UPDATE OutboxEvent SET status='pending'
WHERE status='publishing'
AND created_at < NOW() - INTERVAL '5 minutes';
Key Interview Points
- Same-DB transaction is the key insight: The outbox works because both the business record and the event record live in the same database. One ACID transaction writes both atomically. The relay worker publishes to Kafka separately — if it fails, the event stays in the outbox and is retried.
- Events may be published more than once: The relay worker publishes at-least-once. Consumers must be idempotent (use the event.id as a deduplication key). The outbox pattern guarantees no events are lost; it does not guarantee exactly-once delivery.
- Order preservation: Processing outbox events in id (insertion) order preserves causality. A payment.captured event will always be published after the corresponding order.created event.
- FOR UPDATE SKIP LOCKED enables multiple relay workers: Run multiple relay workers in parallel; each claims a disjoint batch. Add workers to increase throughput.
{“@context”:”https://schema.org”,”@type”:”FAQPage”,”mainEntity”:[{“@type”:”Question”,”name”:”What problem does the outbox pattern solve and what is the alternative?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”The dual-write problem: when a service writes to the database AND sends a message to Kafka/RabbitMQ in the same request, either operation can fail independently. If the DB write succeeds but the Kafka publish fails, downstream systems never receive the event — the system is silently inconsistent. The naive fix (retry the Kafka publish) risks publishing twice if the first publish actually succeeded but the ACK was lost. The outbox pattern solves this by writing the event to an Outbox table in the SAME database transaction as the business data. The event is only visible to the message relay after it commits atomically with the DB write. A separate relay process reads the outbox and publishes to Kafka with at-least-once semantics.”}},{“@type”:”Question”,”name”:”How does the outbox relay achieve at-least-once delivery and how do consumers handle duplicates?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”The outbox relay reads unprocessed rows from the Outbox table (WHERE status=’pending’ ORDER BY id FOR UPDATE SKIP LOCKED), publishes each to Kafka, then marks it processed. If the relay crashes after publishing but before marking processed, it will re-publish on restart — producing a duplicate. Consumers must be idempotent: check a processed_event_ids set (Redis SET or DB table) before processing. The event_id (stored in the Outbox row and included in the Kafka message) is the idempotency key. This is the standard at-least-once + idempotent consumer pattern — it guarantees exactly-once business effect without distributed transactions.”}},{“@type”:”Question”,”name”:”How do you prevent the Outbox table from growing unboundedly?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Delete processed rows after a retention period. Add a cleanup job: DELETE FROM Outbox WHERE status=’processed’ AND processed_at < NOW() – INTERVAL ‘7 days’. Run it nightly or via a background worker. Keep 7 days of processed events for debugging and replay if a downstream consumer needs to reprocess. The Outbox table should be small (typically <10K rows for active systems) — if it grows large, it indicates the relay is falling behind. Add an alert: if MAX(created_at) – NOW() for pending rows exceeds 30 seconds, the relay is stalled.”}},{“@type”:”Question”,”name”:”Can Debezium CDC replace the outbox pattern?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Yes — Debezium with the outbox event router is the production-grade version. Instead of writing a custom relay, Debezium reads your PostgreSQL WAL and publishes Outbox table inserts directly to Kafka. This eliminates the relay process entirely. The Debezium outbox event router transforms the raw CDC event (which would go to a single outbox.events Kafka topic) into events routed to the correct topic based on the aggregatetype column in the Outbox row. For teams already running Debezium for other CDC use cases, this is the preferred approach. For teams that want a simpler setup without Kafka and Debezium, a custom relay with polling is a valid alternative.”}},{“@type”:”Question”,”name”:”How do you scale the outbox relay for high-throughput systems?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Partition the Outbox table by aggregate_id (shard key) and run one relay instance per partition. Use FOR UPDATE SKIP LOCKED so multiple relay instances can process different rows concurrently without blocking each other. Each relay instance processes its assigned partition: relay-0 handles aggregate_id % 4 = 0, relay-1 handles aggregate_id % 4 = 1, etc. This gives linear throughput scaling. Alternatively, use Debezium which inherits Kafka’s partitioning and scales relay throughput by adding Kafka consumer instances. For most systems, a single relay polling every 100ms is sufficient for <10,000 events/second.”}}]}
Outbox pattern and payment event reliability design is discussed in Stripe system design interview questions.
Outbox pattern and reliable event delivery design is covered in Uber system design interview preparation.
Outbox pattern and order processing reliability is discussed in Shopify system design interview guide.