Low Level Design: Multi-Tenant Architecture

By /

Multi-tenancy is an architecture where a single software instance serves multiple customers (tenants) with logical isolation between them. It is the fundamental architecture of SaaS products. The design challenge is balancing isolation (tenants cannot see each other data or affect each other performance) with cost efficiency (sharing infrastructure instead of running separate instances per customer). Understanding multi-tenancy is essential for system design interviews at SaaS companies like Salesforce, Stripe, and Datadog.

Three Isolation Models

Three database isolation models exist with different trade-offs: Silo (separate database per tenant): maximum isolation, easy data deletion (GDPR), simple backup/restore per tenant. High cost — dedicated infrastructure per tenant. Best for enterprise customers with strict compliance. Bridge (shared database, separate schema/namespace per tenant): moderate isolation with lower cost. Good for mid-market. Schema migrations must run for every tenant. Pool (shared database, shared tables with tenant_id column): lowest cost, highest density. Every query must include a tenant_id filter. Risk of data leakage if queries miss the filter. Best for SMB tiers with thousands of tenants.

-- Pool model: all queries MUST include tenant_id
-- Enforce via Row Level Security (PostgreSQL)
CREATE POLICY tenant_isolation ON orders
  USING (tenant_id = current_setting('app.current_tenant_id')::uuid);

ALTER TABLE orders ENABLE ROW LEVEL SECURITY;

-- Application sets tenant context before each query
SET LOCAL app.current_tenant_id = '550e8400-e29b-41d4-a716-446655440000';
SELECT * FROM orders WHERE status = 'pending';
-- RLS automatically adds: AND tenant_id = 550e...

Tenant Context Propagation

Every request carries tenant context through the system. The API gateway extracts the tenant from the JWT claim or API key, resolves the tenant ID, and adds it to request metadata (HTTP header or gRPC metadata). Every service reads this context and passes it downstream. Database connection pools set the tenant context variable at the start of each transaction. Middleware enforces that tenant context is always present — requests without valid tenant context are rejected at the gateway. Log every operation with tenant_id for audit trails.

Noisy Neighbor Mitigation

In the pool model, one large tenant running heavy queries degrades performance for all others (noisy neighbor problem). Mitigations: Query timeouts: enforce maximum query duration per tenant. Connection limits: assign each tenant a maximum connection pool size. Read replicas per tier: route enterprise tenants to dedicated read replicas. Tenant-aware rate limiting: limit API requests per tenant per second. Workload segregation: move top 10 tenants by query volume to the bridge or silo model automatically, keeping them off the shared pool.

Tenant Onboarding and Schema Migration

In the bridge model, creating a new tenant requires provisioning a new schema and running all migrations. This must be fast (sub-second for self-serve signup). Pre-provision empty tenant schemas from a pool of ready schemas. At signup, claim a schema from the pool and rename it to the tenant ID. A background job replenishes the pool. For the pool model, a new tenant is just a new row in the tenants table — no schema work needed. Schema migrations in pool model run once across all tenants; in bridge model they run N times (one per tenant schema), requiring a parallel migration runner.

Key Interview Discussion Points

  • Data residency: some tenants require data to stay in specific regions — silo model enables per-region database clusters
  • Custom domains and SSL: each tenant gets a custom subdomain (tenant.yourproduct.com) requiring wildcard SSL certificates
  • Tenant-specific feature flags: certain features enabled only for specific tenants or tiers
  • Cost attribution: track infrastructure cost per tenant to price plans correctly and identify unprofitable tenants
  • Disaster recovery isolation: in silo model, one tenant database failure does not affect others; in pool model, a corrupted table affects all tenants
Scroll to Top