The CAP theorem is the most cited theorem in distributed systems — and the most misunderstood. It states that a distributed system can provide at most two of three guarantees: Consistency, Availability, and Partition tolerance. This guide goes beyond the surface-level explanation to cover what CAP actually means in practice, the PACELC extension, consistency models, and how production databases make their tradeoff choices — essential for senior system design interviews.
What CAP Actually Says
Consistency (C): every read receives the most recent write or an error. All nodes see the same data at the same time. Specifically, CAP means linearizability — the strongest consistency model. Availability (A): every request receives a (non-error) response, without the guarantee that it contains the most recent write. Every non-failing node must return a response. Partition tolerance (P): the system continues to operate despite arbitrary network partitions (messages between nodes may be lost or delayed). The theorem: during a network partition, you must choose between consistency and availability. You cannot have both. Why partition tolerance is not optional: in any distributed system, network partitions will happen (switches fail, cables are cut, cloud providers have outages). You must tolerate them. The real choice is: during a partition, do you return an error to maintain consistency (CP) or return potentially stale data to maintain availability (AP)? Important nuance: CAP only applies during a partition. When the network is healthy, you can have both consistency and availability. The CAP choice is about failure mode behavior, not normal operation.
CP vs AP Systems
CP systems (choose consistency over availability during partition): during a network partition, the minority partition refuses to serve requests (returns errors) to prevent serving stale data. The majority partition continues operating normally. Examples: HBase, MongoDB (with majority write concern), etcd, ZooKeeper, CockroachDB, Spanner. When to choose CP: financial systems (showing an incorrect balance is worse than showing an error), inventory management (overselling is worse than a temporary error), and coordination services (a split-brain lock service is catastrophic). AP systems (choose availability over consistency during partition): during a partition, all nodes continue serving requests. Nodes in different partitions may serve different (stale) data. After the partition heals, nodes reconcile their states. Examples: Cassandra (with low consistency level), DynamoDB (eventually consistent reads), CouchDB, DNS. When to choose AP: social media feeds (showing a slightly stale feed is better than an error), shopping carts (users can add items even during a partition — merge later), and content serving (serving a cached page is better than an error).
PACELC: Beyond CAP
CAP only describes behavior during partitions. PACELC extends it: during a Partition, choose Availability or Consistency. Else (normal operation), choose Latency or Consistency. This captures the full tradeoff space. Even without partitions, achieving consistency requires coordination (round-trips to replicas), which adds latency. You trade latency for consistency even when the network is healthy. Examples: DynamoDB: PA/EL — during partition, choose availability. In normal operation, choose latency (eventually consistent reads are faster than strongly consistent reads). CockroachDB: PC/EC — during partition, choose consistency. In normal operation, choose consistency (linearizable reads require leader confirmation, adding latency). Cassandra: PA/EL with tunable consistency — the default (ONE consistency level) is PA/EL. With QUORUM, it becomes PC/EC. The user chooses per query. MongoDB: PC/EC by default (majority write concern + majority read concern). Can be configured to PA/EL with lower concerns. PACELC is more useful than CAP for real-world system design because it describes the tradeoff you make on every request, not just during failures.
Consistency Models Spectrum
Consistency is not binary. There is a spectrum from strong to weak: (1) Linearizability (strongest) — every operation appears to take effect at a single instant between invocation and response. The system behaves as if there is one copy. Expensive: requires coordination (round-trips to majority). (2) Sequential consistency — all operations are seen in some sequential order that is consistent with the order observed by each individual process. Weaker than linearizable (operations may not appear at their real-time order). (3) Causal consistency — operations that are causally related are seen in the same order by all nodes. Concurrent operations may be seen in different orders. Practical and efficient. (4) Read-your-writes — a user always sees their own writes. Other users may see stale data. Common in web applications (write to primary, read from the user session-pinned replica). (5) Eventual consistency (weakest useful model) — if no new updates are made, all replicas will eventually converge to the same value. No guarantee about when. A read may return any version. Stale reads are possible. DynamoDB eventually consistent reads, DNS, CDN caches. For system design interviews: specify which consistency model you need and why. “We use eventual consistency for the news feed (stale feeds are acceptable) but linearizable consistency for the payment ledger (financial accuracy is critical).”
Practical Implications for System Design
How CAP affects your architecture decisions: (1) Database selection — need strong consistency for financial data? Choose PostgreSQL with synchronous replication, CockroachDB, or Spanner. Need availability for user-facing reads? Choose Cassandra or DynamoDB with eventual consistency. (2) Per-operation consistency — many databases offer tunable consistency per operation. DynamoDB: strongly consistent reads (CP) or eventually consistent reads (AP). Cassandra: consistency level per query (ONE = AP, QUORUM = CP, ALL = CP but no availability). Use strong consistency for writes and critical reads; eventual consistency for non-critical reads. (3) Caching — a cache introduces eventual consistency by definition (the cache may be stale). This is usually acceptable because the performance benefit outweighs the staleness risk. Set TTLs to bound staleness. (4) Cross-service consistency — in microservices, achieving consistency across services requires distributed transactions (2PC — slow, fragile) or eventual consistency with sagas (compensating transactions). Most production systems use eventual consistency between services and strong consistency within each service database. (5) Multi-region — cross-region strong consistency adds 50-200ms latency per write (inter-region round-trip). Most multi-region systems use eventual consistency for cross-region replication and strong consistency within each region.
{“@context”:”https://schema.org”,”@type”:”FAQPage”,”mainEntity”:[{“@type”:”Question”,”name”:”What does the CAP theorem actually mean in practice?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”CAP states: during a network partition, a distributed system must choose between consistency and availability. Consistency (linearizability): every read returns the most recent write. Availability: every non-failing node returns a response. Partition tolerance: the system operates despite network failures. Since partitions are inevitable in distributed systems, the real choice is: during a partition, return errors to maintain consistency (CP) or return potentially stale data to maintain availability (AP). Important nuance: CAP only applies during partitions. When the network is healthy, you can have both consistency and availability. CP systems (etcd, CockroachDB, Spanner): minority partition stops serving to prevent stale reads. AP systems (Cassandra with CL=ONE, DynamoDB eventual reads): all nodes serve requests, may return stale data during partition. After partition heals, nodes reconcile.”}},{“@type”:”Question”,”name”:”What is PACELC and how does it extend CAP?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”PACELC extends CAP to cover normal operation: during a Partition, choose Availability or Consistency. Else (no partition), choose Latency or Consistency. Even without partitions, strong consistency requires coordination (round-trips to replicas), adding latency. You trade latency for consistency on every request, not just during failures. Examples: DynamoDB (PA/EL): during partition choose availability; normally choose latency (eventually consistent reads are faster). CockroachDB (PC/EC): during partition choose consistency; normally choose consistency (linearizable reads need leader confirmation). Cassandra (tunable): with CL=ONE it is PA/EL; with CL=QUORUM it is PC/EC. Per-query choice. PACELC is more practical than CAP because it describes the tradeoff on every request, not just rare partition events.”}},{“@type”:”Question”,”name”:”What is the consistency models spectrum from strongest to weakest?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”From strongest to weakest: (1) Linearizability — operations appear instant at some point between invocation and response. System behaves as one copy. Most expensive (majority coordination). Used by: Spanner, CockroachDB, etcd. (2) Sequential consistency — all operations seen in some sequential order consistent with each process order. May not match real-time order. (3) Causal consistency — causally related operations seen in same order by all. Concurrent operations may differ. Practical and efficient. (4) Read-your-writes — users see their own writes. Others may see stale data. Common in web apps (read from primary after write). (5) Eventual consistency — with no new updates, all replicas converge eventually. No timing guarantee. Stale reads possible. Used by: DynamoDB eventually consistent reads, DNS, CDN caches. In interviews: specify which model per component. Strong consistency for payments, eventual for feeds.”}},{“@type”:”Question”,”name”:”How should CAP theorem influence your database selection in system design?”,”acceptedAnswer”:{“@type”:”Answer”,”text”:”Match consistency requirements to database choice: Financial data (payments, balances, inventory): use CP databases — PostgreSQL with synchronous replication, CockroachDB, or Spanner. Wrong balances or overselling are unacceptable. User-facing reads (feeds, profiles, search): use AP databases or eventual consistency — DynamoDB eventually consistent reads, Cassandra CL=ONE, or cache-backed reads. Slightly stale data is acceptable; errors are not. Tunable per operation: DynamoDB offers both strongly consistent and eventually consistent reads. Cassandra offers per-query consistency levels. Use strong consistency for critical writes and reads; eventual for non-critical reads. Cross-service: use eventual consistency between services (saga pattern) and strong consistency within each service database. Multi-region: cross-region strong consistency adds 50-200ms per write. Most systems use eventual consistency for cross-region replication. Key interview phrase: We use eventual consistency for the news feed because slightly stale content is acceptable, but linearizable consistency for the payment ledger because financial accuracy is critical.”}}]}