Backward Compatibility Low-Level Design: Tolerant Reader, Postel’s Law, and API Contract Testing

The Tolerant Reader Pattern

The tolerant reader pattern (Martin Fowler, 2011) describes how a service should consume data from external sources in a way that tolerates changes in the source schema. The rules are simple:

Ignore unknown fields. If the response contains fields your schema does not know about, discard them silently. Do not throw an error.
Use defaults for missing fields. If an expected field is absent, use a sensible application-level default rather than failing.
Extract only what you need. Do not deserialize the entire payload into a strict schema; pick out only the fields your service actually uses.

Most modern JSON libraries (Jackson, System.Text.Json, serde_json) ignore unknown fields by default or with a single annotation. Strict deserialization that rejects unknown fields is a common source of production incidents when a provider adds a new field.

Postel's Law

Postel's Law (the robustness principle from RFC 793): Be conservative in what you send, be liberal in what you accept. Applied to APIs:

Conservative sender: Send only documented fields. Do not send fields the spec doesn't define. Use the most specific, unambiguous types.
Liberal receiver: Accept variations in optional fields, handle missing fields gracefully, tolerate minor type coercions (quoted integers, null where empty string is expected).

Postel's Law has critics — extreme liberalism in parsers has historically led to security vulnerabilities (HTML parsers, XML parsers). The practical balance is: be liberal about presence/absence and unknown fields, but strict about security-relevant values.

Consumer-Driven Contract Testing with Pact

Consumer-driven contract testing inverts the testing relationship. Instead of the provider defining what it publishes and hoping consumers adapt, each consumer defines the exact contract it depends on, and the provider's CI verifies that contract is still satisfied.

// Consumer test (JavaScript/Pact)
await provider.addInteraction({
  state: 'user 123 exists',
  uponReceiving: 'a request for user 123',
  withRequest: { method: 'GET', path: '/users/123' },
  willRespondWith: {
    status: 200,
    body: { id: like(123), email: like('a@b.com') }
  }
});
// Pact generates a contract JSON file

# Provider verification (in provider CI)
pact-verifier --provider-base-url http://localhost:8080 
              --pact-broker-url https://broker.example.com

The Pact Broker stores contracts and tracks which consumer versions have been verified by which provider versions. The can-i-deploy command answers whether a given provider version is safe to deploy given all known consumer contracts.

Feature Detection vs Version Detection

Version detection checks the API version and branches: if version >= 2 then use new field. Feature detection checks capability directly: if response.has('new_field') then use it. Feature detection is more resilient — it does not require coordinating on version numbers and naturally handles partial rollouts and feature flags on the provider side.

Semantic Versioning for Libraries

SemVer (major.minor.patch) encodes backward compatibility in version numbers:

Patch (1.0.X): Bug fixes, no API changes.
Minor (1.X.0): New features, backward-compatible additions.
Major (X.0.0): Breaking changes. Consumers must explicitly upgrade.

Package managers use these semantics for dependency resolution. A consumer declaring "^1.2.0" accepts any 1.x.y where y >= 2 but not 2.0.0. This makes the social contract between library authors and users machine-enforced.

Database Backward Compatibility

Databases must also evolve without breaking running services. Safe additive migrations:

Add a nullable column with a default value
Add a new table
Add an index (with CREATE INDEX CONCURRENTLY in Postgres)

Column renames require a multi-phase migration to avoid downtime:

Phase 1: Add new column (display_name), write to both columns, read from old.
Phase 2: Backfill display_name from full_name for existing rows.
Phase 3: Switch reads to display_name.
Phase 4: Drop full_name after all service versions are updated.

Each phase is a separate deploy. This ensures no running service version ever encounters a missing column.

Binary Protocol Backward Compatibility

Binary protocols use length-prefixed fields and type-length-value (TLV) encoding so parsers can skip unknown fields without failing:

[field_tag: 2 bytes][field_length: 4 bytes][field_data: N bytes]

If a parser encounters an unknown tag, it reads field_length bytes and advances, preserving the stream position. This is the mechanism that gives Protobuf and Thrift their forward compatibility without schema registry lookups.

HATEOAS for Discoverable API Changes

Hypermedia as the Engine of Application State (HATEOAS) embeds links in responses that describe available actions and related resources. Clients follow links rather than constructing URLs, which decouples them from URL structure changes:

{
  "order_id": 789,
  "status": "PLACED",
  "_links": {
    "self":   { "href": "/orders/789" },
    "cancel": { "href": "/orders/789/cancel", "method": "POST" },
    "payment":{ "href": "/payments?order=789" }
  }
}

When URLs change, clients that follow links rather than hardcoding paths continue to work without modification. HATEOAS is rarely implemented in full, but even partial link embedding (for pagination, related resources, and available actions) meaningfully reduces coupling between API versions.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “What is the tolerant reader pattern?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “The tolerant reader pattern instructs a consumer to ignore unknown fields and use safe defaults for missing ones rather than failing on any deviation from the expected schema. This makes consumers resilient to additive changes in producers and decouples their release cycles, since a consumer deployed before a new field is added will continue working correctly after the producer starts sending it.”
}
},
{
“@type”: “Question”,
“name”: “How does consumer-driven contract testing work with Pact?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “In Pact, each consumer defines a contract — a set of example interactions specifying what requests it sends and what response fields it actually uses — and publishes it to the Pact Broker. The provider's CI pipeline then replays those interactions against its real implementation and fails the build if any consumer's expectations are violated, catching breaking changes before they reach production.”
}
},
{
“@type”: “Question”,
“name”: “What changes are safe to make without breaking backward compatibility?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Safe changes include adding new optional request fields, adding new response fields, adding new endpoints or enum values (if consumers use tolerant reader), and expanding a field's valid range. Unsafe changes include removing or renaming fields, changing a field's data type or encoding, making a previously optional field required, or altering the semantics of an existing status code or error format.”
}
},
{
“@type”: “Question”,
“name”: “How is backward compatibility maintained in database migrations?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Database migrations are kept backward compatible by expanding before contracting: first add the new column as nullable (or with a default), deploy all application versions that write to both old and new columns, then — once no code reads the old column — drop it in a later migration. This multi-phase approach prevents downtime and ensures any rolled-back application version can still read and write the schema it expects.”
}
}
]
}