Document Vault Low-Level Design: Encrypted Storage, Access Control, and Retention Policy

What Is a Document Vault?

A document vault is a secure long-term storage service for sensitive documents with strong access control, client-side or server-side encryption, versioning, configurable retention policies, and legal hold capability. Use cases include financial records, medical documents, legal contracts, and compliance archives. The design must balance security, auditability, and performance for large-scale document management.

Requirements

Functional Requirements

Store and retrieve documents with version history
Encrypt documents such that the service operator cannot read plaintext without proper key access
Fine-grained access control: per-document permissions for read, write, delete, and share
Configurable retention policies: auto-delete after N days or on a specific date
Legal hold: exempt specific documents from retention deletion
Audit log of all access and mutation events

Non-Functional Requirements

Encryption at rest and in transit; key separation per tenant or document
Sub-second metadata queries; acceptable latency for downloads via presigned URLs
Retention enforcement must be reliable and not miss deletion deadlines
Legal holds must override any automated deletion

Data Model

documents: document_id, vault_id, owner_id, name, content_type, status (active, deleted), created_at
document_versions: version_id, document_id, s3_key, encrypted_dek (base64), size_bytes, checksum_sha256, uploaded_by, created_at, is_current
access_policies: policy_id, document_id, principal_type (user, group, role), principal_id, permissions (bitmask or JSON array), granted_by, granted_at, expires_at
retention_policies: policy_id, vault_id, rule_type (delete_after_days, delete_on_date), value, applies_to_tags (JSON)
legal_holds: hold_id, document_id, reason, created_by, created_at, released_at
vault_audit_log: log_id, document_id, version_id, actor_id, action, ip_address, result, created_at

Each version stores an encrypted Data Encryption Key (DEK). The DEK is encrypted with a Key Encryption Key (KEK) managed in a KMS (AWS KMS, HashiCorp Vault). This envelope encryption pattern means rotating the KEK only requires re-encrypting the DEK records, not re-encrypting the document content.

Core Algorithms

Envelope Encryption

On upload: generate a random 256-bit AES DEK, encrypt the document content with AES-GCM using the DEK, call KMS.Encrypt(DEK, key_id) to get the encrypted DEK, store the ciphertext document in S3 and the encrypted DEK in document_versions. On download: call KMS.Decrypt(encrypted_dek) to recover the DEK, decrypt the document bytes using AES-GCM with the DEK. The plaintext DEK never persists to disk.

Fine-Grained Access Control

On every API request, resolve the caller identity, then evaluate access_policies for the document_id where principal matches the caller (user, any group membership, or role). Compute the union of all matching permission sets. Check the required permission bit against the union. Cache resolved policies in Redis keyed by (document_id, principal_id) with a short TTL (60s) to avoid per-request database lookups.

Retention Policy Enforcement

A scheduler runs daily, queries documents where created_at + retention_days <= TODAY and status = active. For each candidate, check the legal_holds table for any active hold (released_at IS NULL). If a hold exists, skip deletion and emit a warning metric. Otherwise, mark the document deleted, delete all version S3 objects, and log the action. The legal hold check must be inside a transaction or checked atomically to prevent a race between a hold being placed and the deletion job running.

API Design

POST /vaults/{id}/documents — upload new document; returns presigned S3 PUT URL
POST /documents/{id}/versions — upload new version
GET /documents/{id}/versions/{vid}/download — returns presigned S3 GET URL after access check
PUT /documents/{id}/access — set access policy for a principal
POST /documents/{id}/holds — place legal hold
DELETE /documents/{id}/holds/{hold_id} — release legal hold
GET /documents/{id}/audit — fetch audit log for document

Scalability Considerations

Use presigned S3 URLs for all data plane transfers; the API tier only handles metadata and access decisions, never proxies document bytes. This keeps the API tier lightweight and horizontally scalable.

Partition vault_audit_log by created_at. For high-volume vaults (thousands of accesses per second), write audit events asynchronously via a Kafka topic consumed by a separate audit writer service; accept eventual consistency on audit log visibility in exchange for not blocking API responses.

For multi-tenant vaults, scope KEK per tenant so a compromised tenant key cannot decrypt another tenant documents. Use IAM policies or KMS key policies to enforce this boundary at the infrastructure level in addition to the application-level access_policies table.

Summary

A document vault uses envelope encryption to separate document keys from content, stores encrypted DEKs in the database, and delegates key management to a KMS. Access control is evaluated at request time from a policy table with short-lived caching. Retention enforcement checks legal holds atomically before any deletion, ensuring compliant long-term storage.

{
“@context”: “https://schema.org”,
“@type”: “FAQPage”,
“mainEntity”: [
{
“@type”: “Question”,
“name”: “How does client-side envelope encryption work for a document vault?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Generate a random Data Encryption Key (DEK) per document on the client. Encrypt the document with the DEK using AES-256-GCM. Encrypt the DEK with the user's Key Encryption Key (KEK), which is derived from their credentials or held in a KMS. Store only the encrypted DEK alongside the ciphertext blob. The server never sees plaintext — decryption requires the client to unwrap the DEK first. This is envelope encryption.”
}
},
{
“@type”: “Question”,
“name”: “How do you implement fine-grained access control for documents?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Use an ACL table: (document_id, principal_id, principal_type, permission) where permission is an enum (viewer, commenter, editor, owner). Evaluate access on every request by joining the document with its ACL. Support group principals by expanding group membership at evaluation time or pre-materializing effective permissions. Layer org-level policies (e.g., domain sharing) as inherited rows that individual ACLs can restrict but not elevate beyond.”
}
},
{
“@type”: “Question”,
“name”: “How is document version history managed?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “Store each save as an immutable version row: (document_id, version_number, s3_key, created_by, created_at, size, checksum). The current version pointer is a separate column on the document row. Restoring a version creates a new version row pointing to the old blob — never mutate history. Delta compression (storing diffs) can reduce storage cost; reconstruct the full document by replaying deltas from a snapshot base.”
}
},
{
“@type”: “Question”,
“name”: “How do retention policies and legal holds interact?”,
“acceptedAnswer”: {
“@type”: “Answer”,
“text”: “A retention policy defines a minimum hold period per document class (e.g., financial records: 7 years). A legal hold is an override that suspends deletion regardless of policy. Implement as two flags on the document: retention_expires_at (nullable) and legal_hold (boolean). The deletion job skips any document where legal_hold = true OR retention_expires_at > NOW(). Legal holds must be released explicitly by an authorized admin before the normal retention clock resumes.”
}
}
]
}