Low Level Design: TCP vs UDP — Network Protocol Internals

By /

TCP and UDP are the two dominant transport-layer protocols underpinning all internet communication. TCP provides reliable, ordered, connection-oriented delivery with flow control and congestion control. UDP provides unreliable, unordered, connectionless delivery with minimal overhead. Choosing between them is a fundamental systems design decision that affects latency, throughput, reliability, and implementation complexity. Every real-time system — video games, video calls, DNS, HTTP/3 — has made a deliberate choice between these protocols.

TCP: Reliable Delivery Mechanisms

TCP reliability comes from: Three-way handshake: SYN → SYN-ACK → ACK establishes the connection and synchronizes sequence numbers before data transfer. Sequence numbers and ACKs: every byte is numbered; the receiver ACKs received bytes; the sender retransmits unacknowledged segments after a timeout. Sliding window: the receiver advertises a receive window (rwnd) — how many bytes it can buffer — preventing sender overflow. Congestion control: slow start, congestion avoidance, fast retransmit, fast recovery (CUBIC, BBR algorithms) prevent network overload. Flow control: sender limits transmission rate to match receiver processing speed. Connection teardown requires four-way FIN/ACK exchange. Head-of-line blocking: a lost packet blocks delivery of all subsequent packets in the stream until retransmitted.

# TCP connection state machine (simplified)
# Client:  SYN_SENT → ESTABLISHED → FIN_WAIT_1 → FIN_WAIT_2 → TIME_WAIT → CLOSED
# Server:  LISTEN → SYN_RCVD → ESTABLISHED → CLOSE_WAIT → LAST_ACK → CLOSED

# TCP header: 20 bytes minimum
# Source port (2) | Dest port (2) | Seq num (4) | Ack num (4)
# Data offset | Flags (SYN/ACK/FIN/RST/PSH/URG) | Window size (2)
# Checksum (2) | Urgent pointer (2) | Options (variable)

# Key TCP socket options:
# SO_KEEPALIVE: detect dead connections (sends probes)
# TCP_NODELAY: disable Nagle algorithm (send small packets immediately)
# SO_REUSEADDR: allow port reuse immediately after TIME_WAIT
# TCP_BACKLOG: max pending connections before accept() (default 128)

UDP: Low-Overhead Datagram Delivery

UDP header is only 8 bytes (source port, dest port, length, checksum). No connection state, no handshake, no retransmission. Each UDP datagram is independent — no ordering, no duplicate detection. Why choose UDP? Latency: no RTT for handshake, no head-of-line blocking. A single lost packet does not block subsequent packets. Real-time tolerance: for video calls, a late frame is worse than a dropped frame — retransmission is useless. Broadcast/multicast: send one packet to many receivers (TCP is point-to-point). Custom reliability: build only the reliability semantics you need (QUIC, WebRTC, game networking libraries implement selective reliability over UDP).

QUIC: HTTP/3 over UDP

QUIC (HTTP/3) reimplements transport reliability over UDP to solve TCP limitations. Key improvements: 0-RTT connection establishment: subsequent connections to known servers can send data immediately (cached TLS session resumption). No head-of-line blocking: QUIC multiplexes multiple independent streams over one connection; a lost packet only blocks the affected stream, not all streams. Connection migration: QUIC connections have their own connection IDs separate from IP+port; when a mobile client switches from WiFi to cellular, the QUIC connection survives without reconnection. Google reports 15% improvement in page load times using QUIC vs. TCP+TLS for tail latencies.

When to Use TCP vs. UDP

Use TCP for: file transfers (FTP, SCP), HTTP/1.1 and HTTP/2, database connections (PostgreSQL, MySQL, Redis), email (SMTP, IMAP), SSH — any protocol where data integrity and ordering are required. Use UDP for: DNS queries (single request/response, retry on timeout), DHCP, NTP (time sync), video/audio streaming (WebRTC, RTP), online game state updates, IoT sensor telemetry where a missed reading is acceptable. Use QUIC/HTTP/3 for: web traffic where both low latency and reliability are needed, especially for mobile clients with poor connectivity or frequent network changes.

Key Interview Discussion Points

  • TIME_WAIT state: after closing a TCP connection, the client waits 2×MSL (typically 60-120s) before reusing the port — prevents delayed packets from a prior connection being misinterpreted by a new connection on the same port
  • Nagle algorithm: TCP buffers small writes and sends them as one packet to reduce overhead; disable with TCP_NODELAY for latency-sensitive apps like game state updates or terminal emulators
  • BBR congestion control: Google’s BBR estimates bottleneck bandwidth and RTT to maintain a full pipe without causing queue buildup (bufferbloat); replaces loss-based CUBIC for long-distance connections
  • UDP reliability layers: QUIC, WebRTC data channels, and game networking libraries (ENet, GameNetworkingSockets) implement selective reliability, ordering, and congestion control on top of UDP
  • Multicast: UDP supports IP multicast (one sender to many receivers simultaneously) — used in financial market data feeds, live streaming protocols, and cluster membership protocols
Scroll to Top