Low Level Design: JVM Garbage Collection — Algorithms and Tuning

Garbage collection (GC) automatically reclaims memory occupied by objects no longer reachable from the application. The JVM provides multiple GC algorithms with different trade-offs between throughput, latency, and memory footprint. Understanding GC is critical for backend engineers at companies running Java/JVM-based services at scale — GC pauses can cause request latency spikes, and poor GC tuning is a common source of production incidents at high-traffic services.

Generational Hypothesis and Heap Layout

The generational hypothesis: most objects die young. The JVM heap is divided into generations: Young Generation (Eden + two Survivor spaces S0, S1): newly allocated objects go to Eden. When Eden fills, a Minor GC copies surviving objects to a Survivor space (surviving objects alternate between S0 and S1). Objects surviving N Minor GCs (default: 15) are promoted to the Old Generation. Old Generation: long-lived objects. Major GC (or Full GC) collects the Old Generation — much more expensive than Minor GC. Metaspace: class metadata (replaced PermGen in Java 8+).

# JVM GC logging and key flags
java -Xms4g -Xmx4g                # heap size (set equal to avoid resizing)
     -XX:+UseG1GC                  # use G1 collector
     -XX:MaxGCPauseMillis=200      # target max pause time
     -XX:G1HeapRegionSize=16m      # G1 region size
     -Xlog:gc*:file=/var/log/gc.log:time,uptime,level 
     -XX:+HeapDumpOnOutOfMemoryError 
     -XX:HeapDumpPath=/tmp/heap.hprof 
     MyApplication

# Key GC metrics to monitor:
# gc.pause.time (p99 pause duration)
# gc.frequency (how often GC runs)
# gc.heap.used_after_gc (heap utilization trend)
# gc.promotion.failed (objects cannot be promoted = Full GC)

GC Algorithms: G1, ZGC, and Shenandoah

G1GC (Garbage-First): divides heap into equal-sized regions (1-32MB each). Prioritizes collecting regions with the most garbage first (“garbage-first”). Targets a configurable max pause time (200ms default). Good general-purpose collector for heaps 4-16GB. ZGC (Java 15+): concurrent GC with sub-millisecond pauses regardless of heap size. Uses colored pointers and load barriers to perform most work concurrently with the application. Handles heaps up to 16TB. Slightly lower throughput than G1 due to barrier overhead. Shenandoah: similar to ZGC, concurrent compaction, low pause times. Available in OpenJDK builds.

Stop-The-World Pauses and Their Causes

Stop-The-World (STW) pauses halt all application threads while GC runs, causing request latency spikes. Common causes: Allocation rate too high: Eden fills faster than objects die — increase Young Generation size or reduce allocation rate. Object promotion failure: Old Generation full, cannot accept promoted objects — causes Full GC. Increase heap size or reduce object tenure. Humongous allocations: large objects (G1: >50% of region size) bypass Young Generation and go directly to Old Generation — avoid large temporary objects in hot paths. GC overhead limit exceeded: JVM spending >98% of time in GC — memory leak, insufficient heap, or pathological allocation pattern.

Go GC: Tricolor Mark-and-Sweep

Go uses a concurrent tricolor mark-and-sweep GC. Objects are colored: white (unvisited), gray (reachable but children not yet scanned), black (reachable, children scanned). GC starts with all objects white. Roots are marked gray. Gray objects are scanned concurrently with mutator threads (write barriers prevent missed objects). When no gray objects remain, white objects are swept (reclaimed). Go GC targets sub-1ms STW pauses. The GOGC environment variable controls GC frequency: GOGC=100 (default) means GC when heap size doubles since last GC.

Key Interview Discussion Points

• Memory leak in GC languages: objects still referenced by a long-lived collection (map, list) cannot be collected even if the application considers them dead — use weak references or explicit eviction
• GC tuning priority order: first ensure correct heap sizing, then choose the right GC algorithm, then tune algorithm-specific parameters — premature tuning of complex parameters without understanding the workload is counterproductive
• Off-heap memory: use direct ByteBuffers or off-heap allocators (Netty, Agrona) for large caches to avoid GC pressure — memory is managed manually outside the heap
• Allocation profiling: use async-profiler or JFR (Java Flight Recorder) to find hot allocation sites before tuning GC parameters
• Epsilon GC: a no-op GC for latency-sensitive applications that pre-allocate all memory and never need collection — crashes when heap is exhausted