The transformer questions that go past the diagram
7 min read The transformer internals interviewers push on, from why attention scales by sqrt(d_k) to KV cache memory and grouped-query attention. Read article
AI/ML Interview Questions
System Design
Designing an LLM serving system that hits its latency budget
6 min read Prefill and decode, the KV cache budget, continuous batching, and autoscaling GPUs, walked through the way the interview actually goes. Read article
System Design
Design an LLM Inference API
6 min read Design an LLM inference API — the service that accepts user prompts and returns model completions, like the OpenAI API, Read article
