Gimlet Labs sells the pitch that no single chip is good at every part of running a large model. Prefill, the pass that reads your whole prompt, is compute-bound and wants raw FLOPS. Decode, the token-by-token generation that follows, is memory-bandwidth-bound: every generated token streams the model’s weights out of HBM, and the KV cache only dominates that traffic at long context or large batch. Tool and agent calls are network-bound and mostly wait. Per its Series B announcement, Gimlet’s software disaggregates a model so each of those phases lands on the silicon that fits it, then stitches the results back into one response. Its two public products are Gimlet Cloud, serverless inference aimed at AI agents, and kforge, which autonomously generates optimized low-level kernels straight from PyTorch. The company says the platform yields three-to-ten-times speedups at the same cost and power, on hardware that otherwise runs at 15 to 30 percent utilization. The $300 million Series B sits on top of roughly $92 million raised earlier, including an $80 million Series A led by Menlo Ventures, for $392 million in total.
The founding team is the tell for what this company values. CEO Zain Asgar and co-founders Michelle Nguyen, Omid Azizi, and Natalie Serrino worked together at Pixie, the Kubernetes observability startup acquired by New Relic in 2020. That is a systems-and-compilers pedigree, not a model-training one: Pixie was eBPF and query engines running at fleet scale with almost no overhead. Gimlet is hiring for compiler, distributed-systems, and GPU-architecture roles, and the interview leans on exactly those muscles. One practical warning before you research it: searching “gimlet interview” pulls up Gimlet Media, the podcast company Spotify bought, and gimlet.io, a Kubernetes GitOps tool, and none of that is this company. Anchor on Gimlet Labs, Zain Asgar, and multi-silicon inference.
Why the questions look the way they do
The whole product is a bet that heterogeneity beats a homogeneous fleet, and that bet sets the interview apart from its neighbors. Serverless-GPU and inference-API companies like Modal, Fireworks AI, and Together AI mostly serve models on large, uniform NVIDIA fleets, so their loops center on getting the most out of one well-understood architecture. Gimlet’s premise is that prefill might run on an NVIDIA GPU while decode runs on a wafer-scale Cerebras part or an SRAM-heavy d-Matrix card whose on-chip SRAM sidesteps the HBM bandwidth ceiling that bounds decode on a GPU, with an Arm or Intel CPU orchestrating and the KV cache moving between them mid-request. That means the questions push on cross-vendor compiler work, on scheduling across chips with different instruction sets and memory models, and on the arithmetic of moving state between them without giving back the speedup you just won. If you have only ever reasoned inside CUDA on one card, the heterogeneous framing is where a Gimlet loop will find the edge of what you know.
What the Gimlet Labs interview loop probably contains
No candidate has posted a Gimlet interview report as of September 2026; the company is small and only came out of stealth in October 2025. So treat the stages below as a representative inference-infra loop reconstructed from how comparable companies hire for these roles. Confirm the real shape with your recruiter on the first call, since an early-stage startup changes its loop often. That call also covers work authorization, and Gimlet has not published a visa-sponsorship policy, so ask about sponsorship then if you need it.
| Representative Gimlet Labs interview stage (reconstructed, not reported) | Format and length | What it screens for |
|---|---|---|
| Recruiter screen | ~30 min phone | Background, which track you fit (compiler, distributed systems, GPU/kernels), work authorization, on-site expectations |
| Hiring-manager technical call | 45-60 min | A systems project you owned end to end, the bottleneck you found, how you measured it, what the numbers were |
| Role-specific technical screen | 60 min | Compiler IR and codegen, an inference-serving design, or a CUDA/kernel problem, depending on track |
| On-site: coding rounds | 2 rounds, ~45-60 min each | Practical data-structure and concurrency problems in C++, Rust, or Go, not competitive-programming trivia |
| On-site: system-design or deep-dive round | ~60 min | Design a phase-disaggregated inference path, or go deep on your specialty across the silicon boundary |
| Behavioral / values round | ~45 min | Ownership, moving fast in a small org, collaborating across compiler, runtime, and hardware teams |
What a loop like this rewards is whether your numbers survive follow-up. Infra interviewers chase a number until it breaks: say a kernel is memory-bound and they ask for the arithmetic intensity that makes it so, name a latency and they ask which stage of the pipeline owns it. Come with the exact figures from your past work; the round ones fall apart the moment someone asks why.
Compiler and code-generation roles
This is where Gimlet’s problem gets hardest, because supporting six vendors means lowering one model graph to many backends. Expect to reason about lowering a model from a framework IR to CUDA, ROCm, a Cerebras or Gaudi SDK, and Arm, and about what happens when a target lacks an operator the graph needs. Representative ground: operator fusion and when it stops paying off, memory planning and buffer reuse across a compute graph, how you tile a matmul for a given cache hierarchy, autotuning a kernel across shapes, and quantization from FP16 down to FP8 or INT4 with the accuracy tradeoffs spelled out. A common prompt in this space is to design the fallback path for an op one accelerator does not support, deciding whether to emulate it, offload it to the CPU, or reshape the graph so the op disappears. Gimlet’s kforge generates kernels autonomously from PyTorch using a multi-agent search with strict correctness checks, so expect questions about how you would prove a machine-written kernel is correct and how you would search a space of kernel designs without a human judging each one. Compiler-flavored coding rounds tend to involve graph or tree transforms, so a refresh on time and space complexity and the usual coding patterns covers the algorithmic slice, though the domain reasoning is what separates candidates.
Distributed systems and inference orchestration roles
Someone has to route a request across chips and keep the pieces coherent, and that is the distributed-systems track. The signature problem is the KV cache handoff: prefill produces the cache on one device, decode consumes it on another with a different memory model, and you have to move it across NVLink, PCIe, or RDMA over Ethernet without eating the latency budget. Representative prompts: design phase-aware request routing that sends prefill and decode to different pools and still meets a time-to-first-token and time-per-output-token target; implement continuous batching so new requests join an in-flight batch without stalling the ones already running; and handle a decode node dying mid-generation, deciding whether you can recover the KV cache or must replay prefill. These roles usually include a practical coding round in C++, Rust, or Go touching concurrency and lock-free structures, and a system-design round; our system-design interview guides cover the general frame, but rehearse the disaggregated-serving version specifically.
GPU architecture and kernel roles
The lowest-level track is about wringing performance out of individual chips and the links between them. Expect roofline reasoning, where you place a kernel against a chip’s compute and bandwidth ceilings and argue whether it is compute- or memory-bound. Be ready to do HBM-bandwidth arithmetic for a decode step, estimate occupancy, and talk through writing an attention kernel in CUDA or Triton, including why FlashAttention avoids materializing the full attention matrix. kforge targets CUDA, ROCm, and Metal, so NVIDIA, AMD, and Apple GPUs; a portable-kernel question across those three backends would fit this track. Interconnect questions are fair game because the KV handoff lives on them: NVLink versus PCIe versus Ethernet bandwidth and latency, and how each caps the phase split. Numerics come up too: know where FP8 and INT4 cost you accuracy and where they are free.
How to read the comp
Gimlet posts no public salary band that was reachable while writing this, so anchor to the inference-infra peers it competes with for the same engineers. As of September 2026, senior infrastructure and ML-systems roles at companies like Modal, Fireworks, Together, and Baseten, and at the chip firms Gimlet recruits from, run on levels.fyi from roughly $180,000 base for a mid-level engineer to $250,000 and up for staff, before equity; treat that as order-of-magnitude, not a Gimlet quote. Under California pay-transparency law the range will appear on the actual job req, so read that on your exact posting and ask the recruiter for the current band on the first call. The offer will be base plus equity, and at a fresh $3 billion price the equity math dominates, so pin down whether it is options or RSUs, the strike or grant price, the vesting schedule, and your level, because leveling moves the number more than negotiation does. Run the package through a total-comp calculator against that valuation, and a salary-negotiation walkthrough covers how to push back. Equity this early is upside, not salary.
How to prepare
Prepare by track, not by grinding generic problems. Compiler candidates should be able to walk a lowering pipeline from framework IR to a specific backend and defend a fusion or quantization decision on real numbers. Distributed-systems candidates should rehearse disaggregated serving out loud: routing, batching, KV-cache movement, and failure mid-request. Kernel candidates should be able to do roofline and bandwidth arithmetic cold and explain an attention kernel line by line. Everyone gets a project deep-dive, so pick one or two efforts you owned and expect the interviewer to keep asking until you land on the measurement that surprised you and what you did about it. The fastest way to sound fluent in phase disaggregation is to read the papers it comes from: DistServe (OSDI 2024) and Splitwise (ISCA 2024) on splitting prefill and decode across separate hardware pools, and Mooncake (Moonshot AI, 2024) on a KV-cache-centric disaggregated serving stack. Spread the prep over a couple of weeks with a lightweight study plan. To see how this loop compares to other frontier-infra bets, the AI-startup interview difficulty index and the broader set of AI-native company interview guides put it in context.
Two things will likely decide a loop like this. Can you own your layer down to the measured numbers, and can you still reason about the chip on the other side of the KV handoff once your part is done. The second is the whole reason this one exists, and it is where a multi-silicon interview will make its call.
Practice the behavioral round:
