Optiver and IMC both like a version of the same opening question for low-latency roles: a UDP multicast market-data packet lands on your network card, walk me through everything that happens before your trading strategy sees the price. Candidates who stall treat the network as a black box that produces bytes. The ones who get a callback know that the default answer, the kernel receive path, is the slow answer, and they can say exactly why.
Start with what the operating system actually does. The card copies the frame into a ring buffer in host memory over DMA, then raises a hardware interrupt. The CPU stops what it was doing, jumps to the driver’s interrupt handler, and schedules a softirq to do the heavier work a moment later. That softirq allocates an sk_buff, walks it up through the IP and UDP layers, and drops the payload into the receive queue of your socket. Your thread was probably asleep in recvmsg, so the kernel marks it runnable and waits for the scheduler to put it back on a core. When it does, one more copy moves the bytes from kernel memory into your buffer. Only now does your code run.
Count the expensive parts: an interrupt, at least one context switch, two copies, and a trip through a general-purpose stack that also carries your SSH session and your logging traffic. On a tuned box the mean is somewhere around five to fifteen microseconds from wire to user space. The mean is not what gets you fired. The tail is. An interrupt that lands while the core is busy, a scheduler that parks your thread for 200 microseconds because a cron job woke up, a page fault on a buffer you assumed was resident. Interviewers who do this work ask about p99 and p99.9 for a reason: a market maker quoting on prices that are a few microseconds stale during a burst is exactly when adverse selection eats the desk’s edge.
What “bypass the kernel” actually means
Kernel bypass maps the card’s receive rings directly into your process’s address space and lets you read packets with the kernel out of the path. No interrupt, because you poll. No context switch, because your thread never sleeps. No copy into a socket buffer, because you read the frame where the card left it. You give up a whole CPU core, which now spins in a tight loop checking for new descriptors, and you get latency that drops to roughly a microsecond and, more to the point, stops jittering.
The polling part is the trade that trips people up. A busy-poll loop burns 100% of a core doing nothing most of the time. That is the price. You pin that thread to an isolated core with isolcpus or a cpuset so the scheduler never drops anything else onto it, you allocate its memory on the same NUMA node as the card, and you back it with huge pages so the TLB stops missing. A sleeping thread wins on throughput per watt. A spinning thread wins on predictable latency, and this desk picks predictable latency every time.
Picking a bypass path
There is no single “fast” answer here, and a good interviewer wants to hear you weigh the options against how much control and portability you’re willing to trade. The rough shape of the field:
| Receive approach | Typical wire-to-user latency | How you use it | Main tradeoff |
|---|---|---|---|
| Kernel stack with SO_BUSY_POLL | 3 to 8 microseconds | A socket option plus interrupt and affinity tuning | Easy and portable, but still pays for copies and the full stack |
| AF_XDP (Linux 4.18+) | 1 to 3 microseconds | An XDP program that steers frames into a user-space ring | Open source and in-tree, but driver support varies and you write more plumbing |
| DPDK | ~1 microsecond | A full user-space poll-mode driver that owns the NIC | Vendor-neutral and fast, but it takes the whole card and a large codebase |
| Solarflare ef_vi | Sub-microsecond | A raw layer-2 API to the card’s rings and event queue | Lowest latency, but ties you to the hardware and hands you everything above the frame |
| OpenOnload | 1 to 2 microseconds | An LD_PRELOAD shim that services normal socket calls in user space | Transparent, no code change, but less control than ef_vi |
Onload gives you sockets, ef_vi gives you the wire
A question that separates people who have read about this from people who have shipped it: you have a Solarflare card, do you use OpenOnload or ef_vi? Onload is a shim you load with LD_PRELOAD. Your code still calls socket, bind, and recv, and Onload quietly services those calls against the card from user space. You change nothing and you keep most of the win. ef_vi is a lower-level API where you manage the receive rings, the event queue, and the packet buffers yourself, and you speak raw Ethernet frames. It is faster because there is less code between you and the DMA, and it is more work because there is less code between you and the DMA. The answer that lands in an interview is that you reach for Onload first, measure, and drop to ef_vi only for the one path where the last few hundred nanoseconds pay for the extra code.
The phrasings recur across desks. A few that show up close to verbatim:
- “Why is the kernel network stack too slow for market data, specifically?”
- “You’ve got a Solarflare card. Onload or ef_vi, and why?”
- “Your p99 tick-to-trade doubled overnight. How do you tell whether it’s the network or your code?”
- “Sketch a lock-free single-producer single-consumer queue between the poll thread and the strategy thread.”
- “When would you not bypass the kernel?”
You can’t tune what you don’t timestamp
Every serious low-latency shop hardware-timestamps packets at the card. The NIC stamps each frame with its own clock the instant it arrives, before any of your software runs, and that clock is disciplined with PTP so it agrees with the exchange clock to within tens of nanoseconds. Now you can measure the thing that matters, the gap between when the price hit your wire and when your order left it, without your own software clock lying to you about it. A common follow-up is how you would prove a latency regression came from the network rather than your strategy. You compare the hardware receive timestamp against the timestamp on your outbound packet, and if that segment is flat, the regression is in your code.
The kernel was never your only problem
Bypass gets the packet to your code quickly. What your code does next is where most software latency actually hides, and interviewers steer here once they see you know the network path. Allocating memory on the hot path calls into the allocator and can fault, so you preallocate pools and reuse them. Chasing pointers through a tree scattered across the heap misses L1 and L2 and pays about a hundred nanoseconds per trip to main memory, so you lay data out flat and contiguous. A branch the CPU predicts wrong flushes the pipeline, so you keep the hot path straight and push the rare cases out of line. Logging with a lock or a write in the middle of quoting is a self-inflicted stall, so you push records into a preallocated ring buffer and drain it from another core. None of that is exotic. It is the difference between a system that looks fast on a slide and one that is fast on the wire.
There is a floor you cannot cross in software, and good interviewers want to see that you know where it sits. The fastest tick-to-trade paths do not run on a CPU at all. They run on an FPGA wired between the fiber and the host, parsing market data and firing a pre-canned order in tens of nanoseconds, with the software layer behind it handling the decisions that can afford a microsecond. Claim you will beat that in C++ and you have told them you do not know the game.
The other thing they are listening for is restraint. Most systems that ask for kernel bypass do not need it. If your strategy holds positions for seconds and your edge lives in the model rather than the queue position, the kernel stack with busy-poll tuning is plenty, and you have saved yourself a core, a pile of hardware-specific code, and a debugging surface where a dropped descriptor becomes a silent gap in your book. The candidate who says “I would measure the kernel path first and bypass only the segment that needs it” usually knows more than the one who reaches for ef_vi on the first slide.
Keep sharpening your system design:
