# Where the microseconds go in a trading-platform design round

Source: https://www.techinterview.org/post/3233476395/trading-platform-system-design-matching-engine/
Updated: 2026-07-16 · techinterview.org

The quickest way to fail a trading-platform design round is to reach for a thread pool. Say "I'll shard the order book across worker threads for throughput" and an interviewer who has actually shipped a matching engine, whether at Citadel Securities, an exchange team, or a crypto venue, already has your number. A matching engine that honors price-time priority wants exactly one thread touching the book. Almost all the real design work is arranging everything around that thread so it never blocks, never allocates, and never waits on a disk.

That one sentence reframes the whole problem. You're not designing for parallelism. You're designing a deterministic pipeline where a market order and a cancel that arrive 200 nanoseconds apart always resolve in the same order, on every replay, forever. Get that framing out loud in the first two minutes and the rest of the interview is you filling in a picture the interviewer already believes.

## The single-threaded core, and why that's the fast choice

People assume single-threaded means slow. For a matching engine it's the opposite. A modern CPU core can run tens of millions of matches per second if you keep the working set in L1 and L2 cache and never take a lock. The instant you add threads to the book you need locks or lock-free structures, and both bring nondeterminism: the way two threads interleave depends on cache-coherency traffic you don't control. An exchange can't live with that. Members and regulators expect that the same input sequence produces the same trades, which means the match step has to be a pure function of a totally ordered input stream.

The pattern most people converge on is the one LMAX published years ago with its Disruptor: a ring buffer feeding a single business-logic thread, with input decoding, journaling, and output encoding running on separate cores that hand work to that thread through the ring. Nasdaq's INET, a descendant of the old Island ECN sequencer design, works on the same principle and still clears billions of messages a day. If you can name that lineage in the room, do it. It signals you know this is a solved shape, not something you're inventing on the spot.

## The sequencer is where correctness and latency collide

Before an order reaches the book it passes through a sequencer: the component that stamps every inbound event with a monotonic sequence number and writes it to a durable journal. This is the part candidates most often skate past, and it decides whether the rest works.

The sequencer does two jobs that pull against each other. It has to persist every event before acting on it, so that after a crash you can replay the journal and rebuild the exact book state. And it has to do that in a couple of microseconds so it isn't the bottleneck. You can't call fsync per message and hope. The real answers are batching writes to an append-only log on NVMe, replicating the sequenced stream to a hot standby over a low-latency link before acknowledging, or both. Say out loud that you acknowledge only after the event is safely replicated, and explain the tradeoff: a few microseconds of added latency buys you a standby that can take over mid-session without dropping a single order.

Determinism falls out of this once the sequencer owns ordering. The matching engine consumes the sequenced stream and nothing else. No wall-clock reads, no random tie-breaks, no querying an external service mid-match. If you need a timestamp in a fill, it comes from the sequenced event, not from a gettimeofday call inside the matcher. That discipline is what lets you replay a whole trading day bit-for-bit to settle a dispute about who got filled first.

## Price-time priority and the data structure they'll push on

Once ordering is settled, the interviewer usually zooms into the book itself. The question underneath is always the same: how do you find the best bid and ask, and the resting orders sitting at those prices, in close to constant time?

A sorted map keyed by price, a red-black tree, is the textbook answer and it's fine for a first pass: O(log n) to locate a price level, then a FIFO queue per level for time priority. But if you're interviewing for a latency-sensitive seat, push further. Prices are discrete ticks over a bounded range, so you can replace the tree with an array indexed by price level and keep pointers to the current best bid and ask. Now hitting a level is O(1) and cache-friendly, and you only walk outward when a level empties. Each level holds an intrusive doubly linked list of orders, so a cancel is O(1) once you've found the order by id, which you keep in a separate hash map from order id to node.

Here's the detail that separates people who've built one from people who've read about it. Cancels and modifies dominate real order flow, often more than 90% of messages, and most of them never trade. Your design should make cancel the cheapest operation, not matching. If your answer optimizes the match path and treats cancel as an afterthought, you've optimized the rare case and slowed the common one.

One more thing worth raising before they ask: not every venue runs strict price-time. Several CME futures match pro-rata, splitting an incoming order across resting orders at the best price in proportion to their size. Saying you'd make the allocation policy a pluggable strategy at a single price level shows you're treating the engine as a product with multiple markets, not a single homework problem.

## Market data fanout without stalling the matching thread

Every match and every book change has to reach subscribers, and there can be thousands of them. The rule that keeps the system fast is that the matching thread never touches a socket. It publishes book deltas into a ring, and a separate set of publisher threads encode and send them. The matcher's job ends the moment it drops an event on the ring.

Fanout itself is usually UDP multicast on the exchange's own network. The matcher publishes once, the switch fabric replicates to every subscriber, so cost doesn't grow with the number of listeners. Because UDP drops packets, you attach a sequence number to every market-data message and run two identical feeds, an A line and a B line, on separate paths. Clients arbitrate between them, use the sequence gaps to detect loss, and then either request a retransmit or pull a fresh snapshot from a recovery service. That snapshot-plus-incremental scheme is what lets a client who just connected build the full book without replaying the whole day.

Conflation is the tradeoff to name here. A retail-facing feed can collapse ten updates to a price level into the latest one, because a phone doesn't need every tick. A co-located firm needs every message in order. Good designs run more than one market-data product off the same event stream, tuned for different consumers, instead of pretending one feed serves everyone.

## Where the microseconds actually go

When the interviewer asks where latency hides, they want to hear that you know the enemy is variance, not the average. A p50 of 5 microseconds with a p99.9 of 2 milliseconds is a worse system than a steady 8 microseconds, because the tail hits exactly when the market is moving against you. Almost every bad tail traces back to something stalling that one thread.

The usual suspects are a garbage-collector pause (which is why these engines are written in C++, Rust, or very carefully allocation-free Java), a page fault on memory you forgot to pre-touch, a syscall sneaking onto the hot path, false sharing where two cores fight over the same cache line, or a log write that blocked. The fixes are mechanical once you name the cause: pin threads to isolated cores, pre-allocate and reuse every object, bypass the kernel network stack with something like Solarflare OpenOnload or DPDK, and move logging off the critical path by writing binary records to a ring that another thread drains to disk.

The numbers below are rough and swing with hardware, but the shape is what carries weight in an interview. The fixed cost of getting on and off the wire dwarfs the actual matching, which is why shaving nanoseconds off a book update buys you far less than not touching the kernel at all.

| Stage inside the exchange, order arrival to acknowledgment | Approximate budget | What dominates the cost |
| --- | --- | --- |
| Wire to application at the order gateway (kernel-bypass NIC) | 0.8-2 us | DMA and poll-mode driver, no interrupt or syscall |
| Session decode and pre-trade risk check | 200-500 ns | Parsing plus lookups in position and limit tables |
| Sequencer assigns sequence number and durably journals | 1-5 us | Replication to the standby before acknowledging |
| Matching engine matches, generates fills, updates book | 100-500 ns | Cache locality of the order book |
| Encode execution report and push back to gateway | 0.8-2 us | Serialization and DMA back to the wire |
| Market-data delta published to the multicast fabric | 1-2 us | One publish, the switch replicates to all subscribers |
| Total order-to-acknowledgment on a competitive venue | about 5-12 us | Fixed wire and durability cost, not matching |

If you want to rehearse, these land close to how the questions are actually phrased in a system design round for an exchange or a low-latency trading team:

- "Walk me through exactly what happens, component by component, when a marketable limit order arrives and crosses the spread."

- "Two co-located clients report a different top-of-book at the same wall-clock instant. Bug, or expected? Defend your answer."

- "Your matching engine crashed at 10:31:04. Walk me through recovery, and tell me how you know you lost nothing."

- "Where do the pre-trade risk checks live, and what do they cost you on the hot path?"

The candidates who do well here aren't the ones who list the most components. They're the ones who keep returning to the two invariants that make an exchange an exchange: one totally ordered input stream, and a match step that's a pure function of it. The multicast feeds, the A/B lines, the risk gateway, the hot standby, all of it is plumbing you bolt around that spine without ever letting it touch the matching thread. Design outward from the invariant and you'll answer half the follow-ups before they're asked.