tools & reviews

When Kafka beats RabbitMQ, and when it’s the wrong tool

The one-sentence version: Kafka is a durable, replayable log that consumers read at their own pace, and RabbitMQ is a smart broker that routes each message to the right consumer and pushes it out the instant it lands. Almost every real difference between the two falls out of that split.

People reach for the wrong one because both get filed under “message queue,” and both sets of docs talk about producers and consumers. But a Kafka topic and a RabbitMQ queue behave nothing alike once you have real traffic, node failures, and a second team that wants the same data. Here is how to tell which one fits, with the tradeoffs that actually bite in production.

The log versus the broker

Kafka keeps an append-only log per partition. Producers append, the data sits on disk for however long your retention says (hours, weeks, forever), and consumers track an offset and pull batches. Reading a message does not remove it. Ten different consumer groups can read the same partition independently, and a brand-new group can rewind to offset zero and reprocess three months of history. That replay ability is the whole reason Kafka exists.

RabbitMQ works the other way. A producer publishes to an exchange, the exchange routes the message to one or more queues based on rules you define, and the broker pushes it to a consumer. Once the consumer acks, the message is gone. There is no rewind, no history, no second team quietly subscribing without you adding a binding. What you get instead is routing: topic and header exchanges, priority queues, dead-letter exchanges, per-message TTL, and RPC-style request/reply. Kafka has none of that at the broker. You build it in application code or you do without.

So the first question is not “which is faster.” It is whether your problem looks like a stream of events several systems care about, or a stream of tasks that each need to reach exactly one worker with specific routing.

Throughput and latency, with real numbers

Kafka wins throughput and it is not close. On commodity hardware a single Kafka 4.1 broker sustains roughly a million messages per second, because it batches aggressively, writes sequentially, and lets the OS page cache do the heavy lifting. RabbitMQ classic queues top out around 40,000 to 50,000 messages per second per node. The gap comes straight from the design. Kafka trades per-message latency for batch throughput; RabbitMQ tracks and acks every individual message so it can guarantee delivery and routing.

Latency flips the ranking. RabbitMQ pushes a message to a ready consumer in sub-millisecond time under light load, since it never waits to fill a batch. Kafka consumers poll, and with default batching you are usually looking at single-digit to low-double-digit millisecond end-to-end latency. You can tune Kafka’s latency down by shrinking batch size and linger, but you give back the throughput that made you pick it in the first place. For an order-routing service where a caller is blocked waiting on the other end, RabbitMQ’s push model feels sharper. For an analytics firehose, nobody notices five milliseconds.

Ordering and delivery guarantees

Kafka gives you ordering per partition. Route by key (say, user ID) and every event for that user lands in the same partition in order. Across partitions there is no global order, which is the price of horizontal scale. Kafka also supports exactly-once semantics inside its own boundaries through idempotent producers and transactions, which matters for stream processing where a reprocess must not double-count.

RabbitMQ guarantees order within a single queue read by a single consumer. The moment you add competing consumers for throughput, strict order is gone: a slow worker can hold an unacked message while a faster one races ahead. Default delivery is at-least-once, so you design consumers to be idempotent with a dedup key or a database unique constraint. Quorum queues, now the recommended default, put the queue on a Raft group across nodes for strong consistency and clean leader election when a broker dies. They cost more memory and throughput than the old mirrored queues, but they do not drop acked messages the way mirrored queues sometimes did.

One thing that trips people up: exactly-once is not a magic setting in either system. Kafka’s version holds inside Kafka. The instant you write to an external database, the end-to-end guarantee is back on you.

Side by side

Dimension Apache Kafka RabbitMQ
Core model Append-only partitioned log; consumers pull by offset Broker routes messages to queues and pushes to consumers
Peak throughput ~1M msgs/sec per broker (Kafka 4.1, commodity hardware) ~40–50K msgs/sec per node (classic queues); ~1M/sec with Streams on tuned clusters
Typical latency Single-digit to low-double-digit ms (batched polling) Sub-millisecond under light load (push delivery)
Ordering Strict within a partition; keyed routing keeps an entity ordered Strict only with a single queue and single consumer
Retention and replay Kept for a configured window or forever; any consumer can rewind Deleted on ack; no replay for classic or quorum queues
Broker-side routing None; handled in application code Topic/header/fanout exchanges, priority, TTL, dead-letter, RPC
Consensus (2026) KRaft only; ZooKeeper removed in Kafka 4.0 Raft-based quorum queues; native Streams for log-style reads
Best fit Event sourcing, analytics pipelines, high fan-out, replay Task queues, RPC, complex routing, low-latency service-to-service

The “can I add a consumer later” test

Here is a cheap decision rule. Imagine six months from now a data-science team wants every event your service has ever emitted, and they want to reprocess the back catalog to backfill a feature. With Kafka that is a new consumer group and a rewind to an old offset, zero changes on your side. With RabbitMQ that data is gone the moment your original consumers acked it, so the team gets nothing unless you had already been copying events into a store. If future replay and unknown future consumers are plausible, Kafka’s retention earns its operational weight. If each message has exactly one owner and no afterlife, RabbitMQ’s delete-on-ack keeps things simpler and cheaper.

What changed in 2026, and what it means for ops

Kafka 4.0, early in the year, removed ZooKeeper entirely; KRaft is the only consensus mode now. That kills the second distributed system you used to babysit, and it speeds up controller failover and metadata propagation, but it also means older runbooks and tooling that assumed ZooKeeper no longer apply. Standing up new Kafka, you start on KRaft and never think about ZooKeeper again.

RabbitMQ 4.1 leaned harder into quorum queues and native Streams. Streams give RabbitMQ a Kafka-like append-only log with offset tracking and consumer groups, reachable over a dedicated binary protocol that skips AMQP framing overhead. On tuned clusters Streams reach into the million-messages-per-second range, which narrows the throughput story RabbitMQ used to lose on. It does not turn RabbitMQ into Kafka. Streams are newer, the surrounding ecosystem is thinner, and if you mainly want a durable replayable log with a mature stream-processing story, Kafka Streams and Flink are still where the tooling lives.

Day to day, RabbitMQ is the gentler system to run small. A single node, or a three-node quorum cluster, handles a lot before you have to think about it. Kafka wants more care: partition counts you cannot easily shrink, consumer-group rebalancing that stalls under churn, and retention sizing that will fill disks if you look away. Managed offerings blunt this. Confluent Cloud, Amazon MSK, and hosted RabbitMQ options like CloudAMQP take the cluster babysitting off your plate, at a cost that climbs with throughput and retention, so price the specific plan for your volume rather than trusting a headline figure.

Picking for your workload

Reach for Kafka when sustained volume is high (tens of MB/s and up), when you need event sourcing or replay, when many independent consumers want the same stream, when retention is measured in weeks, or when the data feeds a lake, a warehouse, or a stream-processing job. Recommendation engines, clickstream and metrics pipelines, change-data-capture, and audit logs all sit squarely in Kafka territory.

Reach for RabbitMQ when the job is task distribution and coordination between services: a web request drops work on a queue, one worker picks it up, does it once, and acks. Pick it when you need broker-side routing you would rather not hand-roll, priority handling, per-message TTL, dead-lettering for retries, request/reply RPC, or the lowest per-message latency you can get. Background jobs, email and push dispatch, order processing, and command routing between microservices are the sweet spot.

When the workload is modest and there is no real event-streaming requirement, RabbitMQ is usually the faster thing to ship and the cheaper thing to run. Teams that default to Kafka because it sounds serious end up maintaining partitions and consumer groups for a load a single RabbitMQ node would have shrugged off.

Running both on purpose

Plenty of mature systems use each for what it is good at. Kafka becomes the durable event spine that every system reads from and analytics hangs off; RabbitMQ becomes the routing fabric that dispatches specific tasks between services with low latency. Bridging them is routine now, with the Kafka Connect AMQP connector or RabbitMQ’s shovel plugin moving messages across the boundary. If you find yourself contorting one tool to do the other’s job, cross-posting between a Kafka topic and a set of RabbitMQ queues, that is a sign you have two problems, not one, and it is fine to solve them with two tools.

newsletter

What's actually being asked right now

Interview patterns & comp trends, straight to your inbox.

No spam. Unsubscribe anytime.

newsletter

What's actually being asked right now

Interview patterns & comp trends, straight to your inbox.

No spam. Unsubscribe anytime.

1972 Soviet postage stamp commemorating the Mars 2 probe

worth a read

Mars For The Rest of Us — a weekly-or-more deep dive on the technical side of Mars exploration: rocket propulsion, microbiology, mission architecture, and everything in between. Written by Maciej Ceglowski.

Read it on Substack →
Scroll to Top