# Kafka interview questions that separate users from operators

Source: https://www.techinterview.org/post/3233477380/kafka-interview-questions/
Updated: 2026-09-01 · techinterview.org

Kafka interviews probe four things: partitions (ordering and parallelism live here, and only here), consumer groups (one partition maps to one consumer in a group, so partition count caps your parallelism), rebalancing (where "why does my consumer keep dying" comes from), and delivery semantics (at-most-once, at-least-once as the sane default, exactly-once only inside Kafka via idempotent and transactional producers). Strong candidates answer with the config knob and the failure mode, not the definition.

The fastest way to fail a Kafka interview is to recite that "Kafka is a distributed commit log" and stop there. Anyone can memorize the topology. What the interviewer is actually testing is whether you've been paged at 3am because a consumer group wouldn't stop rebalancing, or because one partition was sitting at 40 million messages of lag while the other eleven were empty. The questions are designed to find out if you've operated the thing or just read about it.

Here's the set of questions that come up again and again, and how a strong candidate reasons through them out loud.

## "A topic has 12 partitions. Explain what that number actually controls."

This is the warm-up, and it's a trap for people who answer "it's for scale." Partitions control two coupled things: ordering and parallelism. Kafka guarantees message order *within* a partition and makes no promise across partitions. So the partition is your ordering boundary. It's also your unit of parallel consumption, which is the other half of the answer.

A message goes to a partition by hashing its key (murmur2 of the key, mod the partition count). A null key doesn't spread records evenly per message the way people assume: since Kafka 2.4 the default is the sticky partitioner, which fills one partition's batch then rotates, so keyless records land in bursts per partition rather than one-at-a-time round-robin. The consequence a strong candidate reaches for immediately: if you need all events for a given user processed in order, you key by user ID, and every event for that user lands on the same partition and is read in order. If you key by something low-cardinality, or leave a hot key like a single popular product ID, you get **partition skew**: one partition carrying most of the traffic while the rest idle. That partition's consumer becomes the bottleneck, its lag climbs, and your dashboards still show healthy average utilization.

The follow-up almost always comes: "Can you just add partitions later?" Yes, and it's a landmine. Adding partitions changes the key-to-partition mapping (the mod changes), so a key that used to land on partition 3 now lands on partition 7, and ordering for existing keys breaks across the resize point. If ordering matters, you over-provision up front rather than resize a live topic. That single insight ("adding partitions silently breaks per-key ordering") is the kind of thing that tells the interviewer you've actually shipped this.

## "You have 12 partitions and you start 15 consumers in one group. What happens?"

Three consumers sit idle. Within a consumer group, each partition is assigned to exactly one consumer, so the number of partitions is a hard ceiling on useful consumers in that group. Twelve partitions means at most twelve consumers doing work; the extra three are hot spares that only pick up load if one of the active twelve dies.

The clean framing: partitions scale consumers, consumer groups scale *independent* readers. Two services that both need every message each get their own group and their own offset cursor. That's why Kafka is pub/sub and a queue at once: multiple groups fan out, multiple consumers in one group share the work.

## "Your consumers keep rebalancing and throughput tanks. Debug it."

This is the operator question, and it's where most candidates thin out. A rebalance is Kafka reassigning partitions across the consumers in a group. It happens when a consumer joins, leaves, or is presumed dead. During a stop-the-world (eager) rebalance, every consumer stops processing while assignments are recomputed. Do that in a tight loop and you have a **rebalance storm**: the group spends more time reassigning than consuming, and lag climbs even though nothing is technically down.

The usual root cause isn't crashes. It's a consumer that takes too long between polls. Kafka decides a consumer is dead two different ways, and candidates constantly conflate them. `session.timeout.ms` covers the background heartbeat: miss heartbeats and you're kicked. `max.poll.interval.ms` covers processing: if your handler takes longer than this between calls to `poll()`, the broker assumes you're wedged and triggers a rebalance, even while heartbeats are still flowing. So a slow downstream call inside your message handler, or a batch that grew too large, kicks the consumer, which triggers a rebalance, which redistributes load onto consumers that then also fall behind. That's the storm.

The reasoning an interviewer wants to hear: first, check whether processing time per poll batch is bumping against `max.poll.interval.ms`, and either shrink `max.poll.records` or move slow work off the poll thread. Second, if the churn is from deploys and pod restarts, use **static group membership** (`group.instance.id`) so a consumer that restarts within the session window reclaims its old partitions instead of forcing a reassignment. Third, prefer the cooperative sticky assignor (the incremental rebalance protocol) so a rebalance only moves the partitions that need to move instead of revoking everything. Naming that protocol shift, from eager stop-the-world to cooperative incremental, is a strong signal. For a 2026 dedicated round, also know the newer broker-coordinated protocol (KIP-848, GA in Kafka 4.0), which moves rebalance logic server-side so the whole group no longer stops to reconcile assignments; Kafka 4.0 also finished swapping ZooKeeper for KRaft, so a nod to either shows you've tracked where the platform is going.

## "Walk me through at-most-once, at-least-once, and exactly-once."

The definitions are table stakes. The differentiator is that each semantic is a *choice about when you commit the offset relative to when you do the work*, and each has a concrete failure mode.

| Delivery guarantee | How you get it | Failure mode it accepts | When to pick it |
| --- | --- | --- | --- |
| At-most-once | Commit the offset before processing the message | Crash after commit, before work: message is lost, never redelivered | Metrics, sampled telemetry, anything where a dropped record is cheaper than a duplicate |
| At-least-once | Commit the offset after processing succeeds (the default) | Crash after work, before commit: message is redelivered and reprocessed | Almost everything; pair it with idempotent handlers so duplicates are harmless |
| Exactly-once (within Kafka) | Idempotent producer + transactions: consume, process, and commit offsets in one atomic transaction | None inside Kafka, but does not extend to an external database or API unless that side is also transactional or idempotent | Kafka-to-Kafka pipelines and stream processing where duplicates corrupt results (counters, joins) |

At-least-once is the right default, and the real framing is that you make duplicates *harmless* rather than making them impossible. You design idempotent consumers: dedupe on an event ID, use upserts keyed by a natural key, make the operation naturally repeatable. That's cheaper and more robust than reaching for exactly-once everywhere.

This is also the answer if you've only used Kafka as a client and never operated a cluster. You don't need a war story here. "At-least-once plus idempotent handlers" is what a lot of production systems actually run, and saying why (duplicates are cheap to absorb, lost messages usually aren't, and exactly-once buys less than it costs once a database is in the path) reads as judgment, not thin experience. Faking operational fluency is what gets caught; reasoning from the tradeoff doesn't.

## "So what does exactly-once actually buy you, and where does it stop?"

Two mechanisms, and interviewers like to see you separate them. The **idempotent producer** (`enable.idempotence=true`, on by default since Kafka 3.0) stops producer retries from writing duplicates: the broker tracks a producer ID and a per-partition sequence number and drops a record it has already seen. That handles the "my send timed out so I retried and now there are two" case. It does *not* give you end-to-end exactly-once by itself.

For that you need **transactions**: a producer with a `transactional.id` writes its output records and its input offsets as one atomic unit, and consumers reading downstream set `isolation.level=read_committed` so they never see records from an aborted transaction. This is the machinery under Kafka Streams' exactly-once processing, and if you want to see how it's implemented at the storage layer, [the low-level design of Kafka's internals](https://www.techinterview.org/post/3233473237/lld-kafka-advanced-internals/) walks through it. The critical caveat, and the thing that separates a real answer from a memorized one: exactly-once is a guarantee *within the Kafka boundary*. The moment you write to Postgres or call a payment API inside your consumer, Kafka's transaction can't cover that side. You either use the transactional outbox pattern or make the external write idempotent. Candidates who claim Kafka gives you exactly-once to your database are wrong, and good interviewers will push exactly there.

Worth adding: durability is a separate axis from delivery semantics. `acks=all` plus `min.insync.replicas=2` stops acknowledged writes from vanishing when a broker dies, and none of the exactly-once machinery helps if you've acked writes that only landed on one replica.

## "How do you know a consumer is falling behind?"

Consumer lag: the log-end offset of a partition minus the consumer group's committed offset for it. It's the single most important health metric because it's the direct measure of "how stale is my data right now." Rising lag on one partition while the others are flat points straight back to partition skew or a slow handler on that assignment. Flat-but-high lag across all partitions means you're under-provisioned on consumers, and since partition count caps consumers, the fix might be a topic redesign rather than more pods.

The offset detail that trips people up: committed offset is the offset of the *next* message to read, not the last one processed, and off-by-one bugs here cause either skipped messages or silent reprocessing. And auto-commit (`enable.auto.commit=true`) commits on a timer regardless of whether your processing actually finished, which quietly turns your at-least-once system into an at-most-once one during a crash. If ordering and durability matter, you commit manually after the work lands.

## How this shows up across a loop, and what to study next

In a [system design interview](https://www.techinterview.org/system-design-interview-guides/), Kafka usually appears as a component inside a larger problem (an [ad-click aggregator](https://www.techinterview.org/post/3233476913/design-ad-click-aggregator/) or event pipeline), and the interviewer probes your partition-key choice and your delivery guarantee. In a data or infra screen it's a dedicated Kafka round with exactly these operational questions. Either way, the pattern holds: name the config knob and the failure mode together.

If you're weighing Kafka against other brokers, that's a different question with its own tradeoffs, covered in [Kafka vs RabbitMQ](https://www.techinterview.org/post/3233476392/kafka-vs-rabbitmq/). If you want to understand why these guarantees are hard to build rather than just how to use them, the [design-a-message-queue walkthrough](https://www.techinterview.org/post/3233476899/design-message-queue-hard-parts/) derives them from first principles. And if you're targeting a streaming-focused company, the [Confluent interview rounds](https://www.techinterview.org/post/3233476180/confluent-engineering-interview-rounds/) guide covers what that specific loop looks like. To turn this into a study schedule, the [study plan generator](https://www.techinterview.org/study-plan/) can slot these topics against your timeline.

The tell an experienced interviewer listens for is small. Say "at-least-once, and I make the consumer idempotent" and they know you've run this in production. When they ask about rebalancing and you ask back "is it churning on deploys or on slow processing?", the interview is basically over, in your favor.