The moment that separates a generative-AI engineer loop from a normal machine-learning loop usually lands in the third round, when the interviewer asks you to design something whose output you can’t grade with one number. A ranking model either hits 0.82 AUC or it doesn’t. A support assistant built on retrieval and a language model can be wrong in a dozen quiet ways that no single score catches, and the interviewer is watching to see whether you already know that before they have to tell you.
Most GenAI roles in 2026 hire off a loop with five recognizable parts, even if the names differ. A recruiter or hiring-manager screen, a practical coding round, a retrieval-and-LLM design round, an evaluation round that used to hide inside system design and now often stands on its own, and a behavioral round. Startups usually add a take-home: build a small RAG service or an agent over a corpus they hand you, then defend the choices live. Meta, the frontier labs, and a lot of Series B startups all run some version of this.
| Interview round | Typical prompt | What a strong candidate does |
|---|---|---|
| Recruiter / hiring-manager screen | “Walk me through a GenAI feature you shipped and how you knew it worked.” | Names a metric they moved and a failure they caught in evaluation, not the stack alone. |
| Practical coding (about an hour) | Chunk a document, dedupe embeddings, or wrap a flaky model API with retries and a token budget. | Treats rate limits, partial failures, and cost as first-class, not the happy path plus a to-do. |
| Retrieval + LLM design | “Design an assistant over 2 million support articles with a two-second latency budget.” | Starts with pgvector and hybrid search, states the cost-versus-recall tradeoff, defers scale until numbers force it. |
| Evaluation | “How do you know a prompt change didn’t make things worse?” | Describes a golden set, the RAG triad, and a regression run before shipping. |
| Behavioral | “Tell me about a model that shipped and then misbehaved in production.” | Talks about monitoring, rollback, and what the evaluation gap turned out to be. |
The coding round is still a coding round
People assume the coding bar drops because the job is prompts and pipelines. It doesn’t. What changes is the flavor. Instead of a graph problem you get something shaped like the actual work: chunk a long document on sentence boundaries without splitting mid-word, dedupe a batch of embeddings by cosine similarity above a threshold, or wrap a flaky model endpoint so it retries on a 429, respects a token budget, and fails cleanly when the context window is blown.
Questions you’ll actually hear:
- “Given a 40-page PDF and a 512-token limit, write the chunker. What do you do with tables?”
- “Here’s a function that calls the model once per row. It times out on large inputs. Fix it.”
- “Cache these embedding lookups. What’s your key, and when do you invalidate it?”
The candidates who pass treat rate limits, partial failures, and cost as first-class concerns, not things they’d ‘add later.’ If your solution assumes the API never errors and tokens are free, that reads as someone who hasn’t run this in production.
The RAG design round, and where people lose it
The prompt is some version of this: design an assistant that answers questions over two million internal support articles, with a two-second latency budget and a fixed monthly inference spend. It sounds open-ended. It is really a test of whether you reach for the boring, measurable option first.
Strong answers start narrow. Chunk the docs, and say why you picked fixed-size, semantic, or structure-aware splitting for this corpus. Embed them. Store the vectors somewhere you can query fast. For a few million vectors on a team already running Postgres, pgvector is a defensible first stop; you graduate to Qdrant, Weaviate, or a managed store like Pinecone when recall or scale actually forces it, not because the name sounds impressive. Then hybrid retrieval, because dense vectors catch meaning while BM25 keyword search catches the exact error code or product name the embedding smears over, and a reranker trims the merged set down to what fits your token budget.
The tradeoff the interviewer wants named out loud is context against cost. Stuff twenty chunks into the prompt and you lift recall, but you pay for every token on every call and you dilute the model’s attention. Rerank down to five and you cut cost and latency but risk dropping the passage that had the answer. The right move is to pick a starting point and say how you’d measure whether it was the wrong one. That last clause is what earns the next round.
The evaluation round is the one that fails people
This is the round that barely existed five years ago and now decides offers. The question is blunt: how do you know your assistant is any good, and how do you know a prompt change didn’t quietly make it worse?
Start with a golden set, a few hundred real questions paired with correct answers and the source passages that support them. Score retrieval and generation separately, because they fail separately. The common frame is the RAG triad: context relevance (did retrieval pull the passages that actually contain the answer), faithfulness (does the generated answer stay grounded in those passages instead of inventing), and answer relevance (does the answer address what was asked). Libraries like Ragas compute versions of these, but the point in the room is that you know what each one catches. Low faithfulness with high context relevance means retrieval works and the model is confabulating. Low context relevance means your problem is upstream, in chunking or embeddings, and no prompt tweak will save it.
Expect a follow-up on LLM-as-judge, since grading free-text answers by hand doesn’t scale. Say you use it, then say where it lies. Judges favor longer answers, favor the first option shown, and favor text from their own model family. Swap the judge model and your scores shift even though the system didn’t change. So you calibrate the judge against human labels on a sample, and you hold the judge fixed across comparisons. Treat a prompt edit as a deploy that needs a regression run, not a quick fix you eyeball once and push.
“What if it’s confidently wrong?”
Every grounding conversation ends here. A wrong answer delivered with total confidence is worse than no answer, and interviewers want to hear you say so. The tools are concrete. Cite the source passage behind every claim, so a user and your evaluation harness can both check it. Let the system abstain and say it doesn’t know when retrieval confidence falls below a threshold. Route low-confidence responses to a human instead of guessing. For high-stakes domains you can add a verification pass that checks each generated claim against the retrieved text before the answer goes out. It costs an extra model call. In a medical, legal, or financial setting, that is cheap insurance.
When they bring up agents
Late in the loop someone asks how you’d handle a task that takes several steps, and they are listening for whether you’ll over-build. The strong answer often argues against a full agent. A single well-constructed retrieval call is cheaper and far easier to evaluate than a ReAct-style loop that can wander, retry, and rack up ten model calls per request. Reach for an orchestration framework like LangGraph when the task genuinely branches and has to hold state across steps. For “look something up and answer,” an agent is a liability. Say that, and explain that every extra step multiplies both the cost and the surface area your evaluation has to cover, since failures compound across a chain.
The people who get GenAI offers in 2026 tend to volunteer two things before anyone asks: how they’d measure the system, and what it costs to run. They pick pgvector before Pinecone and build a golden set before they build a demo. The ones who lead with framework names and a diagram full of boxes, then go quiet when asked how they would know any of it works, are the ones who stall at the evaluation round.
