“Design an agent that can plan and book a trip.” That’s the prompt, and most candidates answer by naming frameworks. LangGraph, CrewAI, AutoGen, maybe the OpenAI Agents SDK. The interviewer stops listening around the second name. What they actually want is the loop: how a language model decides to call a tool, what it does with the result, and when it stops. Get that part right and the rest of the hour tends to follow.
This prompt shows up now in almost every AI-focused loop at Anthropic, OpenAI, and the wave of Series A companies building on top of them, and increasingly in plain infra interviews too. It looks open-ended, but it has a spine. The interviewer is checking whether you treat the model as a reliable planner (wrong) or as a fast, occasionally-lying component you have to design around (right).
The loop is the actual system
Start by drawing the single-agent tool-use loop, because a multi-agent answer that skips this reads as hand-waving. The model gets a task and a set of tool definitions. Each tool is a name, a description, and a JSON schema for its arguments. The model emits a tool call, your runtime executes it, and you append the result back into the conversation as an observation. Then you call the model again. That repeats until the model returns a final answer or you hit a stop condition. This is the ReAct pattern, and in 2026 the tool interface is usually spoken through the Model Context Protocol so the same tool server works across different agents.
Write the loop out concretely so the interviewer can see the state:
state = [system_prompt, user_task]
for step in range(MAX_STEPS):
resp = model.generate(state, tools=TOOLS)
if resp.is_final:
return resp.text
result = execute(resp.tool_call) # may raise, time out, or return junk
state.append(resp.tool_call)
state.append(observation(result))
raise StepBudgetExceeded()The two lines that matter most are the ones people skip: MAX_STEPS and the comment on execute. A candidate who names the step budget and the failure surface of a tool call is already ahead of one who drew a clean happy path.
Where the happy path falls apart
Now the interviewer starts poking, and this is where the interview is won. Push on failure yourself before they do.
Loops that never end are the first problem. The model calls the same search tool with the same query four times because the observation didn’t help. A step budget catches the runaway, but a better answer detects a repeated tool call with identical arguments and either reprompts with “that returned nothing useful, try a different approach” or bails. Say out loud that agents get stuck in ruts and you have to design an escape.
Context growth is the quieter killer. Every observation gets appended, so a run with 40 tool calls can push past 100k tokens, and cost and latency climb with it while quality drops. Two fixes worth naming: compact the history by summarizing older steps once the window gets large, and store big tool outputs (a scraped page, a query result set) outside the context, passing a short reference the model can re-fetch on demand. That second pattern also keeps the model from drowning in text it doesn’t need this step.
Tool failures are constant in production and interviewers know it. An API returns a 500, a search times out, a payment endpoint rejects the request. The agent has to see the real error in the observation and decide: retry with backoff, try a different tool, or stop and report. The wrong behavior, which weak agents exhibit, is to treat the error as if the action succeeded and march on. Feeding structured error observations back to the model, rather than silently swallowing them, is a specific thing you can point to.
Then there are malformed tool calls. The model invents a tool that doesn’t exist, or passes a string where the schema wants an integer. Validate every tool call against its schema before executing, and on a mismatch return a validation error as the observation instead of running anything. The model corrects itself surprisingly often when you hand it the exact error.
Memory, and what actually needs to persist
Working memory is just the current context window: the task and the running trace of tool calls. It dies when the session ends, which is fine for a one-shot task and useless for an assistant that should remember you hate red-eye flights. That’s the opening to talk about long-term memory, and interviewers like to see you separate the kinds instead of saying “we’ll use a vector database” and stopping.
| Memory type | What it holds | Typical storage | When the agent writes and reads it |
|---|---|---|---|
| Working (short-term) | Current task, recent tool calls and observations | The context window itself | Read and written every step; discarded when the session ends |
| Episodic | Records of past runs and their outcomes | Document or vector store keyed by user or task | Written after a task completes; retrieved to reuse or avoid a prior approach |
| Semantic | Durable facts: user preferences, account details, learned constraints | Vector store plus a structured DB for exact lookups | Written when a stable fact appears; retrieved by embedding similarity at task start |
| Procedural | Reusable skills or tool-call sequences that worked before | Prompt library or a tool the agent can call | Curated offline or promoted from successful episodes; loaded into the prompt |
The tradeoff to name: retrieval is not free and it is not always helpful. Pull too many memories into the window and you dilute the signal for the current step, which pushes accuracy down and cost up. A reasonable default is embed the task, fetch a small top-k, and let a cheap relevance check drop anything marginal before it hits the prompt.
When a second agent earns its keep
The follow-up is almost guaranteed: “Would you use multiple agents?” The strong answer is usually “not yet.” One agent with a well-scoped tool set is easier to build, cheaper to run, and far easier to debug. You reach for more agents when a single one’s tool list and context get unwieldy, when different subtasks need genuinely different tools or permissions, or when you want isolation so a flaky research step can’t corrupt the writer’s context.
The common shape is orchestrator-and-workers. A planner agent decomposes the task and delegates to specialists, each of which runs its own ReAct loop over a narrow tool set and returns a result. Plan-and-execute sits at the top, ReAct within each worker. For deeper problems you nest this into a hierarchy of orchestrators, though the coordination cost compounds fast and failures start to cascade across boundaries. What you must say in the same breath is that cost: every hop between agents is more tokens, more latency, and a new place for things to break, and debugging a five-agent trace at 2am is genuinely painful. Interviewers respect a candidate who won’t spawn agents for the fun of it.
State so a crash isn’t fatal
Agentic tasks can run for minutes or hours, and processes die. Design the orchestrator to be stateless and keep the state in an external store, Postgres or Redis, so any orchestrator instance can load a run’s state and resume from the last completed step. This is the same instinct as any long-running distributed job, and reaching for it here signals you’ve built real systems.
Idempotency deserves a sentence of its own. If the agent already called the booking API and then the process restarted, you cannot let the retry book a second flight. Give write tools idempotency keys and record which side effects have fired, so a resumed run picks up without repeating irreversible actions.
Guardrails around the tools that bite
Separate read tools from write tools early. A search or a lookup is safe to call freely. Sending an email, charging a card, or deleting a record is not, and those belong behind a gateway that enforces permissions and rate limits, and often a human confirmation step before anything irreversible executes. Framing the tool layer as a permissioned boundary rather than an open API, and putting the risky calls behind approval, is exactly the kind of judgment the question is fishing for.
How you’d know it works
If time remains, get ahead of evaluation, because most candidates forget it. Judging an agent on its final answer alone misses the point; a right answer reached by a broken path won’t generalize. Build a test harness of recorded tasks and score the whole trajectory: did it pick the right tools, in a sane order, without burning the step budget, and at what cost and latency per task. For open-ended outputs, an LLM-as-judge with a rubric gets you a signal, but keep a set of tasks with checkable ground truth so you aren’t grading vibes.
A few of the ways this actually gets asked:
- “Design a coding agent that can read a repo, run tests, and open a pull request.”
- “Your agent keeps calling the same tool in a loop. Walk me through why and how you’d stop it.”
- “When would you split this into multiple agents, and when is that a mistake?”
- “How do you keep a long-running agent from double-charging a customer after a restart?”
The candidates who do well don’t have a favorite framework. They treat the model as an unreliable component and build the scaffolding that makes it safe to be wrong, the same way you’d design around a network that drops packets. Show that instinct and the framework names take care of themselves.
Keep sharpening your system design:
