Job titles like “AI product engineer” or “applied AI engineer” did not exist in 2022. By 2026 they are among the most-posted senior engineering roles in tech. The job sits at the intersection of product engineering, prompt design, and rough ML literacy — building the AI features inside consumer and SaaS products. The interview is unique because the role is.
What the role actually is
- Build AI features inside a product (chat, summary, search, autocomplete, agents)
- Design prompts and prompt-engineering pipelines
- Choose models, providers, and orchestration patterns
- Build evaluation harnesses for the features they ship
- Own latency, cost, and quality of the AI surface
- Partner with product, design, and ML
It is not a research role and not an infrastructure role. It is shipping-features-with-LLMs.
The companies hiring
- SaaS companies adding AI: Notion, Linear, Figma, Atlassian, GitHub, Stripe
- Consumer apps: Duolingo, Quizlet, Khan Academy, Headspace
- Vertical AI startups: Harvey (legal), Glean (workplace search), Hippocratic (health), Cursor (dev tools)
- AI labs’ applied teams: OpenAI applied, Anthropic applied, Google AI applied
The interview process
- Recruiter screen — standard plus AI-feature questions
- Technical phone: coding (medium DSA, often with API or pipeline framing) plus a prompt-design conversation
- Onsite virtual:
- 1–2 coding rounds (medium DSA)
- 1 AI feature design (the unique round)
- 1 system design (often LLM-flavored)
- 1 craft deep-dive (your past AI work)
- 1 behavioral
The AI feature design round
Common prompts:
- “Design an AI summarization feature for our notes app”
- “Design a chat assistant for a customer support tool”
- “Design AI autocomplete for a code editor”
- “Design an AI-powered search over user documents”
What interviewers reward:
- Clarifying questions about user need first, model choice second
- Discussing the prompt design alongside the system design
- Identifying failure modes (hallucination, latency, cost) and mitigations
- Choosing the right model tier for the task (cost-quality trade-off)
- Building in evaluation from day one, not as an afterthought
- Privacy and PII handling for the data you send to the LLM
- Streaming UX for long generations
- Caching strategy (prompt cache, response cache)
The system design round
LLM-flavored variants of classic prompts:
- “Design a RAG system for our internal docs”
- “Design an agent that can take actions in the product”
- “Design a system for fine-tuning per-customer models”
Bring up: vector store, chunking, embedding choice, reranking, evaluation, observability, fallbacks, and rate-limit handling.
The craft deep-dive
Be ready to walk through a real AI feature you shipped, with:
- The user problem and why AI was the right tool (not always)
- The model and prompt design tradeoffs
- How you measured quality and what your evals look like
- What failure modes shipped and how you addressed them
- Your cost and latency profile
- How the feature performed in production
What to prepare
- Be fluent in 2–3 model providers (Anthropic, OpenAI, Google) and their tradeoffs
- Know the standard orchestration patterns (LangChain, LlamaIndex, raw API)
- Know how to write evals from first principles
- Have at least one shippable AI feature on a side project to walk through
- Be able to estimate cost per request given a token count
- Be able to estimate latency given model size, output length, and streaming
Compensation
AI product engineers are paid in the senior-SDE band at most companies, with a 10–20% premium over similarly-tenured generalists at AI-shipping companies. At AI-first companies (Cursor, Linear, Notion AI team) the premium can be larger. AI lab applied teams pay in their senior-engineer band.
How to break in
- Ship a non-trivial AI feature on a side project; document the design and evals publicly
- Contribute to LangChain, LlamaIndex, or a similar OSS framework
- Read the Anthropic and OpenAI cookbooks until they feel obvious
- Apply to companies whose AI features you have actually used; specificity in cover letters works here
Frequently Asked Questions
Do I need ML experience?
No formal ML training is required, but you should be able to read papers and have rough intuition about model size, fine-tuning vs prompting, and evaluation methodology.
How is this different from “ML engineer”?
ML engineer trains and serves models. AI product engineer uses models that already exist to build product features. Different skill set; sometimes the same person, often not.
Is this role going away as AI products mature?
The opposite — AI feature surface is expanding rapidly. The role is becoming standard at most product companies, and is unlikely to consolidate back into generalist SDE for at least a few years.