Seventeen questions from the 2026 AI engineer loop, in seven clusters: RAG, agents, evals, production and inference, the fine-tuning decision, prompt engineering, and LLM fundamentals. Each one comes with what it really tests, the follow-up probes you will get, and a framework for the answer.
Interviewing for an AI engineer role? Paste the posting and Calibrd predicts the questions for that exact role and level. Your first mock is free.
01
Why AI engineer interviews look different
The loop in 2026
AI engineers sit at #1 on LinkedIn's list of fastest-growing tech jobs for 2026, and the interview looks nothing like a classic software loop. There is less whiteboard coding and more conversation: interviewers want to hear how you think about systems that behave well in production. Retrieval that finds the right context. Agents that stop instead of looping. Evals that catch regressions before users do. Inference that stays cheap at scale.
The two designs that come up in most loops: inference at scale, and an agent or RAG system under a cost cap. Everything below is organized around the clusters interviewers actually probe: RAG, agents, evals, production, the fine-tuning decision, prompt engineering, and LLM fundamentals. Each question comes with what it really tests, the follow-ups you'll get, and a framework for answering.
02
The questions
The full bank, with what each one tests
RAG
01Documents change daily. How do you keep the index fresh without re-embedding everything?
What they're really asking
Incremental indexing, versioning, and invalidation in a production pipeline.
Follow-ups you'll get
How do you handle deleted documents?
A chunk's source changed; how do you know which answers are now stale?
Answer framework
Change detection at ingest (hashes, timestamps).
Versioned chunks with source lineage.
Tombstones for deletes.
Re-embed only what changed, and flag answers built on superseded chunks.
02Your RAG system answers confidently and wrong. Debug it out loud.
What they're really asking
Structured debugging, and whether you separate retrieval failure from generation failure.
Follow-ups you'll get
What do you log to tell the two apart?
The retrieved chunks look right but the answer is still wrong, now what?
How do you stop this regressing next month?
Answer framework
Check what was retrieved first: citations or it didn't happen.
If retrieval is right, the problem is grounding, so tighten the prompt, require quotes, lower the temperature.
If retrieval is wrong, fix chunking or the query, add query rewriting.
Add the failing case to a golden eval set so it can't come back.
03How do you keep p99 latency under two seconds on a RAG pipeline?
What they're really asking
Production trade-offs under a latency budget.
Follow-ups you'll get
What do you cut first when the budget is blown?
How do you stream partial results without misleading the user?
Answer framework
Budget the pipeline stage by stage (retrieval, rerank, generation).
Shrink the expensive stage first (smaller reranker, fewer chunks, cached embeddings).
Stream tokens early so perceived latency drops.
Precompute and cache for repeated queries.
Agents
01Design a research agent that reads the web and writes a report with sources.
What they're really asking
Planning decomposition, source trust, and cost bounding on an open-ended task.
Follow-ups you'll get
How do you stop it citing garbage sources?
The report costs $4 in tokens; how do you get it under $1?
How do you run five searches in parallel safely?
Answer framework
Decompose into search, read, and synthesize with a planner.
Source scoring with an allowlist; every claim needs a cited source.
Budget per report: max steps, model routing (cheap model for skimming, strong model for synthesis).
Parallel tool calls with a join step.
02How do you evaluate an agent?
What they're really asking
Whether you know task success is not a vibe, and how you measure multi-step behavior.
Follow-ups you'll get
What's wrong with grading agents with another LLM?
How do you test the failure paths, not just the happy path?
Answer framework
Task success rate in a sandbox with scripted tools.
Trajectory checks: steps taken, tool calls made, cost per task.
LLM-as-judge only with human calibration and known blind spots.
Red-team the tools: timeouts, bad data, adversarial users.
03Traces show your agent takes 40 tool calls for a task that should take 5. What do you do?
What they're really asking
Efficiency debugging from observability data.
Follow-ups you'll get
How do you tell a planning failure from a tool failure in the trace?
What guardrail would have caught this before the user noticed?
Answer framework
Read the trace: repeated calls mean the planner is stuck; failing calls mean a tool problem.
Fix the loop: better tool descriptions, stop conditions, max steps.
01Answer out loud, not in your head. Every question here is asked spoken; reading the framework is not the same as saying it.
02Take the probes seriously. The follow-ups are where interviews are won and lost. Practice the question, then the probe.
03Rotate clusters. Don't drill RAG five times in a row; real loops jump between clusters.
04Time the two designs. Inference at scale and the agent/RAG under a cost cap are whiteboard questions; practice them in 15 minutes with the trade-offs stated.
Mostly conversation, not coding. Expect scenario questions ("design a RAG system", "your agent loops in production, why") with follow-up probes on trade-offs, cost, latency, and evals. Some loops add a coding round, but the core signal is system judgment.
How is this different from a machine learning interview?
ML interviews test modeling fundamentals (training, regularization, metrics). AI engineer interviews test building with models: retrieval, agents, evals, inference, cost. Applied judgment over theory.
Do I need ML fundamentals, or is LLM experience enough?
For most applied AI engineer roles, shipped LLM work (RAG, agents, evals) matters more than theory. Research and training-focused roles are the exception and will say so in the posting.
What should I build before interviewing?
One RAG pipeline or one agent, with tracing and a small eval set. It gives you real answers to the debugging and eval questions, which are the ones candidates most often fake.
Should I use the resume review or the free job scan?
This page covers the general bank. The free scan reads your specific posting and returns the questions for that exact role and level, a pay benchmark matched to the role and, with your CV, a read on the gaps an interviewer will probe. No posting yet? The free resume review reads the CV on its own.
02Qarera: most in-demand skills 2026A resume-tool company's report on 360,336 postings, 27 Dec 2025 to 16 Jun 2026. AI 19.8%, Python 18.6%, SQL 11.7%, AWS 11.3%.
Loops change. This reflects what candidates and hiring teams report as of 2026; treat it as a map of the clusters, and read the posting for which one your loop weights.
Walk in ready
Walk into your AI engineer interview ready.
Paste your actual posting and Calibrd predicts what that company asks for this role, where your CV is thin, and what it should pay. Then rehearse the round out loud with honest feedback until you're confident. Free to start.