3. LLMs, Prompting and RAG (Questions 9-12)
This is the highest-weighted category in 2026 because almost every AI role now touches LLMs. Test for practical experience, not vibes. The Hugging Face State of Open Source AI 2025 shows 60% of new AI projects in production are LLM-based.
Q9. Walk me through a RAG pipeline you built. Where did it break and how did you fix it?
What good looks like: They name the chunking strategy and why. They describe the embedding model and why they chose it (cost, latency, recall on their domain). They name the vector store.
They describe a real failure: bad chunk boundaries cutting off code blocks, retrieval missing rare terms, the LLM ignoring retrieved context.
They can quote latency (P50 and P95) and an actual quality metric (e.g. RAGAS scores or human eval).
Red flags: Generic answer that could come from a tutorial. Cannot name a chunking decision. No latency numbers.
Follow-up: “What would you change if your corpus was 10x larger?”
Q10. When would you fine-tune instead of using few-shot prompting?
What good looks like: Default to prompting.
Fine-tune when (1) the task has a strict output format that few-shot keeps breaking, (2) latency or cost matters and you can run a smaller fine-tuned model, (3) the domain is so specific that no foundation model has seen it.
They mention LoRA or QLoRA as the cheap path. They know fine-tuning rarely teaches new facts (that is what RAG is for).
Red flags: “Always fine-tune for best results.” Not knowing fine-tuning does not add knowledge.
Follow-up: “How do you decide between LoRA and full fine-tuning?”
Q11. Your LLM hallucinates 8% of the time on your benchmark. How do you cut it to under 2%?
What good looks like: Layered approach. Better prompting (chain of thought, structured output, “say I don’t know if unsure”). Retrieval augmentation if facts are the issue. Constrained decoding (function calling, JSON mode, regex).
Output validation with a second LLM call. They mention the tradeoff. Every layer adds latency and cost.
Red flags: “Just use GPT-5.” Treating hallucination as a model bug rather than a system property.
Follow-up: “How do you measure hallucination rate without humans-in-the-loop?”
Q12. Design a simple agent that books meetings. What are the failure modes?
What good looks like: They name the loop (perceive, plan, act, verify). They mention tool use for calendar and email APIs. They describe at least 3 failure modes.
Wrong attendee disambiguation, infinite loops on tool errors, prompt injection from email content, hallucinated meeting times, double-booking.
They mention guardrails: max iterations, confirmation steps for destructive actions, output validation.
Red flags: “Just give it tools and a goal.” No mention of guardrails or failure modes.
Follow-up: “How would you test this in CI without spamming a real calendar?”