Top 30 LLM and LangChain Interview Questions and Answers [2026]

TL;DR: LLM engineer interviews in 2026 test whether a candidate can make retrieval measurable and cheap: chunking, hybrid search with reranking, retrieval metrics, judge bias and KV cache math. On the framework side, expect questions on LangChain 1.x (create_agent, middleware, ToolStrategy and ProviderStrategy) and LangGraph checkpointers, plus what moved to langchain-classic.

LangChain 1.0 and LangGraph 1.0 shipped in October 2025, and the release moved legacy chains, retrievers and the hub module into a separate langchain-classic package.

A year later, langchain 1.4.6 (October 14, 2026) is current, and code written against 0.3 tutorials often no longer imports. The questions below cover how models behave, how RAG fails and how the framework works today.

For whiteboard system design, use our ML system design interview guide.

How LLMs Work

1. What is a token, and why should an engineer care about tokenization?

A token is the unit the model reads and writes. It can be a word, part of a word, a punctuation mark or a byte sequence, and each model's tokenizer splits text its own way.

Every limit and every bill is counted in tokens, not characters or words.

Three practical consequences a candidate should name:

• Cost and context are per model. The same text produces different token counts under different tokenizers, so a prompt that fits one model's window can overflow another's. Count with the target model's tokenizer, or read usage_metadata on the response, which LangChain chat models expose with input_tokens, output_tokens and total_tokens.
• Non-English text and code cost more. Tokenizers trained mostly on English split other scripts and dense code into more pieces, which raises cost and shortens the effective window.
• Odd failures come from token boundaries. Counting letters, reversing strings and exact character limits are hard because the model never sees characters directly.

2. A model has a long context window. Why not put every document in the prompt?

Because accuracy, latency and cost all get worse as the prompt grows, even when it fits. The "Lost in the Middle" study found that models often do best when the relevant passage sits at the start or end of the input.

Accuracy drops when it sits in the middle of a long context.

Long prompts also cost more per call and take longer to prefill, so time to first token rises. The senior answer treats the window as a budget. Retrieve a small, ranked set of passages and put the best ones at the edges.

Keep long, stable instructions at the front so the provider can cache them (question 23).

Long context earns its place when the task needs the whole document, such as summarizing one contract. It does not replace retrieval over thousands of files.

3. How do temperature and top-p change the output, and can you make a model deterministic?

Both change how the next token is sampled from the model's probability distribution. Temperature rescales the distribution: low values sharpen it toward the most likely token, high values flatten it.

Top-p, or nucleus sampling from the Holtzman et al. paper, keeps only the smallest set of tokens whose probabilities add up to p and samples from that set.

Temperature 0 makes decoding close to greedy, but it does not guarantee identical outputs across calls. Server-side batching changes floating-point order, and providers update model versions behind the same name.

Good candidates design for this. They pin model versions, validate outputs against a schema, and rerun evals when a score sits near a threshold.

Extraction and classification get low temperature; brainstorming gets a higher one plus a filter afterward.

4. What is the KV cache, and how much memory does it need?

The KV cache stores the attention keys and values of every token already processed, so each new token attends to them without recomputing the whole prefix.

It turns generation from quadratic recomputation into one step per token, and it is usually what limits how many requests a GPU can serve at once.

The size per token is 2 (keys and values) times layers times KV heads times head dimension times bytes per value. Using the published Qwen3-8B config (36 layers, 8 KV heads, head dimension 128, bfloat16):

per_token = 2 * 36 * 8 * 128 * 2      # 147,456 bytes = 144 KiB
per_32k   = per_token * 32_768         # about 4.5 GiB for one request

The model has 32 attention heads but only 8 KV heads. That is grouped-query attention, and it makes the cache 4 times smaller than full multi-head attention would. A strong candidate also separates the two phases.

Prefill processes the whole prompt in parallel and sets time to first token. Decode produces one token per step and is limited by memory bandwidth. A long prompt hurts the first; a long answer hurts the second.

Prompting and Output

5. How do you structure a production prompt?

Stable instructions first, variable data last, and untrusted text clearly fenced off. A typical order is role and task, rules and output format, a few worked examples, then retrieved context and the user's question.

• Put the stable part first so provider prompt caching and vLLM prefix caching can reuse it across requests.
• Delimit data. Wrap retrieved chunks and user files in labeled tags and tell the model that text inside them is data, not instructions. It lowers the risk from injected text but does not remove it (question 25).
• Prefer examples to adjectives. Two short input and output pairs fix format drift better than "be concise and precise."
• Version prompts like code, with the eval set they were tuned on, so a change can be measured rather than argued about.

Prompt specialists go deeper on this; see the prompt engineer role overview for how the job differs from LLM engineering.

6. How do you get reliable structured output from a chat model in LangChain?

Call with_structured_output() on the chat model with a Pydantic model, TypedDict or JSON Schema.

Per the models docs, the method argument chooses how. "json_schema" uses the provider's native structured output, "function_calling" forces a tool call that follows the schema, and "json_mode" needs the schema described in the prompt.

from pydantic import BaseModel, Field
from langchain.chat_models import init_chat_model

class Invoice(BaseModel):
    vendor: str
    total_usd: float = Field(description="Grand total in US dollars")
    due_date: str | None = None

model = init_chat_model("gpt-5.5")
extractor = model.with_structured_output(Invoice, include_raw=True)
result = extractor.invoke("Extract the invoice fields: ...")
# result["parsed"] is an Invoice or None; result["raw"] is the AIMessage

Two details separate experienced answers. include_raw=True returns the raw message alongside the parsed object, so a parse failure can be logged and retried instead of raising mid-pipeline.

And the docs warn that structured output does not support a model with tools already bound by bind_tools. Tools plus a final schema need the agent pattern in the next question.

7. In create_agent, what is the difference between ToolStrategy and ProviderStrategy?

Both go in response_format. ToolStrategy produces the structured answer through an artificial tool call, so it works with any model that can call tools.

ProviderStrategy uses the provider's native structured output, where the provider enforces the schema. Both come from langchain.agents.structured_output, and the result lands in result["structured_response"].

If you pass a bare schema type, the structured output docs say create_agent picks ProviderStrategy when the model supports native output and ToolStrategy otherwise. Since langchain 1.1, it reads that support from the model's profile data.

A JSON Schema dict is never auto-detected and must be wrapped explicitly.

from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy

agent = create_agent(
    model="gpt-5.5",
    tools=[search_orders],
    response_format=ToolStrategy(Invoice, handle_errors=True),
)
result = agent.invoke({"messages": [{"role": "user", "content": "Summarize order 4411"}]})
invoice = result["structured_response"]

handle_errors on ToolStrategy controls retries. The default, True, sends a standard error back to the model, a string sends a fixed message, an exception type retries only for that error, and False raises.

ProviderStrategy has a strict flag, added in 1.2, for providers that support strict schema adherence.

RAG End to End

8. How do you choose a chunking strategy?

Chunk along the document's own structure, size chunks so one chunk answers one kind of question, and keep enough metadata to cite and filter. There is no universal size; it is a parameter you tune against retrieval metrics (question 13).

LangChain's knowledge base tutorial uses RecursiveCharacterTextSplitter from langchain_text_splitters, which tries paragraph breaks first, then lines, then words:

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=200, add_start_index=True
)
chunks = splitter.split_documents(docs)   # each chunk keeps metadata["start_index"]

• Structured sources (Markdown, HTML, code) split better on headings or syntax than on character counts. LlamaIndex ships MarkdownNodeParser and CodeSplitter for this.
• Small chunks retrieve precisely but lose context; large chunks carry context but dilute the embedding. A common fix is to retrieve small and return the parent: LlamaIndex's HierarchicalNodeParser and SentenceWindowNodeParser do this.
• Tables and lists should never be cut in the middle; extract them as their own chunks with a heading attached.

9. How do you pick an embedding model, and what happens when you change it?

Pick on your own retrieval eval, not a public leaderboard, then weigh dimensions, cost, latency and language coverage. A leaderboard measures someone else's queries over someone else's documents.

Changing the model means re-embedding the whole corpus. Vectors from two embedding models live in different spaces, so a new query vector cannot be compared with old document vectors.

Strong candidates plan for this from day one. They store the model name and version with each vector, build the new index next to the old one, run the eval on both, then switch reads.

They also know that some models expect different prefixes or task types for queries and for passages.

10. What is hybrid search, and how do you merge the two result lists?

Hybrid search runs a lexical search such as BM25 and a vector search on the same query, then merges the results. BM25 catches exact terms that embeddings blur: product codes, error strings, names and acronyms.

Vector search catches paraphrases that share no words with the query.

The standard merge is reciprocal rank fusion. The Cormack, Clarke and Buettcher paper (SIGIR 2009) scores each document as the sum of 1 / (k + rank) across the lists.

The authors fixed k = 60 in pilot runs and found the exact choice was not critical:

def rrf(result_lists, k=60):
    scores = {}
    for results in result_lists:
        for rank, doc_id in enumerate(results, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

The point to make is that RRF uses ranks, not scores. BM25 scores and cosine similarities sit on different scales, and adding them directly lets one system drown out the other.

In LangChain, EnsembleRetriever now lives in langchain-classic and applies a weighted rank fusion over several retrievers.

11. Why add a reranker, and where does it sit?

A reranker reads the query and each candidate chunk together and scores how well the chunk answers the query. That is more accurate than comparing two independent embeddings, but too slow to run over a whole corpus.

So it runs second: retrieve a wide candidate set cheaply, rerank it, and pass only the top handful to the model.

The trade-off to discuss is latency. A cross-encoder over 50 to 100 candidates adds a model call to every request.

Teams cap the candidate count, run the reranker on a GPU next to the retriever, and check that recall at the final cut improves. If the generator only ever sees 5 chunks, the reranker's job is to make those 5 right.

12. A chunk retrieves well in isolation but loses meaning out of context. What do you do?

Add the missing context to the chunk before indexing it. A chunk that says "Revenue grew 3% over the previous quarter" is useless if it does not say which company and which quarter.

Anthropic's contextual retrieval write-up (September 19, 2024) prepends a short, model-written note on where each chunk sits in its document. The note is usually 50 to 100 tokens and goes in before embedding and before BM25 indexing.

On their tests, the top-20 retrieval failure rate fell from 5.7% to 3.7% with contextual embeddings alone. Adding contextual BM25 took it to 2.9%, and reranking on top to 1.9%.

Cheaper variants exist: prepend the document title and section heading path to every chunk, or store them as metadata and filter on them.

13. How do you evaluate retrieval separately from generation?

Build a set of questions with the chunk IDs that answer them, then score the retriever alone with ranking metrics. If retrieval is wrong, no prompt will fix the answer, so this is measured first.

• Recall@k: did any correct chunk appear in the top k? This is the metric that matters when the generator sees k chunks.
• MRR and nDCG: how high the correct chunks rank, for when order matters.
• Context precision and context recall: the Ragas metric list uses these names for LLM-graded versions, alongside Faithfulness and Response Relevancy for the generation side.

Gold sets come from real queries in logs, labeled by domain experts, topped up with synthetic questions generated from chunks.

Synthetic questions tend to copy the chunk's wording, which flatters lexical search, so candidates should say they check the mix. The RAG design question in the ML system design guide covers the serving side of the same pipeline.

14. How do you make a RAG answer cite its sources reliably?

Give every chunk a stable ID in the prompt, require the model to cite those IDs, and verify the citations in code after generation. Citations the application never checks are decoration.

context = "\n".join(
    f'<doc id="{i}" source="{d.metadata["source"]}">{d.page_content}</doc>'
    for i, d in enumerate(chunks, start=1)
)
# Ask for {"answer": str, "citations": [int]} via structured output,
# then reject any citation id that is not in range and flag answers with none.

With add_start_index=True each chunk keeps its character offset, so the UI can link a citation to the exact passage. Stronger answers add a faithfulness check.

A cheap judge or entailment model confirms that each cited chunk supports its sentence, and the app drops or flags the rest.

15. A user reports a wrong RAG answer. How do you debug it?

Find the failing stage in the trace, in pipeline order. Check indexing, then retrieval, then reranking and truncation, then generation.

1. Indexing: search the index for the expected document directly. Parsing errors, stale data and access filters cause many misses.
2. Retrieval: run the user's query and check the candidate list. A miss here points to chunking, the embedding model or a missing BM25 leg.
3. Ranking and packing: the right chunk was retrieved but cut by the reranker or the token budget.
4. Generation: the right chunk reached the prompt and the model ignored or misread it. That is a prompt, model or context-order issue.

Then add the query to the eval set so the fix stays fixed. Engineers who do this daily are often titled RAG engineers; the RAG engineer role overview lists the wider scope.

LangChain and LangGraph

16. What is a Runnable, and is LCEL still relevant in LangChain 1.x?

Yes. A Runnable is the interface in langchain-core that models, prompts, retrievers and parsers share: invoke, batch and stream, each with an async version.

The Runnable reference says they compose with the | operator into a RunnableSequence, and a dict inside a sequence becomes a RunnableParallel. That composition style is what LangChain Expression Language (LCEL) refers to.

from langchain_core.runnables import RunnablePassthrough

rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | model
)
answers = rag_chain.batch(questions, config={"max_concurrency": 5})
safe_chain = rag_chain.with_fallbacks([backup_chain])

Version 1.0 changed two things around it: the default entry point for agents (create_agent) and the home of legacy chains such as LLMChain, which moved to langchain-classic.

A fixed pipeline with no loop is still a good fit for a Runnable chain; anything with loops, branches or pauses belongs in LangGraph.

17. What does create_agent give you, and what is middleware for?

create_agent from langchain.agents runs the standard tool-calling loop on top of LangGraph, so it gets checkpointing, streaming and human-in-the-loop pauses without hand-built graphs.

It replaced langgraph.prebuilt.create_react_agent, and the prompt argument became system_prompt.

Middleware is how you change that loop without forking it. Per the LangChain 1.0 release notes, a subclass of AgentMiddleware can implement before_agent, before_model, wrap_model_call, wrap_tool_call, after_model and after_agent.

The built-in list covers common needs:

from langchain.agents import create_agent
from langchain.agents.middleware import (
    ModelCallLimitMiddleware, ModelFallbackMiddleware, PIIMiddleware,
)

agent = create_agent(
    model="gpt-5.5",
    tools=[lookup_policy],
    system_prompt="Answer from the policy tool only.",
    middleware=[
        PIIMiddleware("email", strategy="redact"),
        ModelFallbackMiddleware("claude-sonnet-4-6"),
        ModelCallLimitMiddleware(run_limit=8),
    ],
)

A good answer explains why a call limit belongs in middleware: it caps cost per run no matter what the model decides.

Tool-calling loop design in depth belongs to an agent engineering interview; here the test is whether the candidate knows where the framework's extension points are.

18. How do LangGraph checkpointers, threads and interrupts work together?

A checkpointer saves the graph state at every super-step, keyed by the thread_id in the config. That saved state is what makes memory across turns, resuming after a crash, time travel and human approval possible.

• Backends: InMemorySaver for tests, SqliteSaver for local work, and PostgresSaver or MongoDBSaver in production, per the checkpointer docs. PostgresSaver needs setup() once to create its tables.
• Durability: the durability argument trades safety for speed. "exit" saves only when the run ends or pauses, "async" saves while the next step runs, and "sync" saves before the next step starts.
• Inspection: get_state returns a StateSnapshot; get_state_history lists them newest first; update_state writes a new checkpoint rather than editing the old one.

from langgraph.types import interrupt, Command
from langgraph.checkpoint.memory import InMemorySaver

def approve_refund(state):
    decision = interrupt({"refund": state["amount"]})   # pauses here
    return {"approved": decision}

graph = builder.compile(checkpointer=InMemorySaver())
config = {"configurable": {"thread_id": "ticket-981"}}
graph.invoke({"amount": 120}, config)
graph.invoke(Command(resume=True), config)             # same thread_id

The trap: per the interrupts docs, the node runs again from its beginning on resume. Any insert, email or payment placed before interrupt() happens twice unless it is idempotent, so side effects go after the interrupt or use upserts.

19. When would you pick LlamaIndex over LangChain for a RAG project?

Pick it when the project is mainly document question answering and you want the indexing pieces ready-made.

LlamaIndex centers on indexes and query engines; LangChain centers on composable components and, through LangGraph, on stateful control flow.

The two are not exclusive. The LlamaIndex starter example wraps a query engine as a tool, and the same pattern lets a LlamaIndex retriever sit inside a LangGraph flow. The weak answer picks a side on reputation.

The strong one names the parts of the pipeline each would own and how the team would test them.

Evaluation and Cost

20. How do you build an offline evaluation set for an LLM feature?

Start from real inputs, label the expected behavior, and grade each output with the cheapest evaluator that is reliable for that check. LangSmith's evaluation concepts use a shared vocabulary.

A dataset holds examples, an experiment is one application version run on it, and evaluators are human, code, LLM-as-judge or pairwise.

• Code evaluators first: schema validity, exact match on a label, citation IDs in range. They are free and deterministic.
• LLM-as-judge for the rest: tone, faithfulness, completeness, with a rubric and few-shot graded examples.
• Slices: tag examples by intent, language and difficulty so a regression on one slice does not hide inside an average.
• Online checks: production traces have no reference answers, so they get reference-free evaluators and sampling for human review.

Earlier rounds can use the AI skills evaluation questions to screen general ML judgment before this level of detail.

21. What biases does an LLM judge have, and how do you control them?

The MT-Bench and Chatbot Arena paper names three. Position bias favors the first or second answer. Verbosity bias favors longer answers. Self-enhancement bias favors answers from the judge's own model family.

It also reports that strong judges reached over 80% agreement with human preferences, about the level humans reach with each other.

Controls a senior candidate should list:

• Run pairwise comparisons twice with the order swapped, and count a win only if it holds both ways.
• Score against a rubric with specific criteria rather than asking "which is better."
• Use a judge from a different model family than the system under test.
• Calibrate: have humans grade a sample, measure agreement with the judge, and re-check when the judge model or prompt changes.

22. Prompting, RAG or fine-tuning: how do you decide?

Diagnose the gap with an eval set, then pick the cheapest fix for that kind of gap. Missing or changing knowledge calls for retrieval. Consistent format, tone or a narrow skill that prompting cannot hold calls for fine-tuning.

Many problems need neither.

Fine-tuning teaches behavior, not facts that change weekly, and it needs labeled pairs plus a retraining plan whenever the base model changes.

Parameter-efficient methods such as LoRA train small low-rank matrices instead of all the weights, which cuts training memory and lets one server host several adapters.

RAG keeps knowledge outside the model, so it can be updated, filtered by user permissions and cited. The two combine: a model fine-tuned to follow a citation format, fed by retrieval.

23. An LLM feature costs too much and responds too slowly. Where do you start?

Measure where tokens and time go per request, then cut the largest item. Input tokens usually dominate cost in RAG; output tokens usually dominate latency.

• Cache the prefix. LangChain's models docs list provider controls such as prompt_cache_key for ChatOpenAI and cache_control for Anthropic, and note caching often engages only above a minimum prompt length. Cache hits show up in usage metadata, so verify them.
• Route by difficulty. Send classification and extraction to a small model and reserve the large one for hard cases.
• Retrieve less. A reranker that lets you pass 5 chunks instead of 20 cuts input tokens by three quarters.
• Cap output with a max token setting and a terse format; stream so users see the first tokens early.
• Batch offline work with batch() and a max_concurrency limit, and use InMemoryRateLimiter to stay under provider rate limits.

Serving and Safety

24. You self-host a model on vLLM. What settings and behaviors should you understand?

vLLM serves an OpenAI-compatible API on top of PagedAttention, which stores the KV cache in fixed-size blocks so less memory is lost to fragmentation. The settings that matter most are about that cache.

• --gpu-memory-utilization (0.92 by default per the engine arguments page) sets how much GPU memory vLLM claims; what is left after weights becomes KV cache.
• --max-model-len caps prompt plus output. Lowering it is the quickest fix when the cache cannot fit a single maximum-length request.
• --kv-cache-dtype can store the cache in a smaller type to fit more concurrent requests, at some accuracy risk to test.
• Automatic prefix caching reuses cached blocks for shared prefixes. The feature docs state it only speeds up prefill, not decoding, so it helps long shared system prompts and multi-turn chat, not long answers.

A senior candidate also mentions isolation. Per the prefix caching design doc, a per-request cache_salt limits cache reuse to requests with the same salt. That matters when tenants should not share cached prefixes.

Operating this at scale is the LLMOps engineer's day job.

25. How do you defend a RAG system against prompt injection in retrieved content?

Assume any retrieved text may contain instructions, and limit what the model can do when it follows them. OWASP's LLM01 Prompt Injection entry calls this indirect injection: instructions arrive through websites or files the model processes.

It also states that RAG and fine-tuning do not fully mitigate it. The attack class was described early by Greshake et al.

• Least privilege: the model only gets tools and data the current user is allowed to use, enforced outside the model.
• Segregate external content: fence it with labels, and keep system instructions out of reach of retrieved text.
• Validate outputs: schemas, allowlisted URLs and no rendering of model-written links or images that could leak data.
• Human approval for high-risk actions; in LangChain, HumanInTheLoopMiddleware pauses on chosen tool calls.
• Red-team the corpus: plant injected documents in a test index and add them to the eval set.

The weak answer is "tell the model to ignore instructions in documents." That helps, but it is a mitigation inside the component under attack.

What Changed Recently

26. What broke when teams upgraded from LangChain 0.3 to 1.0?

Imports, agent construction and a few defaults. The v1 migration guide lists the changes a candidate who did the upgrade will remember:

• All LangChain packages require Python 3.10 or later.
• Legacy chains (LLMChain, ConversationChain), retrievers such as MultiQueryRetriever, the indexing API, the hub module and langchain-community re-exports moved to langchain-classic.
• create_react_agent became create_agent; pre-model and post-model hooks became before_model and after_model middleware; custom agent state must be a TypedDict extending AgentState.
• Prompted structured output, a (prompt, Schema) tuple in response_format, was removed.
• Message .text() became a property, and langchain-openai now stores Responses API items in message content by default.

LangGraph 1.0 shipped alongside with, per its release notes, no change to graph primitives; its main deprecation was create_react_agent.

27. What did LangChain 1.1 through 1.3 add that changes day-to-day code?

Model profiles, better structured output and a new event stream. From the official LangChain changelog:

• 1.1 (November 2025): chat models expose a .profile of capabilities such as max_input_tokens. SummarizationMiddleware can trigger from it, create_agent infers ProviderStrategy from it, and ModelRetryMiddleware adds retries with exponential backoff.
• 1.2 (December 2025): a strict option on ProviderStrategy and an extras attribute on tools for provider-specific parameters.
• 1.3 (May 2026): version="v3" for stream_events and astream_events on agents, per the 1.3.0 release.

The practical change: code no longer needs a hardcoded table of context sizes per model, because the profile carries it.

28. What changed in LangGraph 1.1 and 1.2?

Type-safe outputs in 1.1, and failure handling plus smaller checkpoints in 1.2.

Per the 1.1.0 release (March 2026), passing version="v2" makes stream() yield typed StreamPart dicts. invoke() then returns a GraphOutput with .value and .interrupts, instead of mixing interrupts into the output dict under "__interrupt__".

It is opt-in; v1 output is unchanged.

The changelog entry for 1.2.0 (May 2026) adds a per-node timeout= for async nodes in Python and a node-level error_handler= for recovery and routing.

It also adds graceful shutdown with RunControl.request_drain() and a beta DeltaChannel that stores incremental deltas to shrink checkpoints. Long-running graphs with large message lists benefit most from the last one.

29. What is new in LangChain 1.4?

MCP support moved into LangChain itself, and human approval became per call.

The changelog dates the 1.4.0 release to September 2026: a beta langchain.mcp namespace with MCPAdapter, built on FastMCP, replaces the separate langchain-mcp-adapters package.

1.4.6 (October 14, 2026) adds interrupt_mode="per_call" to HumanInTheLoopMiddleware, which pauses once per tool call and answers by interrupt ID. Batched mode, one pause per model turn, stays the default for now.

The changelog says LangChain plans to make per-call the default in the next major release. A candidate on a recent codebase should know which mode their approval UI assumes.

30. What changed in vLLM that breaks older serving code?

The old engine and the old structured-output fields are gone. v0.11.0 (October 2, 2025) completed the removal of the V0 engine, including AsyncLLMEngine and LLMEngine, leaving V1 as the only engine. v0.12.0 (December 3, 2025) removed the guided_* config fields.

Per the structured outputs docs, requests now pass a structured_outputs object with json, regex, choice or grammar, or use response_format with a JSON schema:

client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[{"role": "user", "content": "Classify: the refund never arrived"}],
    extra_body={"structured_outputs": {"choice": ["billing", "shipping", "other"]}},
)

The current release is v0.31.0 (October 5, 2026), which also tags prefix-cache extra keys by source so a LoRA name and a cache_salt cannot collide.

Releases arrive about every two weeks, so pin the version and read the breaking-changes list before each upgrade.

Signs of a Strong Answer

• They separate retrieval failures from generation failures and name a metric for each before proposing a fix.
• They work out KV cache memory from the model config and know that grouped-query attention is why the number is manageable.
• They fuse BM25 and vector results by rank, and can explain why adding raw scores goes wrong.
• They know which LangChain imports moved to langchain-classic and when create_agent picks ProviderStrategy.
• They keep side effects after interrupt() or make them idempotent, because the node replays on resume.
• They treat retrieved text as untrusted input and enforce permissions outside the model.

Hiring LLM Engineers

Engineers who have shipped RAG with real evals, and who know the LangChain 1.x and LangGraph APIs rather than 2023 tutorials, are scarce.

Second Talent helps US startups hire vetted AI-Native Engineers from Asia, including LLM developers and LangChain developers, at $29 to $39 USD an hour billed monthly. See the cost to hire an LLM developer for rates by seniority.

Candidates are matched in 24 hours, with a 90-day one-time replacement guarantee. Tell us the role and use these questions in your technical round.