AI Agent Interview Deep Dive: 10 Critical Questions from Architecture to Evaluation

01 What Is an AI Agent? Core Difference from Ordinary LLM Applications

The fundamental difference lies in three keywords: autonomous decision-making, goal-driven, closed loop .

Ordinary LLM Application : Single-turn mapping (ask → answer); user decides every step; no external world interaction; ends after generation; typical examples: translation, summarization, QA.

AI Agent : Goal-driven loop (think → act → observe → think again); agent autonomously decides next step; calls tools, queries data, executes operations; termination when goal reached / limit hit / human stop; typical examples: ticket diagnosis, automated ops, deep research.

Example loop: user asks to check order 12345 logistics and notify customer if delayed. The agent thinks (need to query order), acts ( query_order), observes (shipped but delivery delayed), thinks (delay exists, send email), acts ( send_email), observes (sent), answers. This loop is forced by the autoregressive generation mechanism — the model generates one token at a time, cannot see the full plan upfront, so it must "think one step → act one step → get result → think next step".

Summary: The difference is not component count but whether there is autonomous decision-making and a closed loop. The Agent's loop is dictated by the autoregressive generation mechanism.

02 When to Use Workflow, Single Agent, or Multi-Agent?

Three concrete examples clarify the choice:

Refund progress query → Workflow: Fixed three steps (retrieve order/refund → template → LLM generates reply). Path is predetermined, no branching. Code orchestrates directly — controllable, cheap, stable.

Production ticket diagnosis → Single Agent: Next step depends on previous result (e.g., crash log shows OOM → check memory config; shows timeout → check gateway). Branches decided by intermediate results; path cannot be enumerated upfront, so decision authority goes to the model. "Single" means one decision center — one LLM drives the loop.

Code generation + review → Multi-Agent: One agent writes code, another reviews by running unit tests in a sandbox. The review brings new information (test failures) that the generator could not obtain by re-reading its own output. Division of labor only makes sense when new information enters.

Comparison:

Workflow : Predefined fixed steps; high controllability; low cost/latency; suitable for clear steps, fixed path (e.g., RAG pipeline, ticket routing).

Single Agent : LLM autonomously decides next step; medium controllability; medium cost; suitable for uncertain steps, flexible decisions (e.g., ticket diagnosis, deep research).

Multi-Agent : Multiple agents collaborate; low controllability; high cost; suitable when single agent lacks info/capability (e.g., generation + review separation, multi-perspective review).

Selection criteria: if path can be hardcoded → Workflow; if not → Single Agent; only split to Multi-Agent if collaboration yields new information (test results, screenshots, tool outputs). Three questions before splitting: (1) Is there new information? (2) Why not self-review? (Self-review suffers from confirmation bias.) (3) Is role-switching enough? (No — multiple agents critiquing same text equals burning tokens.) Multi-agent patterns: peer collaboration, manager mode (most business cases), decentralized (message bus + shared state, for large-scale parallelism).

Summary: Predefined steps → Workflow; need autonomous decisions → Single Agent; collaboration yields new information → Multi-Agent. Multi-agent debate without new information = burning money.

03 How to Choose Among ReAct, CoT, Plan-and-Execute, Self-Refine?

Paradigm comparison:

CoT : Pure reasoning (think → think → think → answer); no tools, no loop. Suitable: answer in context (math, reading comprehension, code review). Example: chicken-rabbit puzzle solved purely by reasoning.

ReAct : Reasoning + action alternating (think → act → observe → think); tools yes, loop yes. Suitable: exploratory tasks needing external real-time data (e.g., weather API). Cost: infinite loops, token bloat, one wrong step derails all.

Plan-and-Execute : List full plan first, then execute stepwise; tools optional, loop no. Suitable: decomposable tasks with relatively clear steps. Fixes ReAct's shortsightedness: planner creates full task list (query data, compare last quarter, analyze causes, produce charts, write summary), executor runs each, optional re-planner adjusts if data source unavailable.

Self-Refine : Generate → self-critique → revise → re-critique; tools optional, loop yes. Suitable: high-quality output required. Operates at review layer.

Production systems mix them: Plan-and-Execute decomposes, ReAct executes each subtask, Self-Refine polishes final output. Terminology trap: don't call CoT a "closed loop" — true closed loop is ReAct.

Summary: CoT handles "think", ReAct adds "act", Plan-and-Execute thinks globally first, Self-Refine reviews after generation. Production mixes: Plan decomposes, ReAct executes, Refine gates quality.

04 Complete Function Calling Flow

Iron rule: Model decides, never executes. Six-step flow:

Define tool functions (your code, not LLM).

Pass tool descriptions (JSON Schema) to LLM.

Send user query + tool specs to LLM.

LLM returns a JSON: {"name": "query_order", "arguments": {"order_id": "123"}} — semantic match between user intent and tool description.

Your code parses JSON, calls the actual function.

Return result to LLM (as tool role message) for final answer composition.

Distinction from code generation: Function Calling outputs call intent (JSON), executes pre-registered functions; code generation outputs runnable code text, sandbox executes it. Bonus: Pydantic validation — one definition serves two purposes: auto-generate JSON Schema for LLM, and validate returned arguments before invocation. Validation occurs after LLM returns parameters, not before. Two gates: Pydantic checks format (types, required, enums); business logic (order exists, amount non-negative) checked inside function.

Summary: Model selects tool via semantic matching between tool description and user intent. Model is "decision", never "execution" — execution is always your code.

05 What Is MCP? Relationship with Function Calling?

MCP (Model Context Protocol) is an open protocol by Anthropic standardizing how models connect to external tools and data sources. Analogy: MCP is the USB-C of tools — previously each tool needed custom adapter; with MCP, tool exposes once, any MCP-compatible agent plugs in.

Comparison:

Function Calling : Model capability; solves how model expresses "I want to call tool X"; analogy: manager gives orders; defined by model vendors' APIs.

MCP : Tool protocol; solves how tools standardize integration; analogy: employees use USB-C; defined by Anthropic open standard.

They are orthogonal, no encapsulation. Model uses FC to express call intent; underlying tool transport (MCP or handwritten) is invisible to model — it just sees tool names, descriptions, parameters.

Derived question: "Too many tools (dozens/hundreds) — what to do?" Three orthogonal dimensions:

Tool form (CLI, dedicated function, HTTP call) — implements single tool.

Integration protocol (handwritten registration vs MCP) — governs reuse.

Routing strategy (flat list, hierarchical drill-down, on-demand retrieval) — governs scale.

Misconception: "MCP tools blow context" confuses dimensions. Three real combos:

CLI + handwritten + hierarchical (Claude Code)

Function + MCP + hierarchical (MCP Server exposes groups)

Any form + MCP + flat (stuff hundreds of tools into prompt → explodes)

Explosion depends only on routing, not form or protocol. Routing patterns: hierarchical (category → tool), on-demand retrieval (vector search tool descriptions), namespace (db.query, mail.send). MCP's native Server → Tool two-layer structure supports hierarchical routing natively.

Summary: FC solves "model expresses call", MCP solves "tool standardizes integration" — one model-side, one tool-side, orthogonal and composable. Tool count blows context only if routing is flat; hierarchical or retrieval routing solves it regardless of MCP.

06 Tool Call Failure Handling & Infinite Loop Prevention

Two failure locations, often confused:

A. Parsing failure: Before execution — tool_call not valid JSON. Handle: Pydantic validation + retry + fallback.

B. Execution failure: JSON parsed, function throws error. Handle: feed error back to model, let it adjust parameters or switch tool.

Three frequent failures:

Parsing failure: validate, retry once with error message, then fallback (default / degrade).

Timeout: exponential backoff (1s, 2s, 4s), then soft-fail — feed "tool temporarily unavailable" to model, let it decide alternate tool or degraded answer. Distinguish network timeout (retryable) vs business slowness (e.g., transfer cannot blindly retry).

Data too large: slim before LLM — tool layer (field selection, pagination), agent layer (summarize with small model), huge data to disk/vector store.

Code snippets illustrate each.

Four anti-infinite-loop guardrails:

Max iteration cap: state counter, hard stop at N (say "force terminate/degrade", not "stuck").

Completion flag: planner/reflection node judges goal met, sets done, proactive exit.

Result validation: each round checks if result meets criteria, stop immediately on success.

Human-in-the-loop: pause for approval on high-risk ops or repeated failures.

Validation produces judgment; flag produces action; causal chain: validation passes → flag set → loop terminates . LangGraph implementation shows state with iteration counter, should_continue function checking cap, done flag, tool calls, plus framework-level recursion_limit (default 25) as final safety net.

Summary: Parsing failure → validate + fallback; timeout → soft-fail back to model; large data → compress before LLM. Four guardrails: hard cap, goal flag, result validation, human approval.

07 Short-term/Long-term Memory, Knowledge Base, Logs — How to Distinguish?

Comparison:

Short-term memory : Stores current conversation window; retrieval via context; update: compress/overflow = lost.

Long-term memory : Stores user-specific preferences, historical conclusions; retrieval semantic search; update: rewrite, merge, evict continuously.

Knowledge base (RAG) : Stores public business docs: manuals, policies; retrieval semantic search; update via review process.

Execution logs : Stores what ran this time; retrieval time-based file storage; update append-only.

Short-term memory is the conversation window itself — no design needed, compression/overflow naturally discards. Essence: Knowledge base stores public, static; memory stores private, accumulating; logs are audit trails. Example: refund policy = knowledge base; "customer lives in Shanghai, prefers SF Express" = memory; "Mar 8 he checked logistics once" = log. Litmus test for memory: Will the stored item be semantically retrieved later and influence next decision? "I'm allergic to peanuts" → retrieved next order → menu auto-filters = memory. "This dialogue took 2 minutes" → never affects reply = log.

Lifecycle: Write (explicit user "remember this" or auto-extract reusable preferences/facts), Retrieve (each turn, search relevant memory, inject into context), Evict (time decay, merge similar, capacity-based importance eviction). Conflict resolution: user says "live in Beijing" then "moved to Shanghai". Two schools: (1) Write-time disambiguation (Mem0 v2): detect contradiction, UPDATE/DELETE old — clean but irreversible deletion risk, expensive (every write needs retrieval + second LLM). (2) Append-only + retrieval-time reasoning (Mem0 v3): both facts with timestamps coexist, query uses semantic search + time sort to pick latest. Safer — deletion irreversible, sorting rule adjustable.

Summary: Is it memory? → Will it be semantically retrieved and affect decisions. Conflict handling: append safer than delete because deletion is irreversible.

08 Context Full? Overflow vs Corruption Difference?

Comparison:

Overflow : Essence cannot fit; symptom task fails/errors; stealth obvious immediate; fix compress, sliding window, offload.

Corruption (Context Rot) : Essence fits but cannot find; symptom no errors, decision quality silently drops; stealth hidden, looks normal externally; fix active extraction, structuring, noise cleanup.

Corruption cause: limited attention — longer context dilutes attention per token, irrelevant content crowds out useful signals. Example: 80-turn conversation, refund record at turn 20 still in context, but model replies "no relevant info found" — task runs, answer quietly wrong. Therefore context management must proactively subtract, not wait until full.

Three engineering layers:

Overflow triad: Sliding window (keep last N turns, simple, token-controlled, loses early important info), Summary compression (compress old dialogue to summary, retains key info but may lose detail), Memory offload (store key info in vector DB, retrieve on demand, no sliding loss but needs infra, adds latency).

Isolation beats compression: Compression reduces after entering context — wastes tokens and loses fidelity. Better: keep large intermediate data out of main context entirely . Example: main agent task "find payment callback function". Option A: main agent searches — tens of thousands of tokens enter main context, become permanent noise. Option B: delegate to sub-agent — main context adds only two messages (task description + conclusion), intermediate tokens destroyed with sub-agent. Claude Code's Task tool, Deep Research's retrieval sub-agents use this pattern.

Cost accounting + KV Cache three iron laws: Each LLM call carries full history, cumulative cost grows quadratically. Save money by maximizing KV Cache hits: (1) System prompt and tool definitions frozen — any change (even a space) invalidates cache, earlier changes hurt more. (2) Dynamic info appended at end, never inserted into system. (3) Use standard API format, don't hand-concatenate strings. Real incident: adding "Current time: {{now}}" to system prompt caused first-token latency 0.5s → 3-5s, monthly bill doubled — timestamp changed every call → system changed → all subsequent tokens cache missed. Optimization savings don't add: stable prefix (KV Cache) saves 28.3%, history compression saves 17.5%, together only 30% (not 45.8%) because compression shortens cacheable prefix. Must measure combined effect.

Summary: Overflow = won't fit; corruption = fits but unfindable, stealthier. Fix overflow with compression and isolation — isolation superior. Fix cost with KV Cache three laws. Don't let model passively retrieve; actively feed it distilled knowledge.

09 Complete RAG Pipeline? How to Retrieve More Accurately?

RAG = open-book exam: pure LLM closed-book, hallucinates when unsure; RAG retrieves relevant passages first, then answers. Core is retrieval — wrong retrieval makes generation useless.

Two phases:

Offline indexing (once/periodic): Load docs → chunk → embed → store in vector DB.

Online query (per request): Query → vector recall top 20-50 → rerank to top 3-5 → build prompt → generate.

Walkthrough with "Company policy QA assistant":

Offline: "Travel expense policy v3.2" (8000 words) → load preserving heading hierarchy → recursive chunk by "section → paragraph" ("Local transport reimbursement standard" separate chunk ~256 tokens, 15% overlap to avoid cutting key clause at boundary) → embed each chunk → store vector + original text + metadata (heading path: policy/travel/transport).

Online: Employee asks "Shanghai trip, taxi reimbursable? limit?" → embed query → coarse recall top 30 ("transport standard", "accommodation standard", "approval flow" — fast but noisy) → rerank with cross-encoder ("transport standard" rises to #1, "accommodation" pushed down) → take top 3 chunks into prompt → generate answer citing chunks: "Local transport reimbursed actuals, daily cap 200 CNY". Every number comes from retrieved chunks — if "transport standard" missing from recall, generation can only hallucinate.

⚠️ Both phases must use same embedding model — mismatch = silent wrong results.

Three-layer retrieval quality optimization:

Hybrid search: Dense (vector) vs sparse (BM25). Dense compares meaning — "puppy" finds "young dog". Sparse (BM25) matches exact keywords — "HTTP-403" finds that exact code. Blind spots complement: dense misses exact codes, sparse misses synonyms. Run both, merge with RRF (Reciprocal Rank Fusion): score = Σ 1/(60 + rank). Ranks from both lists fused; documents high in both win.

Rerank (cross-encoder): Recall is coarse ("looks relevant"), feeding all to LLM wastes tokens and risks distraction. Two-step: recall with bi-encoder (doc vectors precomputed, only query vector computed online, instant scan hundreds) → rerank top 20-50 with cross-encoder (query+doc concatenated, deep interaction, accurate but slow, no precompute). Analogy: recall = headhunter resume screen; rerank = interviewer deep dive.

Agentic RAG: Traditional RAG = reflex: always retrieve then answer. Agentic RAG lets model decide whether to retrieve, how many rounds — that decision layer makes it "Agentic":

Need private/realtime data → retrieve. ("Where's my expense report from last week?")

Common knowledge sufficient → skip. ("What is VAT?") saves cost, latency, avoids irrelevant docs.

Multi-hop complex → retrieve multiple rounds. ("Which expenses most often rejected?" round1: rejection reasons; round2: invoice standards; cross-verify before concluding.)

Traditional RAG = "library allows one search then write report"; Agentic RAG = researcher repeatedly checks, cross-verifies, writes when evidence sufficient.

FAQs: Chunk size? Start 256-1024 tokens, recursive/structure-aware splitting, 10-20% overlap. Retrieval metrics? recall@k (did we get the right one), MRR (rank of first correct), nDCG (overall ranking quality).

Summary: RAG is open-book, core is retrieving the right passage. Vector handles semantics, BM25 handles keywords, RRF fuses, cross-encoder reranks. Retrieval costs — let Agent decide when to search.

10 How to Evaluate an Agent? How to Iterate Continuously?

Metrics layer: Task success rate (hard metric), end-to-end latency, tool call count, token consumption, hallucination rate (guardrails for cost/quality). Success rate primary; optimizing others meaningless if success rate low.

Evaluation layer — two types:

Mechanically verifiable tasks (file written, config changed): use verifiers checking machine-verifiable facts — not trusting agent's self-report "all done".

Open-ended tasks (report generation, complaint handling): LLM-as-Judge with scoring rubric.

Verifier design references SWE-bench: split "fix complete" into two independent propositions:

FAIL_TO_PASS: failed before fix, passes after → proves issue resolved
PASS_TO_PASS: passes before and after → proves no regression

Checking only first allows agent to delete test assertions; checking only second equals no check. Both required.

Iteration layer: Run eval → analyze failures → locate prompt/tool/planning issue → fix → rerun, A/B compare; periodically refresh test set to prevent overfitting.

Three insights, each more valuable:

Evaluation can be automated, optimization cannot. "Agent self-optimizes" is wrong — how? Rewrite its own code? Real loop: run eval (auto) → failure attribution (AI-assisted localization + human analysis) → modify prompt/tools/planning (human decision) → rerun.

Pass@k vs Pass^k — one symbol, worlds apart.

Pass@k : At least 1 success in k runs; formula 1-(1-p)^k; measures capability ceiling: "can it occasionally miracle?"; applicable: research, open-ended creation.

Pass^k : All k runs succeed (and no veto items); formula p^k; measures business reliability: "can it deliver consistently?"; applicable: payments, refunds, permission changes.

Example: single-run success p=0.6, k=5: Pass@5 ≈ 99% (looks almost guaranteed); Pass^5 ≈ 7.8% (five consecutive clean runs still hard). Side-effect scenarios must never use Pass@k — that number lies.

Failure attribution must target "first error" and correct layer. Root cause = first deviation in trajectory; subsequent errors are cascades. Example: edit_file old_string mismatch → three retries fail. Root cause: first edit error (1), consequence: three retries (not 3 independent issues). Attribution must separate Harness (scaffolding around model: tool definitions, observation channels, loop framework) from model. Classic AndroidWorld accounting task: agent 32 steps, claims done, verification shows record missing. Replay: step 8 thought "I cannot actually see the content of the image"; step 11 four expense figures hallucinated. Root cause: not "model can't OCR" — this agent is text-only, observation space lacked images. It's a Harness observation channel defect. Mis-attribution → "model capability insufficient" → swap model / fine-tune OCR → money wasted on wrong fix.

Summary: Evaluation automatable, optimization not. Pass@k measures ceiling, Pass^k measures reliability. Attribute to first error, and to correct layer (Harness vs model) — wrong attribution sends optimization in wrong direction.