AI Engineer Top 100 Interview Questions

Retrieval-augmented generation supplies external evidence to a model at request time. It connects generated answers to information outside the model’s learned parameters.

• Evidence access: retrieval can supply private, specialized, or recently changed information that is absent from the model’s general training knowledge.
• Separation: the searchable knowledge collection can be updated independently of model weights, provided ingestion and indexing keep pace with changes.
• Traceability: source identifiers and supporting passages can help users verify claims, although the application must still check that the evidence actually supports the answer.
• Limitations: poor retrieval, stale documents, contradictory sources, and unsupported generation remain possible. External context reduces some failure modes without guaranteeing factual correctness.

The retrieval mechanism can use text search, structured queries, vectors, or a combination chosen for the task.

I would treat ingestion as a quality-controlled transformation rather than simple text extraction. The output must preserve relationships needed to interpret the original information.

1. Identify document types and whether readable text exists. Use character recognition where needed and record extraction confidence and failures.
2. Preserve headings, table row and column meanings, units, and footnotes. Remove repeated navigation or headers without discarding substantive information.
3. Attach stable identifiers, source locations, versions, and permission metadata before creating searchable units from the extracted content.
4. Review representative difficult files and test real queries against their extracted content. Keep originals available for evidence verification and later reparsing.

flowchart TD
  A["Inspect source document"]
  B{"Readable text available?"}
  C["Extract text and layout"]
  D["Recognize scanned content"]
  E["Validate structure"]
  F["Attach metadata and index"]
  A --> B
  B -->|yes| C
  B -->|no| D
  C --> E
  D --> E
  E --> F

Trade-offs:

• Detailed layout preservation can improve answer quality while increasing parsing complexity and processing cost.
• Automatic extraction is scalable, but uncertain or structurally complex documents may require targeted review before indexing.

I would preserve meaningful information units and compare candidate strategies on representative queries. Chunk size is a retrieval-and-context trade-off rather than a universal fixed setting.

1. Inspect document structure and likely questions, identifying boundaries that should remain intact, such as definitions, tables, code blocks, and exceptions.
2. Create several candidate strategies using the representation’s input limits and the generation context budget, with overlap only where it preserves needed meaning.
3. Measure retrieval coverage, duplicate results, final-answer support, and latency. Inspect failures where evidence is divided across neighboring chunks.
4. Choose the best validated strategy and retain parent-document links so that additional context can be recovered without making every chunk excessively large.

flowchart TD
  A["Inspect document structure"]
  B["Create candidate chunks"]
  C{"Evidence split across chunks?"}
  D["Adjust boundaries or overlap"]
  E["Evaluate retrieval and answers"]
  F["Select validated strategy"]
  A --> B
  B --> C
  C -->|yes| D
  D --> B
  C -->|no| E
  E --> F

Trade-offs:

• Small chunks improve focus but may lose qualifying context, while large chunks consume more input budget and can dilute matching signals.
• Overlap protects boundaries but increases storage and repeated evidence, so evaluate its benefit rather than adding it automatically.

I would evaluate retrieval quality on actual query–document relevance judgments. Representation size and generic benchmark performance do not establish usefulness for the target domain.

1. Create representative queries with relevance labels, including domain terminology, languages, exact identifiers, and difficult near-matches that should not rank highly.
2. Apply each candidate representation’s expected input formatting and similarity convention, measuring retrieval quality alongside latency, storage, and operating constraints.
3. Inspect failure categories and verify that improvements persist across important segments rather than only raising the overall average.
4. For replacement, rebuild or migrate document vectors in a versioned index, validate the new path, and retain a rollback option during transition.

flowchart TD
  A["Build relevance evaluation"]
  B["Compare representations"]
  C{"Improvement reliable?"}
  D["Keep current representation"]
  E["Build compatible new index"]
  F["Validate and switch"]
  A --> B
  B --> C
  C -->|no| D
  C -->|yes| E
  E --> F

Trade-offs:

• Larger vectors can increase storage and search cost without improving the particular distinctions the application needs.
• Replacing the representation usually requires re-embedding documents; matching dimensions alone does not make two vector spaces compatible.

A vector index supports similarity-based candidate retrieval. A relational store organizes structured records and relationships with explicit data constraints and transactional behavior.

When to use which:

• Use similarity retrieval when relevant items may use different wording or require learned representations rather than exact field matching.
• Use structured storage for canonical status, ownership, and other facts that need precise filtering, consistency, and transactional updates.
• Combine capabilities when useful, ensuring that retrieved candidates are checked against current authoritative records and access rules before being shown or acted upon.

The boundary is not absolute because one storage system may support both capabilities; choose by requirements rather than labels.

Lexical retrieval matches terms, semantic retrieval matches learned representations, and hybrid retrieval combines evidence from both. The best choice depends on the query and document collection.

When to use which:

• Use lexical matching for codes, names within permitted source content, and terminology whose exact spelling carries essential meaning.
• Use semantic matching when relevant answers may be phrased differently, checking that the representation distinguishes important domain-specific meanings.
• Use a hybrid strategy when both patterns occur frequently, evaluating rank combination and duplicate handling rather than directly mixing incompatible raw scores.

Retrieval quality still depends on source preparation, permission filters, and evaluation labels; combining methods does not guarantee improvement for every query.

Exact search identifies nearest items under the selected metric by considering the eligible search space exhaustively. Approximate search reduces work while allowing some nearest items to be missed.

When to use which:

• Use exact search for small collections, evaluation baselines, or situations where the additional computation is justified by the retrieval requirement.
• Use approximate indexing for larger collections when measured recall remains acceptable at the required latency and memory budget.
• Measure approximate recall against exact results on representative queries, including permission filters, updates, and distributions that differ from index-building data.

An exact neighbor is only mathematically nearest in the representation; exactness does not prove semantic relevance or source quality.

Separate encoding represents queries and documents independently, enabling fast candidate search. Joint reranking processes a query and candidate together to model finer interactions.

When to use which:

• Use separate encoding to retrieve candidates from a large collection while keeping document representations ready for repeated queries.
• Use joint reranking when subtle compatibility or ordering errors remain important and a small candidate set can be processed within the latency budget.
• Combine them when the measured gain justifies the extra stage, tuning candidate count and checking whether relevant evidence enters the first-stage results.

A reranker cannot recover a document that was never retrieved, so diagnose candidate recall before investing only in ranking precision.

Normalized discounted cumulative gain evaluates a ranked list using graded relevance, giving more weight to useful results near the top. Normalization compares the observed ranking with an ideal one.

• Discounted gain: DCG@k = sum((2 ** relevance_i - 1) / log2(i + 1)), where i is a one-based rank and relevance_i is its assigned grade.
• Normalization: NDCG@k = DCG@k / ideal_DCG@k, with the ideal value calculated from the best ordering of the available relevance judgments.
• Interpretation: a value of one represents the ideal ordering under those judgments, while misplaced highly relevant results reduce the score.
• Reporting: specify the cutoff, gain convention, and behavior when ideal gain is zero. Consistent labels and duplicate handling are necessary for meaningful comparisons.

Mean reciprocal rank measures how early the first relevant result appears across queries. It is useful when finding one useful result quickly is the main objective.

• Per-query score: reciprocal_rank = 1 / first_relevant_rank, using one-based ranks and zero if no relevant item appears within the evaluated range.
• Aggregate: MRR = sum(reciprocal_rank_for_each_query) / query_count, giving each evaluated query equal weight unless another convention is specified.
• Interpretation: moving the first useful result from a low position to the top can greatly improve the score, reflecting easier discovery.
• Limitation: relevant results after the first do not increase that query’s score, so pair it with coverage-oriented metrics when multiple pieces of evidence are needed.

These criteria assess different failure modes. An answer can be supported by evidence while still missing the question or omitting information necessary for a correct response.

When to use which:

• Use groundedness checks to verify claim-level support, including whether cited passages actually establish the statement rather than merely mentioning the topic.
• Use relevance checks to ensure that the response resolves the user’s specific request without substituting a broader, easier, or unrelated question.
• Use completeness checks against required facts and conditions, especially where an omitted exception would change how the reader should act.

Source accuracy and freshness require separate attention because an answer can faithfully reproduce an outdated or incorrect document.

I would trace the failure through evidence availability, retrieval, context assembly, and generation. The fix should target the failing stage rather than immediately replacing the model.

1. Confirm that an authoritative current source exists, that it was parsed correctly, and that the user is permitted to access the relevant information.
2. Inspect retrieved candidates and ranking decisions. Check whether the needed passage was excluded by filtering, missed by search, or removed during reranking.
3. Inspect the actual model input for truncation, conflicting evidence, lost qualifiers, or a prompt that encourages unsupported completion.
4. Test with a known-correct passage to isolate generation behavior, then add the failure to a regression set and evaluate the targeted change.

flowchart TD
  A["Locate authoritative evidence"]
  B{"Evidence reaches model?"}
  C["Debug retrieval or context"]
  D{"Answer uses evidence correctly?"}
  E["Debug generation"]
  F["Check evaluation assumptions"]
  A --> B
  B -->|no| C
  B -->|yes| D
  D -->|no| E
  D -->|yes| F

Trade-offs:

• Increasing retrieved context can improve coverage but also introduce distracting or contradictory passages.
• A larger model may improve some generation errors while leaving missing evidence, stale content, and permission problems unchanged.

I would use relevant conversation state to resolve references while preserving the user’s current intent. Query rewriting should clarify context without inventing new requirements.

1. Identify pronouns, omitted entities, and constraints inherited from earlier turns, distinguishing confirmed facts from previous model guesses.
2. Create a standalone retrieval query that preserves applicable constraints and incorporates any explicit changes in the latest user request.
3. Retrieve under the same authorization rules, retaining the original question and relevant history for interpreting the returned evidence.
4. Evaluate rewritten and unrevised queries on real follow-ups, checking whether rewriting improves coverage without dropping constraints or changing the question.

flowchart TD
  A["Read current request"]
  B{"Unresolved reference?"}
  C["Resolve from trusted state"]
  D["Build retrieval query"]
  E["Retrieve permitted evidence"]
  F["Answer original request"]
  A --> B
  B -->|yes| C
  C --> D
  B -->|no| D
  D --> E
  E --> F

Trade-offs:

• Additional context can resolve ambiguity, but excessive history can introduce stale assumptions and increase request cost.
• Multiple rewritten queries may improve coverage while adding latency and making accidental changes of intent harder to diagnose.

I would track document identity, versions, and permissions throughout ingestion and caching. Updating the original record is insufficient if stale derived content remains searchable.

1. Maintain stable document identifiers and content hashes, recording which chunks and cached results depend on each source version.
2. Reprocess changed content and replace affected chunks. Propagate deletions and permission revocations so that obsolete material cannot remain accessible.
3. Use idempotent ingestion and versioned index updates where necessary, monitoring failure rates and the delay between source changes and searchable results.
4. Invalidate or version relevant caches and verify freshness through end-to-end queries, preserving a rollback strategy that does not restore revoked access.

flowchart TD
  A["Detect source change"]
  B{"Deleted or access revoked?"}
  C["Remove affected content"]
  D["Rebuild changed chunks"]
  E["Invalidate dependent caches"]
  F["Verify visible state"]
  A --> B
  B -->|yes| C
  B -->|no| D
  C --> E
  D --> E
  E --> F

Trade-offs:

• Frequent updates improve freshness but increase ingestion work, while batching changes reduces overhead at the cost of temporary staleness.
• Cache reuse lowers latency only when version and authorization boundaries remain correct for the request.

I would enforce authorization in trusted application and storage code before evidence reaches the model. A model-generated statement cannot grant access to a document.

1. Authenticate the caller and derive identity, tenant, and permitted scopes from trusted session information rather than client-supplied claims alone.
2. Attach access metadata to indexed content and apply the required restrictions during retrieval, checking permissions again before exposing content when necessary.
3. Scope conversation memory and caches to the correct authorization context, propagating revocations and deletions across derived copies.
4. Test cross-user and cross-tenant cases, including indirect disclosure through summaries, citations, cached responses, and diagnostic logs.

flowchart TD
  A["Authenticate caller"]
  B["Retrieve eligible candidates"]
  C{"Access permitted?"}
  D["Exclude content"]
  E["Build authorized context"]
  F["Generate response"]
  A --> B
  B --> C
  C -->|no| D
  C -->|yes| E
  E --> F

Trade-offs:

• Permission-aware filtering can affect search quality and index design, but inaccessible content must not be used to improve an answer.
• Shared caches are efficient only when their keys and reuse rules cannot mix different users’ authorization scopes.

An agent chooses actions dynamically from observations, while a workflow follows explicitly defined control paths. Use autonomy when that flexibility produces measurable value.

When to use which:

• Use a workflow when the steps and exception routes are reasonably known, especially when reliability, traceability, and bounded execution matter strongly.
• Use an agent when observations determine which tools or actions are useful next and a fixed path would be unnecessarily restrictive.
• Evaluate the simplest viable design first, adding autonomy only when benefits justify extra failure modes, evaluation needs, latency, and operating cost.

A workflow can include model calls, and an agent still needs application-enforced permissions, budgets, and termination conditions.

I would treat a model’s requested action as a proposal that application code must validate. Execution should remain bounded, authorized, and observable throughout the task.

1. Expose narrow tool descriptions and argument schemas that clearly define permitted inputs, possible errors, and whether the operation changes external state.
2. Validate arguments and authorization before execution, applying timeouts, resource limits, and idempotency controls where retries could repeat a side effect.
3. Return structured results or explicit errors, then allow the model to choose a next step only within the remaining task and budget limits.
4. Check the actual completion state and stop on success, exhaustion, or repeated failure. Escalate requests that need additional information or approval.

flowchart TD
  A["Receive proposed tool call"]
  B{"Authorized and valid?"}
  C["Reject or clarify"]
  D["Execute bounded operation"]
  E{"Task complete?"}
  F["Return verified result"]
  G{"Budget remains?"}
  H["Escalate"]
  A --> B
  B -->|no| C
  B -->|yes| D
  D --> E
  E -->|yes| F
  E -->|no| G
  G -->|yes| A
  G -->|no| H

Trade-offs:

• More tool freedom improves flexibility but expands the consequences of a mistaken or manipulated model decision.
• Retries improve resilience only when the system can distinguish a failed action from an action that completed before its response was lost.

Short-term state supports the current task, while long-term memory supports later tasks or sessions. They need different retention, verification, and access rules.

When to use which:

• Use short-term state for temporary observations, unresolved hypotheses, and progress details that should not automatically become lasting facts about a user.
• Use long-term memory for verified preferences or other permitted information whose future usefulness justifies retention, with correction and deletion controls.
• Store provenance and confidence where appropriate, retrieving only relevant memory rather than inserting every past observation into every new request.

A summary can omit important details, so retain references to authoritative records when later decisions require exact information.

Prompt injection occurs when untrusted content attempts to redirect a model’s behavior. The content may arrive through a user message, retrieved document, or tool result.

• Trust boundary: external text is data for the task, not authority to change permissions, reveal secrets, or authorize new actions.
• Containment: enforce access controls, narrow tool capabilities, and argument validation outside the model so that manipulated text cannot grant new privileges.
• Exposure reduction: minimize accessible secrets and sensitive context, and isolate consequential actions from ordinary text-generation operations.
• Evaluation: test adversarial documents and unexpected tool requests, monitoring whether the system attempts unauthorized behavior and whether application controls actually block it.

Clear instructions and delimiters can help, but no prompt wording guarantees that malicious content will never influence the model.

I would evaluate actual task outcomes and permitted actions, not only the final response. A convincing completion message is insufficient if the required state was never achieved.

1. Define success and failure in observable terms, including required final state, prohibited actions, acceptable cost, and the conditions requiring escalation.
2. Run representative tasks in controlled environments with reproducible tool behavior, adding cases involving missing information and recoverable dependency failures.
3. Inspect traces for incorrect arguments, unnecessary actions, loops, and policy violations, and repeat trials when behavior is stochastic.
4. Combine automated state checks with human review, then compare release candidates on reliability, latency, and cost before broader deployment.

flowchart TD
  A["Run controlled task"]
  B{"Required state achieved?"}
  C["Analyze task failure"]
  D{"Actions permitted?"}
  E["Record successful trial"]
  F["Record policy failure"]
  A --> B
  B -->|no| C
  B -->|yes| D
  D -->|yes| E
  D -->|no| F

Trade-offs:

• Outcome checks are robust but may miss risky intermediate actions, so evaluate both the final state and execution trace.
• Repeated trials provide stronger reliability evidence while increasing test cost, making representative coverage and failure severity important priorities.