AI Engineer Top 100 Interview Questions

I would define measurable task success and collect cases that represent real use, including difficult failures. The set should guide decisions without becoming another training dataset.

1. Define outputs, required evidence, acceptable uncertainty behavior, and evaluation rules before collecting examples, involving people who understand the actual user task.
2. Sample common requests and important edge cases, preserving relevant language, document, and user-group coverage without unnecessary sensitive information.
3. Create reliable labels or rubrics and review disagreements. Include cases where missing or conflicting evidence makes abstention the correct outcome.
4. Keep a held-out portion independent of iteration, version the set, and report quality alongside latency and cost under the same evaluation conditions.

flowchart TD
  A["Define task rubric"]
  B["Collect representative cases"]
  C{"Judgments consistent?"}
  D["Clarify rubric and labels"]
  E["Reserve held-out cases"]
  F["Compare candidate systems"]
  A --> B
  B --> C
  C -->|no| D
  D --> C
  C -->|yes| E
  E --> F

Trade-offs:

• Common cases estimate average performance, while rare severe failures need deliberate coverage even when they are uncommon in traffic.
• Repeatedly optimizing against one set can hide weaknesses, so preserve independent checks and refresh coverage as usage changes.

The methods provide different kinds of evidence about quality. Combine them based on what can be checked objectively and where interpretation or domain expertise is required.

When to use which:

• Use deterministic checks for schema validity, exact calculations, required fields, and observable final-state requirements that have clear pass-or-fail definitions.
• Use model-based judges for scalable assessments only after comparing their decisions with trusted labels, checking position bias and sensitivity to irrelevant wording.
• Use human review for ambiguous or consequential cases, calibrating reviewers with examples and measuring disagreement rather than assuming all reviewers are interchangeable.

A model should not be the sole authority for verifying its own unsupported claims when stronger external evidence is available.

The serving pattern should match how quickly results are needed and how inputs arrive. Freshness, throughput, and predictable response time matter more than a preferred architecture.

When to use which:

• Use online inference when a user or service needs an immediate prediction, including deadlines, capacity limits, and graceful handling of overload.
• Use batch inference when a delay is acceptable and many records can be processed efficiently together, with restartable jobs and completion tracking.
• Use event processing when new observations should continuously update results, explicitly defining event ordering, late arrivals, and duplicate-handling behavior.

Streaming generated tokens to a user is a response-delivery technique and does not by itself imply a continuous event-processing architecture.

Training–serving skew occurs when live inputs or transformations differ from those used during development. The model then receives a meaningfully different representation from the one it learned.

• Definitions: identical field names can hide different units, time windows, missing-value conventions, or eligibility rules between offline and online systems.
• Timing: historical feature construction must use information available at the prediction time, including real availability delays rather than only event timestamps.
• Consistency: version preprocessing and feature contracts, share implementations where practical, and compare offline and live outputs on the same representative records.
• Diagnosis: retain sufficient versioned evidence to reproduce failures while avoiding unnecessary exposure of sensitive raw inputs in logs.

Retraining does not fix a persistent representation mismatch unless the data and serving definitions are corrected as well.

Reproduction requires the full path from data to measured results, not only a saved model. Record the versions and settings that can change behavior.

• Data: preserve dataset snapshots, labeling rules, partition assignments, preprocessing, and any filters that determine which examples enter the experiment.
• Model and execution: record starting weights, architecture, training settings, random seeds, dependency versions, and relevant hardware or numerical settings.
• Application context: version prompts, retrieval collections, tool schemas, and output settings when evaluating a system that includes language-model components.
• Measurement: retain the evaluation rubric, scorer, outputs, and resource measurements so that a claimed improvement can be checked rather than merely repeated from memory.

A fixed seed alone does not guarantee identical results across nondeterministic operations, distributed execution, or changing hosted dependencies.

I would release gradually behind explicit quality and operational gates. The deployment plan must include a tested way to restore a compatible previous system.

1. Validate the complete artifact, including preprocessing, input contracts, resource requirements, and dependent indexes or prompts, using representative offline tests.
2. Observe candidate behavior on copied traffic where appropriate, ensuring that shadow processing cannot execute unintended real-world side effects.
3. Expose a controlled portion of live traffic and compare outcome quality, errors, tail latency, and cost against agreed expansion or rollback thresholds.
4. Expand only when evidence supports it, retaining compatible previous artifacts and monitoring delayed outcomes that may not appear during the initial release window.

flowchart TD
  A["Validate release package"]
  B["Observe limited traffic"]
  C{"Release gates pass?"}
  D["Expand traffic gradually"]
  E["Rollback and investigate"]
  F["Monitor delayed outcomes"]
  A --> B
  B --> C
  C -->|yes| D
  C -->|no| E
  D --> F

Trade-offs:

• Small releases reduce exposure but may need longer observation to reveal rare failures or statistically meaningful outcome changes.
• Rollback becomes harder when models, schemas, and indexes change together, so plan compatibility before deployment rather than during an incident.

Data drift changes the distribution of inputs. Concept drift changes the relationship between inputs and outcomes, so familiar observations may no longer support the same predictions.

When to use which:

• Monitor data drift to detect new populations, collection problems, or changed usage patterns, then investigate whether those changes affect important predictions.
• Monitor outcome quality and subgroup errors to identify changed relationships, accounting for label delays and changes in how outcomes are measured.
• Use retraining only after diagnosing the cause and validating a candidate replacement, because automatic updates can amplify corrupted inputs or unstable labels.

A distribution alert is a reason to investigate rather than evidence that the deployed model has necessarily become inaccurate.

I would profile the full request path before changing the model or architecture. Optimize the dominant delay while preserving required quality and acceptable behavior under load.

1. Separate queue time, retrieval, reranking, input processing, first-token delay, and output generation, measuring tail latency as well as averages.
2. Remove unnecessary sequential work and parallelize independent operations within concurrency limits. Avoid increasing load beyond dependency capacity.
3. Reduce unnecessary context or output length, evaluate smaller adequate models, and use compatible batching or caching where their freshness and isolation rules hold.
4. Load-test the revised system with representative request lengths and concurrency, checking quality, failure rates, and cost before accepting the optimization.

flowchart TD
  A["Profile request stages"]
  B{"Retrieval dominates?"}
  C["Optimize search path"]
  D["Optimize model path"]
  E["Load-test candidate"]
  F{"Quality retained?"}
  G["Accept optimization"]
  H["Revise approach"]
  A --> B
  B -->|yes| C
  B -->|no| D
  C --> E
  D --> E
  E --> F
  F -->|yes| G
  F -->|no| H

Trade-offs:

• Batching can increase throughput while adding queue delay, so choose based on the service’s response-time requirements.
• Token streaming improves perceived responsiveness but does not automatically reduce total computation or the time to a complete answer.

I would distinguish temporary failures from permanent request problems and apply bounded recovery. Every retry must fit the request deadline and respect side-effect safety.

1. Set per-dependency timeouts and an overall deadline, with clear error categories for overload, unavailable services, invalid input, and unauthorized requests.
2. Retry appropriate transient failures with bounded backoff and jitter, respecting server guidance and using concurrency limits to prevent a retry storm.
3. For operations that change state, use duplicate-safe execution or check completion status after ambiguous failures before repeating the action.
4. Provide a validated fallback or explicit failure response when the budget is exhausted, and monitor errors, queues, and recovery success over time.

flowchart TD
  A["Observe dependency failure"]
  B{"Transient and retry safe?"}
  C["Wait within retry budget"]
  D["Retry operation"]
  E{"Deadline remaining?"}
  F["Fallback or report failure"]
  A --> B
  B -->|no| F
  B -->|yes| E
  E -->|yes| C
  C --> D
  D -->|failure| A
  E -->|no| F

Trade-offs:

• Retries improve availability for temporary faults but can worsen overload and duplicate changes if recovery is not carefully controlled.
• Fallbacks preserve service only when they still meet the request’s privacy, freshness, and minimum quality requirements.

I would identify which memory category grows before choosing a remedy. Model weights, activations, gradients, optimizer state, and attention caches require different interventions.

1. Reproduce the failure with measured batch size, sequence length, concurrency, precision, and lifecycle stage, checking for accidentally retained tensors or graphs.
2. During training, examine activation and optimizer-state costs. Compare smaller microbatches, accumulation, recomputation, or reduced trainable state where appropriate.
3. During inference, disable unnecessary gradient recording, control request and context sizes, and inspect cache usage under concurrent long requests.
4. Evaluate supported precision reduction, offloading, or state partitioning when needed, then benchmark quality, latency, and throughput after the change.

flowchart TD
  A["Measure memory growth"]
  B{"Training failure?"}
  C["Inspect activations and training state"]
  D["Inspect weights and request caches"]
  E["Apply targeted reduction"]
  F["Verify capacity and quality"]
  A --> B
  B -->|yes| C
  B -->|no| D
  C --> E
  D --> E
  E --> F

Trade-offs:

• Recomputation saves activation memory but increases processing work, while offloading can introduce transfer-related latency.
• Lower precision and smaller contexts may improve capacity but must be evaluated for numerical or information-loss regressions.

Multiply the number of stored vectors by their dimension and bytes per component. This gives raw vector storage before index structures, text, metadata, and operational overhead.

• Vector count: vector_count = document_count * average_chunks_per_document when every chunk produces one stored vector; measure the actual distribution when it varies.
• Storage formula: raw_bytes = vector_count * dimensions * bytes_per_component, with a consistent numerical representation across the counted vectors.
• Additional costs: include indexing structures, identifiers, permissions, source text, replicas, and temporary capacity needed during updates or migrations.
• Use: estimate scale before implementation, then measure real storage and search performance because raw vector size alone cannot predict total infrastructure requirements.

Calculate input and output charges separately, then aggregate across expected requests. Use the actual billing rules for the deployment and clearly label any planning assumptions.

• Per-request formula: cost = input_tokens * input_rate / unit_size + output_tokens * output_rate / unit_size, where the unit size matches each quoted rate.
• Volume: multiply by representative request counts, accounting for different task types rather than assuming all requests have identical lengths.
• Hidden workload: retries, tool loops, evaluation calls, and intermediate generation can increase total token use beyond the final user-visible answer.
• Uncertainty: use observed distributions and scenario ranges, including caching or other charging rules only when the deployment actually supports and applies them.

Multiply each value vector by its corresponding attention weight, then add the weighted vectors component by component. The result preserves the dimensionality of the values.

• Formula: output_j = sum(weight_i * value_i_j), where i identifies an attended position and j identifies a value-vector component.
• Preconditions: each position must have a compatible value dimension, and the weights should represent the normalized attention distribution for the query.
• Interpretation: larger weights contribute more of their associated values, but value directions and magnitudes also affect the resulting representation.
• Limits: this calculation covers one attention-weighted combination; output projections, multiple heads, residual paths, and later layers add further transformations in a complete network.

I would give each item a contribution based on its rank in each list, add those contributions, and order items by the resulting score.

• Formula: score(item) = sum(1 / (offset + rank)) over lists containing the item, using one-based rank and a positive smoothing offset.
• Motivation: ranking positions can be combined without assuming that different retrieval methods produce raw scores on compatible numerical scales.
• Correctness: count an item at most once per list, preserve stable identifiers, and define deterministic tie-breaking for reproducible output.
• Evaluation: the offset changes the balance between top positions and agreement across lists, so choose settings using relevance judgments and compare with individual retrieval methods.

Evaluate outcomes and errors across meaningful groups and intersections, with attention to sample size and context. One overall quality score can hide large differences in who benefits.

• Data coverage: inspect whether training and evaluation adequately represent the populations, input styles, and operating conditions the application will encounter.
• Labels and proxies: examine how historical decisions and correlated features can encode bias, even when an explicit sensitive attribute is excluded.
• Metrics: compare relevant error rates, coverage, and calibration, recognizing that fairness criteria can conflict and must reflect the decision’s actual consequences.
• Response: investigate root causes and validate changes to data, modeling, or decision policy, then monitor real outcomes rather than declaring fairness from one offline test.

Feature attribution describes how input features relate to a model’s prediction under a particular explanation method. It does not automatically establish causation or justify the decision.

• Scope: distinguish explanations of one prediction from summaries of overall model behavior, because local and global questions require different evidence.
• Assumptions: results can depend on the reference population, treatment of missing features, and assumptions about how correlated inputs vary together.
• Stability: correlated features may share or redistribute importance, so compare explanations across relevant samples and avoid overinterpreting a single ranking.
• Verification: combine attribution with error analysis, domain knowledge, and targeted tests; use appropriate causal methods when the question concerns what an intervention would change.

A persuasive explanation can accompany an incorrect prediction, so explanation quality and predictive correctness must be evaluated separately.

Privacy controls should cover collection, processing, storage, transmission, logging, and deletion. Protecting only the original dataset leaves derived artifacts and operational copies exposed.

• Minimization: collect and retain only information required for the task, separating direct identifiers when the application can operate without them.
• Access: apply authentication, authorization, encryption, and secure secret handling to raw records, derived representations, model inputs, and stored outputs.
• External processing: verify the actual handling of submitted content, retention, and permitted use for the chosen deployment rather than assuming every service behaves identically.
• Operations: avoid unnecessary sensitive data in telemetry, define retention and deletion procedures, and check that access changes propagate into caches, indexes, and backups as applicable.

I would start with user tasks, authoritative sources, permissions, and measurable service requirements. The design should connect evidence quality to controlled serving and ongoing evaluation.

1. Define expected questions, source formats, freshness requirements, access boundaries, traffic, and acceptable errors, establishing a simple baseline and evaluation set.
2. Build versioned ingestion that preserves structure, source locations, and permissions, with monitored indexing and reliable propagation of changes and deletions.
3. Authenticate requests, retrieve eligible evidence, and generate answers only when sufficient support exists, preserving traceable support for important claims.
4. Add verification, explicit uncertainty behavior, observability, load testing, and gradual releases, with clear ownership for failures and source-maintenance responsibilities.

flowchart TD
  A["Authenticate request"]
  B{"Access permitted?"}
  C["Retrieve current evidence"]
  D{"Evidence sufficient?"}
  E["Generate supported answer"]
  F["Verify claims"]
  G["Return result"]
  H["Deny access"]
  I["Clarify or abstain"]
  A --> B
  B -->|no| H
  B -->|yes| C
  C --> D
  D -->|no| I
  D -->|yes| E
  E --> F
  F --> G

Trade-offs:

• More retrieval and verification stages can improve reliability while increasing latency and the number of dependencies requiring monitoring.
• Freshness and strict access controls must remain correct across ingestion, retrieval, memory, and caching rather than existing only at the interface.

Explain the user problem, your own responsibilities, and the evidence behind your decisions. A strong project discussion connects implementation choices to measurable outcomes and remaining limitations.

1. Describe the task and constraints, including what success meant and why the chosen use of AI was appropriate for that problem.
2. Explain the data, baseline, and architecture, clearly separating components you implemented from work owned by other people.
3. Discuss one meaningful trade-off and the evaluation used to decide, including important failure cases rather than only the best aggregate result.
4. Describe deployment, monitoring, and a problem you diagnosed after testing or release, using measurements you can substantiate.
5. Close with the outcome and the next improvement you would prioritize, acknowledging uncertainty where evidence is incomplete.

What the interviewer is listening for: clear ownership, sound technical judgment, credible measurement, and understanding of the complete system.

Use a real situation to show how you reasoned, tested assumptions, and responded to evidence. The useful lesson matters more than presenting every decision as a success.

1. State the context and the hypothesis or disagreement, explaining the practical constraint that made the decision important to the team or users.
2. Describe your responsibility and the alternatives fairly, including what evidence was available and what remained uncertain at the time.
3. Explain the experiment, error analysis, or stakeholder discussion you used to resolve uncertainty, with a clear criterion for deciding what to do.
4. Describe the outcome honestly, including an unsuccessful result if relevant, and explain how it changed your subsequent approach.
5. Identify the transferable lesson without blaming others or claiming improvements that were never measured.

What the interviewer is listening for: intellectual honesty, constructive collaboration, disciplined experimentation, and willingness to revise an approach.