AI/ML Engineer Top 100 Interview Questions

Choose by the model's size and type, the request volume and the latency target. Large neural networks with high, steady traffic benefit from accelerators, while small models and spiky traffic are usually cheaper and simpler on CPUs.

How to decide:

1. Benchmark the real model on both, at the target latency.
2. Compare cost per thousand predictions at expected traffic, including idle time.
3. Optimize the CPU path first, with quantization and an optimized runtime; small models often meet targets without accelerators.
4. Consider a mix: CPU for low-traffic models and accelerators shared across several busy models.

Watch everything you can observe immediately, which is the inputs, the predictions and early proxy outcomes, and measure true performance in cohorts as labels arrive. Each signal answers a different question about whether the model is still healthy.

Practical points:

• Alert on drift only with context: large drift on an unimportant feature may not matter, so weight it by feature importance.
• Validate proxies once against true labels, so you know how well they track real performance.
• Track cohorts by prediction date, so late-arriving labels do not mix different model versions together.
• Keep a small random holdout where possible, giving unbiased labels for evaluation.

The population stability index measures how much a variable's distribution has shifted between a reference period, usually training data, and a recent period. Values are binned, and the differences in each bin's share are summed with a logarithmic weight.

• PSI = Σ (actual% − expected%) × ln(actual% / expected%)

What each term means:

• Bins: usually 10 bands defined by deciles of the reference data, or the model's score bands.
• expected%: the share of reference records in each bin.
• actual%: the share of recent records in the same bin.

Common rules of thumb:

• Below 0.1: no meaningful shift.
• 0.1 to 0.25: moderate shift worth investigating.
• Above 0.25: a major shift; check the data pipeline, then consider retraining.

Practical points:

• Replace zero shares with a small floor, or the logarithm is undefined.
• Keep the bin edges fixed from the reference period.
• It measures change, not harm: a shift in an unimportant feature may not affect accuracy.

Retrain when the same approach still fits and the model only needs fresher data. Rebuild when retraining no longer restores performance, or when the problem, the data available or the requirements have changed enough that new features, a new target or a new model type are needed.

Signs that retraining is enough:

• Performance recovers after retraining on recent data.
• The drift is in the input mix, while the relationships between features and outcome look stable.
• The model still meets latency, cost and explainability requirements.

Signs that a rebuild is needed:

• Retraining stops helping: each retrain recovers less, a sign the relationship itself has changed.
• The business question changed, such as predicting lifetime value instead of churn.
• New, much better data sources exist that the current pipeline cannot use.
• Requirements changed: stricter latency, a new regulation demanding explanations, or a move from batch to real time.
• Technical debt: the pipeline has become too fragile or slow to maintain safely.

Treat a rebuild as a new project: baseline, experiment, and an online test against the existing model. Keep retraining the current model until the replacement proves itself.

A feedback loop occurs when a model's predictions change the data it will later be trained on. The model then learns from a world it shaped itself, which can lock in its own biases and hide its mistakes.

flowchart LR
  A["Model ranks items"] --> B["Users see only top items"]
  B --> C["Clicks only on shown items"]
  C --> D["Training data favors shown items"]
  D --> A

Common examples:

• Recommendations: items never shown get no clicks, so they never get recommended.
• Credit decisions: rejected applicants never get a loan, so the model never learns whether they would have repaid.
• Fraud blocking: blocked transactions never become confirmed fraud or confirmed legitimate, so labels are missing exactly where the model acted.

How to break or reduce it:

1. Explore: reserve a small share of decisions for randomized or diversified choices, so all options keep getting feedback.
2. Keep a holdout group that the model does not influence, for unbiased data and evaluation.
3. Log the probability of each shown option, so training can reweight examples for how likely they were to be shown.
4. Evaluate counterfactually before launch, using logged exploration data.
5. Monitor diversity and coverage, such as the share of the catalog that ever gets shown.

Machine learning systems pile up debt beyond ordinary code: dependencies on data, entangled features and undocumented consumers make them fragile and expensive to change. The model code itself is often a small part of the whole system.

How teams keep it under control:

• Budget time for paying debt down, as with any software system.
• Prefer fewer, well-understood features over many marginal ones; each feature is a dependency to maintain.
• Assign owners to every model, feature and data source.
• Review dependencies before adding a model to a shared platform, since every downstream consumer makes later changes riskier.

Package the model, its preprocessing and its runtime environment as one versioned unit with a defined input and output schema, then prove with test inputs that it produces the same results in production as it did in training.

1. Bundle preprocessing with the model: scaling, encoding and tokenization are saved as part of one pipeline artifact, not reimplemented in the serving code.
2. Pin the environment: exact library versions, ideally in a container image built by the pipeline.
3. Define the signature: expected inputs, types and output format, validated on every request.
4. Store golden examples: a set of inputs with the outputs recorded at training time, checked on every deployment. Any mismatch reveals skew.
5. Use a portable format where useful, such as a standard model exchange format or a compiled graph, for faster and more consistent inference.
6. Fix inference settings: evaluation mode, no dropout, deterministic operations where needed.
7. Version everything together in the registry, so rollback restores the model and its preprocessing as a pair.
8. Add operational hooks: a health check, a warm-up call to load the model before taking traffic, and declared memory and CPU needs.

The most common production bug is preprocessing that differs slightly between training and serving. Golden examples catch it before users do.

A model card is a short, standard document that travels with a model and tells users, reviewers and future maintainers what it does, how well, for whom, and where it should not be used.

Why it matters:

• It prevents misuse: a model built for marketing segments should not quietly be reused for credit decisions.
• It supports reviews and audits with a consistent structure.
• It forces the team to state limitations explicitly, which often reveals missing evaluation.

Keep it short, update it with every significant version, and store it in the model registry so it cannot drift apart from the model it describes.

Federated learning sends the model to where the data lives, such as phones, hospitals or branches, trains it locally on each device's data, and sends back only model updates, which a central server combines. Raw data never leaves its owner.

sequenceDiagram
  participant S as Central server
  participant C1 as Client 1
  participant C2 as Client 2
  S->>C1: Current global model
  S->>C2: Current global model
  C1->>C1: Train on local data
  C2->>C2: Train on local data
  C1->>S: Model update only
  C2->>S: Model update only
  S->>S: Weighted average into new global model

1. The server sends the current model to a sample of clients.
2. Each client trains for a few steps on its own data.
3. Clients send back weight updates, not data.
4. The server averages the updates, weighted by each client's data size, and repeats.

Challenges:

• Uneven data: each client's data differs, which slows and destabilizes training.
• Unreliable clients: devices drop out, so each round uses whichever clients are available.
• Communication cost: updates can be large, so they are compressed and sent infrequently.
• Privacy is not automatic: updates can leak information, so combine federated learning with secure aggregation and differential privacy.
• Poisoned updates from malicious clients need robust aggregation.

Differential privacy is a mathematical guarantee that the output of an analysis, or a trained model, barely changes whether or not any one person's data is included. It is achieved by adding carefully calibrated random noise, and the privacy budget epsilon controls how much: smaller epsilon means stronger privacy and more noise.

• noise scale = sensitivity / epsilon

What each term means:

• Sensitivity: the most one person can change the result; for a count, it is 1.
• Epsilon: the privacy budget. Values around 1 or below are considered strong, and every additional query or training step spends some of it.

In model training: each example's gradient is clipped to a maximum size, then noise is added to the summed gradients before each update. This limits how much any one person can influence the model.

The cost:

• Accuracy drops, most for small groups and rare patterns, because noise swamps signals that come from few people.
• More data reduces the relative cost, since the same noise is spread over a larger signal.
• Training is slower, because per-example gradients must be computed.

Treat the model as an attack surface: attackers can craft inputs to fool it, corrupt the data it learns from, copy it through its API, or extract information about its training data. Each threat needs its own defenses, combined with monitoring.

General practices:

• Threat-model the use case: a fraud model faces active, adaptive adversaries, while a demand forecast mostly does not.
• Keep humans and rules in the loop for high-stakes decisions, so one fooled model is not the only defense.
• Monitor prediction distributions and query patterns for signs of probing.
• Retrain carefully: data from adversarial environments must be filtered before it becomes training data.

Make rollback a fast, rehearsed switch to the previous registered version, which must still be deployable and compatible with the current feature pipeline. Then repair any damage the bad model's decisions caused.

flowchart TD
  A["Alert: quality or business metric drops"] --> B["Confirm the model is the cause"]
  B --> C["Switch traffic to previous model version"]
  C --> D["Verify metrics recover"]
  D --> E["Clean up effects of bad predictions"]
  E --> F["Find root cause and add a test"]

What makes rollback fast:

• Versioned serving: traffic routing points at a model version, so rollback is changing a pointer, not rebuilding anything.
• Keep the previous version warm, or at least quickly deployable, for a period after each release.
• Feature compatibility: if the new model came with new features, the old model must still receive the inputs it expects. Change features and models in separate steps where possible.
• Gradual releases: canary deployments limit how many users the bad model reached.
• Clear authority: the on-call engineer may roll back without waiting for approval.

Cleaning up: cached recommendations, stored scores and downstream decisions made by the bad model may need recomputing. Add the failure case to the pre-release tests so the same problem cannot ship again.

Score every transaction within the payment's latency budget, using precomputed behavioral features and a model combined with rules, and route uncertain cases to review. Delayed fraud labels flow back into training.

flowchart LR
  A["Payment request"] --> B["Scoring service"]
  C["Online feature store: velocity, device, merchant risk"] --> B
  B --> D["Model score plus rules"]
  D --> E{"Decision"}
  E -->|low risk| F["Approve"]
  E -->|medium| G["Step-up check or review"]
  E -->|high| H["Decline"]
  G --> I["Analyst case queue"]
  I --> J["Labels from reviews and chargebacks feed retraining"]
  J --> B

Key design decisions:

• Latency budget: often under 100 ms in total, so features such as "transactions on this card in the last hour" are precomputed by a streaming pipeline.
• Model: gradient-boosted trees on behavioral features are a strong default; rules handle known patterns and regulatory blocks.
• Thresholds from costs: the cost of missed fraud against declined good customers and review capacity.
• Imbalance: fraud is often well under 1% of transactions, so evaluate with precision and recall at the operating threshold, not accuracy.
• Delayed labels: chargebacks arrive weeks later, so monitor drift and proxy outcomes meanwhile.
• Adversaries adapt: retrain frequently and watch for new patterns.
• Explanations of the top reasons help analysts decide quickly.

Use a two-stage design: fast candidate generation narrows millions of products to a few hundred per user, then a ranking model orders them with rich features, followed by business rules and diversity adjustments.

flowchart LR
  A["User visits home page"] --> B["Candidate generation"]
  B --> C["Similar to recent views"]
  B --> D["Collaborative embeddings"]
  B --> E["Trending and popular"]
  C --> F["Ranking model"]
  D --> F
  E --> F
  F --> G["Rules and diversity"]
  G --> H["Recommendations shown"]
  H -->|impressions and clicks| I["Training data"]

Key design decisions:

• Candidates from several sources, each quick: nearest-neighbor search over embeddings, co-purchase data, recently viewed categories and popular items for new users.
• Ranking with a model predicting click or purchase probability from user, item and context features, such as time of day and device.
• Re-ranking for diversity across categories, removing out-of-stock items and applying business rules, such as promotions.
• Cold start: popularity by segment and content features for new users and new items.
• Feedback loops: log impressions, not just clicks, and keep a small exploration share.
• Latency: precompute candidates for active users in batch, and rank in real time within about 100 ms.
• Evaluation: offline ranking metrics on time-split data, then an online test on revenue per visit, with guardrails on diversity and returns.

Forecast demand per product, store and day with a single global model trained across all series, feed it sales history plus price, promotion and calendar information, and produce probabilistic forecasts that drive replenishment.

flowchart LR
  A["Sales, prices, promotions, stock levels"] --> B["Data validation and cleaning"]
  C["Calendar, holidays, weather"] --> B
  B --> D["Feature building: lags and rolling windows"]
  D --> E["Global forecasting model"]
  E --> F["Quantile forecasts per store and product"]
  F --> G["Replenishment and safety stock"]
  F --> H["Planner review and overrides"]

Key design decisions:

• Global model: one gradient-boosted or neural model across thousands of series learns shared patterns and handles sparse items better than one model per series.
• Stockouts: sales are not demand when the shelf was empty, so flag or correct those days, or the model learns to forecast empty shelves.
• Known future inputs: planned promotions, price changes and holidays are features, available in advance.
• Probabilistic output: forecast quantiles, such as the 50th and 90th percentiles, because safety stock depends on uncertainty.
• New products: borrow patterns from similar products by category, price and attributes.
• Evaluation: walk-forward validation, weighted percentage error and bias, per category and store size.
• Operations: nightly runs, monitoring of forecast error, and a way for planners to record overrides that feed back into evaluation.

Separate the possibilities systematically: did the data change, did the serving system change, or did the model itself get worse? Compare the old and new models on the same inputs, and check whether the drop is real or an artifact of measurement.

flowchart TD
  A["Accuracy dropped after release"] --> B{"Measurement correct?"}
  B -->|no| C["Fix labels, joins or metric code"]
  B -->|yes| D{"Old model also worse on current data?"}
  D -->|yes| E["Data or population changed"]
  D -->|no| F{"Same outputs offline and online for same inputs?"}
  F -->|no| G["Training-serving skew"]
  F -->|yes| H["New model regression: compare slices"]

1. Confirm the measurement: check label completeness, metric code and that the comparison periods are equivalent.
2. Run both models on the same recent traffic, using shadow or replayed requests. If both dropped, the data changed, not the model.
3. Compare offline and online outputs for identical inputs; differences point to preprocessing or feature skew.
4. Break the drop down by slice: segment, device, region and feature ranges.
5. Check what else shipped: feature pipeline changes, library upgrades and configuration.
6. Roll back if the impact is serious while the investigation continues.

Write precise guidelines, train and test labelers, measure agreement between them, resolve disagreements systematically, and keep auditing quality. Label quality often limits a model more than its architecture does.

1. Write clear guidelines with definitions, edge cases and many examples, including what to do when unsure.
2. Run a pilot: several labelers label the same small set, then disagreements are discussed and the guidelines refined.
3. Qualify labelers with a test set of known answers before production work.
4. Measure agreement: have a share of items labeled by two or more people, and track chance-corrected agreement, since raw agreement looks high when one class dominates.
5. Resolve disagreements with an expert reviewer or majority vote, and keep the disagreement as a signal of ambiguity.
6. Seed known answers into the queue to monitor each labeler's accuracy over time.
7. Prioritize what to label: uncertain or diverse examples help the model more than random ones.
8. Version labels and guidelines, so models can be traced to the rules their labels followed.

Explain what the model does and how often it is right in the terms of the decision it supports, give concrete examples of where it fails, and agree on how people should use it. Avoid jargon, and never let a single accuracy number stand alone.

1. State the purpose: "It ranks support tickets so urgent ones are answered first."
2. Translate performance into everyday counts: "Of every 100 tickets it marks urgent, about 80 really are, and it misses about 1 urgent ticket in 10."
3. Show where it struggles: new product lines, very short messages or languages with little training data.
4. Explain what it cannot know: information outside its data, such as a customer's phone call yesterday.
5. Define how to use it: as decision support, with clear rules for when people override it, and a way to report errors.
6. Describe the safeguards: monitoring, retraining and who to contact.
7. Use visuals and examples rather than metrics such as AUC.

What the interviewer is listening for:

• Honesty about uncertainty without undermining confidence in a useful tool.
• Translating technical measures into business consequences.
• Recognizing that the right level of trust depends on the stakes of the decision.

Tell a concrete story that shows the decision was based on evidence and trade-offs: what you compared, what the numbers showed, what the simpler model gained, and how it turned out.

1. Context: the problem, the stakes and the constraints, such as latency, explainability or a small team.
2. The options: for example, a regularized linear model or gradient-boosted trees against a deep network.
3. The evidence: offline results with uncertainty, such as "the deep model gained 0.4 points of AUC, inside the run-to-run variation", plus latency, cost and maintenance estimates.
4. The decision and its risks: why the simpler model won, and what you would watch for.
5. The result: production performance, incidents avoided, time saved, or a later point where the decision was revisited.
6. Reflection: what you would do differently, such as running the comparison earlier.

What the interviewer is listening for:

• Judgment: complexity must earn its place with a meaningful gain.
• Rigor: you measured, including uncertainty, rather than relying on taste.
• Communication: you explained the trade-off to people who wanted the fancier option.
• Openness: you would choose the complex model when evidence supports it.

Set measurable success criteria and checkpoints at the start, then stop, or change direction, when evidence shows the project cannot reach a worthwhile result at a reasonable cost. Stopping on evidence is a success of the process, not a failure of the team.

Signals to stop or pivot:

• No lift over the baseline: after reasonable effort, the model does not meaningfully beat simple rules or the current process.
• Data cannot support it: labels are unavailable, too noisy or too expensive, and there is no affordable way to fix that.
• Value below cost: the expected business gain does not cover building, running and maintaining the system.
• A simpler solution exists: a process change, rule or off-the-shelf product solves most of the problem.
• Unacceptable risk: fairness, privacy, legal or safety concerns that cannot be mitigated.
• The need disappeared: the business priority or the underlying problem changed.

How to stop well:

1. Agree on criteria up front, such as "if we cannot reach 70% precision on the pilot by the end of the quarter, we stop".
2. Review at each checkpoint with stakeholders.
3. Document what was learned, including data findings and reusable pipelines.
4. Share the result openly, so others do not repeat the same attempt.

Interview tip: describe a pre-agreed kill criterion; it shows you manage machine learning work as an investment with evidence-based decisions.