AI/ML Engineer Top 100 Interview Questions

An AI/ML engineer turns a business problem into a working model in production and keeps it working: from framing the problem and preparing data, through training and evaluation, to deployment, monitoring and retraining.

flowchart LR
  A["Frame the problem"] --> B["Collect and label data"]
  B --> C["Build features"]
  C --> D["Train and evaluate"]
  D --> E["Deploy and serve"]
  E --> F["Monitor"]
  F -->|drift or new data| B

Responsibilities at each stage:

• Framing: agree with stakeholders on the decision the model supports, the target to predict and how success is measured.
• Data: build reliable pipelines, check quality, and design labeling where labels are missing.
• Modeling: establish baselines, engineer features, train candidates and evaluate them honestly on held-out data.
• Deployment: package the model with its preprocessing, and serve it in batch or in real time within latency and cost limits.
• Monitoring: track data drift, prediction quality and business impact, and trigger retraining.
• Governance: record data, code and model versions, document limitations, and check fairness and privacy.

Compared with a data scientist, the role leans toward production engineering. Compared with a software engineer, it adds statistics, experimentation and data quality.

Start from the decision the business wants to improve, then define exactly what to predict, for whom, when, and how a better prediction changes an outcome you can measure.

1. Name the decision: who acts on the prediction, and what do they do differently?
2. Define the target: a precise, measurable label, such as "cancels within 30 days", not "is unhappy".
3. Define the unit and timing: one prediction per customer per week, made with data available at that moment.
4. Choose an offline metric that reflects how the prediction is used. If only the top 5% get an offer, measure precision in the top 5%, not overall accuracy.
5. Choose the online metric: the business outcome, such as retention rate, measured in a controlled experiment.
6. Set guardrails: metrics that must not get worse, such as offer costs or complaint rates.
7. Set a baseline: the current rule or a simple model the new one must beat.

The most common failure is a well-optimized model for the wrong target. Ten minutes spent agreeing on the label saves months of work on the wrong one.

A model's expected error comes from three sources: bias from assumptions too simple to capture the pattern, variance from being too sensitive to the particular training sample, and noise that no model can remove. Making a model more flexible lowers bias but raises variance.

• expected error = bias² + variance + irreducible noise

Using it in practice:

• Compare training and validation error to tell which problem you have before changing anything.
• Regularization, depth limits and early stopping are dials that move a model along the trade-off.
• Bagging mainly reduces variance; boosting mainly reduces bias.

Very large neural networks can generalize well despite enormous capacity, so treat the trade-off as a diagnostic guide rather than a strict law.

K-fold cross-validation splits the data into k equal folds, trains k models each using k − 1 folds, and validates each on the remaining fold. Averaging the k scores gives a more reliable estimate than a single split, and the spread shows how stable it is.

1. Shuffle the data, unless order matters, and split it into k folds.
2. For each fold, train on the other folds and score on this one.
3. Report the mean and standard deviation of the k scores.
4. Retrain the final model on all the data once choices are made.

How to choose k:

• 5 or 10 is the usual choice, balancing estimate quality against training cost.
• Small datasets benefit from larger k, since each model sees more training data.
• Expensive models may only afford k = 3, or a single hold-out set for very large data.

Rules that keep it honest:

• Fit preprocessing, such as scaling or encoding, inside each fold, or information leaks into validation.
• Use grouped folds when rows from one customer or patient must not be split.
• Use time-ordered splits for forecasting.
• Keep a separate test set untouched for the final estimate, because the cross-validation score was used to make choices.

A learning curve plots training and validation scores against the amount of training data. The gap between the curves, and whether validation is still improving, tells you whether to collect more data, add model capacity or improve the features.

How to build one:

1. Train on increasing subsets, such as 10%, 20% and so on up to 100% of the training data.
2. Evaluate each on the same validation set, ideally averaging over several random subsets.
3. Plot both curves and read the trend at the right-hand end.

Learning curves turn "should we label 50,000 more examples?" from a guess into an estimate. A flat validation curve says the money is better spent elsewhere.

The curse of dimensionality describes how data becomes sparse as the number of features grows: the volume of the feature space grows exponentially, so a fixed amount of data covers it ever more thinly and distances between points become less meaningful.

Effects on machine learning:

• Neighborhoods stop being local: to capture a fixed share of the data, a "neighborhood" must span most of each feature's range.
• Distances concentrate: the nearest and farthest points end up at almost the same distance, which weakens nearest-neighbor methods and clustering.
• More data is needed: the samples required to cover the space grow very fast with dimensions.
• Overfitting gets easier: with many features and few rows, some feature will separate the training data by chance.

How to counter it:

1. Feature selection to remove irrelevant and redundant features.
2. Dimensionality reduction, such as principal components or learned embeddings.
3. Regularization that penalizes using many features.
4. Models that cope well, such as tree ensembles and linear models with regularization, rather than raw distance-based methods.

Real data usually lies near a much lower-dimensional structure. That is why high-dimensional problems are solvable at all.

A discriminative model learns the boundary between classes directly, the probability of a label given the inputs. A generative model learns how the data itself is distributed, often for each class, so it can also generate new examples and compute how likely an input is.

When to use which:

• Use discriminative models for most prediction tasks with enough labeled data, since they focus all capacity on the decision.
• Use generative models when you need to generate data, detect unusual inputs by their low likelihood, handle missing features, or learn from very little data with sensible assumptions.

Modern generative models for images and text are generative in this same sense: they learn the distribution of the data well enough to sample from it.

Start with the simplest model that could work, and move to a more complex one only when it gives a meaningful gain that is worth its costs in explainability, maintenance, latency and risk.

Questions that decide it:

1. How much does the extra accuracy matter in business terms, such as fraud caught or revenue?
2. Must decisions be explained to customers, regulators or doctors?
3. How stable is the data? Simple models often degrade more gracefully when patterns shift.
4. Who will maintain it, and how much monitoring can they support?

A strong simple baseline also makes the complex model's gain measurable. If the complex model wins by 0.3% on a metric nobody acts on, ship the simple one.

Linear regression assumes the target is a linear function of the features plus independent, evenly spread errors. Predictions can still be useful when assumptions bend, but coefficients, confidence intervals and p-values become unreliable.

How to check:

• Plot residuals against predictions: curves suggest non-linearity, and a funnel shape suggests changing variance.
• Plot residuals over time to spot dependence.
• Compute variance inflation factors for collinearity.

Outliers and high-leverage points can also drag the line. Inspect the largest residuals before trusting any coefficient.

Logistic regression is linear in the log-odds of the outcome. Each coefficient is the change in log-odds for a one-unit increase in its feature, holding the others fixed, and its exponential is an odds ratio.

• log(p / (1 − p)) = b0 + b1·x1 + ... + bn·xn
• odds ratio for feature i = e^bi

How to read them:

• A positive coefficient raises the odds of the outcome; a negative one lowers them.
• An odds ratio of 2 doubles the odds per unit; 0.5 halves them.
• The effect on probability is not constant: the same change in odds moves a 50% probability far more than a 2% one.

Cautions:

• Units matter: a coefficient per dollar looks tiny next to one per thousand dollars. Standardize features to compare their importance.
• Correlated features split and destabilize coefficients, so interpret them only with low collinearity.
• Regularization shrinks coefficients toward zero, which biases them for interpretation.
• Coefficients show association, not causation, unless the data comes from a controlled experiment.

K-nearest neighbors stores the training data and predicts a new point by finding the k closest training points: it takes a majority vote for classification or an average for regression. There is no training step; all the work happens at prediction time.

Key points:

• Distance matters: straight-line distance is common, so features must be scaled, or a feature measured in thousands dominates one measured in single digits.
• k controls the trade-off: small k follows noise, which is high variance; large k blurs real boundaries, which is high bias. Choose it by cross-validation.
• Weighted votes, where closer neighbors count more, often help.

Weaknesses:

• Slow predictions on large datasets, since each one compares against all stored points. Approximate neighbor indexes help.
• Memory grows with the training data.
• High dimensions make distances less meaningful.
• Irrelevant features add noise to every distance.
• Imbalanced classes let the majority class win votes.

It remains a useful baseline and is natural for recommendations and similarity search, where "find similar items" is the goal itself.

This classifier, often called the naive probabilistic classifier, scores each class by multiplying the class's prior probability by the probability of each observed feature given that class. It assumes the features are independent within a class, which is rarely true but makes training and prediction extremely cheap.

• score(class) = P(class) × P(feature1 given class) × ... × P(featureN given class)
• Pick the class with the highest score.

Key details:

• Training is counting: class frequencies and feature frequencies per class, in one pass over the data.
• Add-one smoothing: add 1 to every count, so a word never seen with a class does not make its probability zero.
• Use log probabilities: sum logs instead of multiplying many small numbers, which would underflow to zero.

Strengths and weaknesses:

• Fast, works well with little data and many features, and is a strong baseline for text classification.
• Its probabilities are poorly calibrated, often too extreme, because the independence assumption counts correlated evidence twice.
• Correlated features, such as repeated phrases, distort its scores.

A support vector machine chooses the boundary that separates the classes with the widest possible margin, the gap between the boundary and the closest training points. Those closest points, the support vectors, alone determine the boundary.

How it works:

• Maximum margin: among all boundaries that separate the classes, pick the one farthest from both, which tends to generalize better.
• Soft margin: real data overlaps, so some points may sit inside the margin or on the wrong side, at a penalty.
• The C parameter sets that penalty. A large C fits the training data tightly with a narrow margin and risks overfitting; a small C allows more violations for a wider, smoother margin.
• Hinge loss is the loss function this behavior comes from: points correctly classified beyond the margin cost nothing.
• Kernels let the same method draw curved boundaries.

Practical points:

• Scale features, since the margin is measured in feature space distances.
• It works well on small and medium datasets with many features, such as text.
• Training becomes slow beyond tens of thousands of rows with nonlinear kernels.
• Outputs are distances to the boundary, not probabilities. Calibrate them if probabilities are needed.

The kernel trick lets a linear method, such as a support vector machine, learn nonlinear boundaries by computing similarities as if the data had been mapped into a much higher-dimensional space, without ever building that space explicitly.

How it works:

• Many algorithms use the data only through dot products between pairs of points.
• A kernel function returns the dot product the points would have after some transformation, computed directly from the original features.
• The model is linear in the transformed space, which corresponds to a curved boundary in the original one.

Common kernels:

• Linear: no transformation; best for high-dimensional sparse data such as text.
• Polynomial: captures interactions between features up to a chosen degree.
• Radial basis function: similarity that decays with distance; flexible and the usual default for nonlinear problems.

When you need it: the classes are not linearly separable, the dataset is small to medium, and you want a strong, well-founded model without manual feature engineering.

When to avoid it: large datasets, because the method compares every pair of points, so cost grows roughly with the square of the row count. Tree ensembles, neural networks or approximate kernel features scale better.

An unconstrained tree keeps splitting until every leaf is pure, memorizing noise in the training data. Limit its growth while building it, prune it afterwards, or replace a single tree with an ensemble.

How to tune:

1. Choose these values by cross-validation, not by eye.
2. Watch the gap between training and validation scores as you loosen each constraint.
3. Prefer minimum samples per leaf and depth first; they are the most effective and easiest to explain.

If one tree is still too unstable, use a random forest or gradient boosting, which keep the strengths of trees while reducing their variance.

A random forest averages many deep decision trees, each trained on a bootstrap sample of the rows and choosing each split from a random subset of features. Averaging trees whose errors are not strongly correlated cancels much of their individual variance.

Why the randomness matters:

• Bootstrap samples make each tree see a slightly different dataset.
• Random feature subsets at each split stop every tree from choosing the same strong feature first, so trees become more different, and averaging helps more.
• Each tree can stay deep and low-bias, because the average controls variance.

Out-of-bag error:

• Each bootstrap sample leaves out about a third of the rows.
• For every row, predict using only the trees that did not train on it, and score those predictions.
• This gives a validation estimate for free, without a separate hold-out set.

Practical points:

• More trees never increase overfitting; they only cost time. Add trees until the out-of-bag score stops improving.
• Feature importances based on impurity favor features with many distinct values; prefer permutation importance.

Gradient boosting builds trees one after another, each new tree trained to correct the errors of all the trees before it. Each tree's contribution is shrunk by a learning rate, so the model improves in many small, careful steps.

1. Start with a simple prediction, such as the mean of the target for regression.
2. Compute the errors the current model makes on each row. More generally, compute the gradient of the loss for each row; for squared error, that is just the residual.
3. Fit a small tree to predict those errors from the features.
4. Add the tree to the model, multiplied by the learning rate.
5. Repeat until a set number of trees, or until validation error stops improving.

flowchart LR
  A["Initial prediction"] --> B["Compute residuals"]
  B --> C["Fit small tree to residuals"]
  C --> D["Add tree times learning rate"]
  D --> E{"Validation still improving?"}
  E -->|yes| B
  E -->|no| F["Final model"]

Why it works well: each tree focuses on what the model still gets wrong, so the ensemble steadily reduces bias. Small trees and a low learning rate keep each step from overfitting.

Risks: it keeps fitting harder cases, including noise, so it needs early stopping on a validation set.

The most important settings control how many trees are built, how much each one contributes, and how complex each tree is. They interact, so tune them together rather than one at a time.

A practical tuning order:

1. Fix a moderate learning rate, such as 0.1, and use early stopping to find the number of trees.
2. Tune tree complexity: depth or leaves, and minimum samples per leaf.
3. Tune subsampling and regularization.
4. Lower the learning rate and let early stopping add trees, which usually gives a final small gain.

Averaging combines several models' predictions with fixed weights, usually equal. Stacking trains a second-level model to learn how best to combine the first-level models' predictions, using predictions made on data those models did not train on.

How stacking is done correctly:

1. Generate out-of-fold predictions for every base model with cross-validation, so each prediction comes from a model that never saw that row.
2. Train a simple meta-model, often a regularized linear model, on those predictions.
3. At serving time, run every base model, then the meta-model.

When to use which:

• Averaging gives most of the benefit of ensembling when models are diverse and similarly good.
• Stacking helps when models are good in different regions of the data.
• In production, weigh the extra latency and maintenance against a gain that is often small.

They differ in how features are judged. Filter methods score each feature with a statistic before modeling, wrapper methods search for the subset that gives the best model score, and embedded methods select features as part of training the model itself.

When to use which:

• Filter methods for a quick first cut over thousands of features, such as dropping constant or near-duplicate ones.
• Wrapper methods when the feature count is modest and the best subset really matters.
• Embedded methods as the default in practice, through regularized linear models or tree ensembles.

Do feature selection inside cross-validation. Selecting features on the full dataset and then validating leaks information and inflates scores.