AI/ML Engineer Top 100 Interview Questions

One-hot encoding thousands of categories creates huge, sparse inputs and lets rare categories overfit. Use encodings that compress the categories or share information between them.

Practical points:

• Decide how to treat unseen categories at prediction time, such as a shared "unknown" bucket.
• Check whether the feature is really an identifier, such as a user ID, that will not generalize to new rows.
• Combine approaches: frequency plus embedding is a common pairing.
• Compare encodings by validation score and by how the model behaves on rare and new categories, not only on common ones.

Multicollinearity means two or more features are strongly linearly related, so the model cannot separate their individual effects. Predictions may stay fine, but coefficients become unstable, can flip sign, and lose their meaning.

• VIF_j = 1 / (1 − R²_j)

What each term means:

• R²_j: how well feature j is predicted by a linear regression on all the other features.
• VIF_j, the variance inflation factor: how much the variance of feature j's coefficient is inflated by its correlation with the others. A value of 1 means no collinearity; above 5 to 10 is usually a concern.

How to detect it:

1. A correlation matrix catches pairs, but not one feature explained by several others together.
2. Variance inflation factors catch both.
3. Unstable coefficients that change a lot between bootstrap samples or after small data changes are a warning sign.

How to handle it: drop or combine redundant features, use regularization that spreads weight across them, or use principal components. When only prediction matters, collinearity may be acceptable.

A raw timestamp is rarely useful to a model. Break it into the parts that drive behavior, encode cyclical parts so that midnight sits next to 23:00, and measure time relative to meaningful events.

Avoiding leakage:

• Compute every feature as of the prediction time, using only events before it.
• Rolling windows must end before the prediction moment, not at the end of the day or the dataset.
• Watch for timestamps that are filled in later, such as "closed at", which reveal the outcome.

On typical tabular data, gradient-boosted trees and random forests usually match or beat neural networks with less tuning, because their way of building decisions suits how tabular features behave.

Reasons:

• Mixed feature types: tables combine counts, amounts, flags and categories on very different scales. Trees split on thresholds and do not care about scale.
• Irregular patterns: tabular relationships are often step-like, such as "more than 3 failed payments", which trees capture directly while networks prefer smooth functions.
• Uninformative features: trees ignore features that never give good splits, while networks spread attention across them.
• Dataset size: many tabular datasets have thousands to a few million rows, often too few for a network to beat strong tree models.
• Missing values: many tree methods handle them natively.
• Less tuning: good defaults and early stopping get trees close to their best quickly.

When neural networks win on tables:

• Very large datasets.
• High-cardinality categories that benefit from learned embeddings.
• Tables combined with text, images or sequences in one model.
• When the model must share representations with other tasks.

Interview tip: recommend gradient-boosted trees as the default baseline for tabular problems; it shows practical judgment rather than reaching for the most fashionable model.

Log loss, or cross-entropy loss, measures how well predicted probabilities match the true labels. It is the average negative log of the probability the model gave to the correct outcome.

• log_loss = −(1/N) × Σ [y × log(p) + (1 − y) × log(1 − p)]

What each term means:

• y: the true label, 1 or 0.
• p: the predicted probability that the label is 1.
• N: the number of examples.

Why confident mistakes cost so much: the penalty for the correct class is −log(p). At p = 0.9 it is about 0.105, at 0.5 about 0.693, at 0.1 about 2.303, and at 0.01 about 4.605. It grows without limit as the probability for the true class approaches zero, so one confidently wrong prediction can outweigh many good ones.

When to use it: when the probabilities themselves matter, such as pricing risk or ranking by likelihood, because it rewards calibration as well as correct ranking. Clip probabilities away from exactly 0 and 1 before taking logs, to avoid infinite values.

R-squared is the share of the target's variance that a regression model explains. Adjusted R-squared corrects it for the number of features, because plain R-squared never falls when you add a feature, even a useless one.

• R² = 1 − SS_res / SS_tot
• adjusted R² = 1 − (1 − R²) × (n − 1) / (n − p − 1)

What each term means:

• SS_res: the sum of squared differences between actual and predicted values.
• SS_tot: the sum of squared differences between actual values and their mean, which is the error of always predicting the mean.
• n: the number of rows; p: the number of features.

How to read them:

• R² of 0 means no better than predicting the mean; 1 means perfect. On test data it can even be negative, when the model is worse than the mean.
• Adjusted R² rises only if a new feature improves the fit more than chance would, so it is fairer for comparing models with different numbers of features.

When to use them: for explaining linear regression fits. For choosing predictive models, prefer cross-validated error measured in the target's own units, which stakeholders understand more easily.

There is no single correct number of clusters, so combine a quantitative measure of cluster quality with whether the clusters are useful for the purpose, such as distinct customer segments a team can act on.

A practical process:

1. Scale features first, since clustering depends on distances.
2. Compute two or three measures over a range of k, such as 2 to 10.
3. Inspect the candidate solutions: cluster sizes, typical members and their defining features.
4. Check stability on resamples, and prefer solutions that persist.
5. Validate with stakeholders, since four segments people can use beat seven that nobody can tell apart.

If no k looks clearly better, the data may not have distinct clusters, and a continuous score or a density-based method may suit it better.

Centroid-based clustering assigns each point to the nearest of k centers. Density-based clustering finds dense regions separated by sparse ones and marks isolated points as noise. Hierarchical clustering builds a tree of nested clusters that can be cut at any level.

When to use which:

• Centroid-based for large datasets with compact, similar-sized groups, such as quick customer segmentation.
• Density-based for irregular shapes and when noise must be separated, such as geographic hotspots or anomaly detection.
• Hierarchical when the nesting itself is useful, such as product taxonomies, or when you want to explore several levels of detail.

Density-based methods struggle when clusters have very different densities, so check results visually or with domain knowledge.

Pick starting centers, then alternate two steps until nothing changes: assign each value to its nearest center, then move each center to the mean of its assigned values. Every pass can only lower the total squared distance, so the loop converges.

Key points:

• Initialization matters: k-means finds a local optimum. Spreading starting centers across the data, or running several random starts and keeping the best, avoids poor solutions.
• Stop on convergence: when centers stop moving, or after a maximum number of iterations.
• Handle empty clusters: keep the old center, or reseed it from the farthest point.
• Inertia, the total squared distance from points to their centers, measures fit and drives the elbow method.

In one dimension, sorting the data allows exact methods, but the iterative version shown here is the same algorithm used for any number of features, with absolute distance replaced by straight-line distance.

Without labels, model what normal looks like and flag points that fit it poorly. Choose the method by data type and volume, then review a sample of flagged points with domain experts, because "unusual" is not always "bad".

Practical process:

1. Define the purpose: fraud, equipment faults and data errors need different features.
2. Clean the training window, since known incidents in the "normal" data teach the model that they are normal.
3. Set the alert budget: decide how many alerts a team can review per day, and set the threshold from it.
4. Collect feedback: reviewed alerts become labels, which can later train a supervised model that performs much better.

Account for seasonality and context. A spike at 09:00 on a Monday may be normal, while the same value at 03:00 is not.

Compute the first and third quartiles, take their difference as the interquartile range, and flag values below Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR. Because quartiles ignore the extremes, the fences are not dragged around by the outliers themselves, unlike a mean and standard deviation rule.

Key points:

• Robust: a few extreme values barely move the quartiles, so they cannot hide themselves.
• The 1.5 factor is a convention; 3 marks only extreme outliers. Choose it from how many alerts you can review.
• Quartile methods differ slightly between tools; pick one and use it consistently.
• Skewed data, such as response times or incomes, gets many false high outliers. Apply the rule to the logarithm of the values instead.
• Small samples give unstable quartiles, so be cautious below about 20 values.

Flagged values are candidates, not verdicts. A 980 ms response may be a real slow request worth investigating rather than bad data to delete.

Use a linear projection, such as principal components, when you want a faithful global picture: distances and directions mean something, and the axes can be interpreted. Use a nonlinear embedding, such as t-SNE or similar neighbor-preserving methods, when you want to see local cluster structure that a linear view would squash together.

When to use which:

• Linear first, for a quick and honest overview and as input compression for models.
• Nonlinear to explore whether embeddings, such as images or text vectors, form distinct groups.

Reading nonlinear plots safely:

• Do not read meaning into gaps between clusters, cluster sizes or global layout.
• Try several settings and seeds; trust only structure that persists.
• Use the plot to form hypotheses, then test them with the original data.

Collaborative filtering recommends items that similar users liked, using only the pattern of user interactions. Content-based recommendation suggests items whose attributes resemble what the user already liked. Most production systems combine both.

When to use which:

• Collaborative when interaction data is rich, such as a large catalog with many active users.
• Content-based for new catalogs, fast-changing items such as news, and niche users.
• Hybrid approaches feed both kinds of signals into one ranking model, which is the usual production design.

Both can create filter bubbles. Add deliberate diversity and exploration so the system keeps learning about new interests.

Matrix factorization represents each user and each item as a short vector of learned numbers, called latent factors, chosen so that the dot product of a user's and an item's vectors approximates how much the user likes the item. Missing entries in the user-item matrix are then predicted from those vectors.

1. Start with a sparse matrix of known interactions, such as ratings or purchases.
2. Learn user and item vectors, typically 20 to 200 numbers each, by minimizing the error on known entries plus a regularization penalty.
3. Add biases: a global average, a per-user offset and a per-item offset capture generous users and universally popular items.
4. Predict every unseen pair with global mean + user bias + item bias + user · item.
5. Recommend the highest-scoring unseen items.

Practical points:

• For clicks and purchases rather than ratings, use implicit-feedback variants that treat unobserved items as weak negatives.
• The learned factors often correspond to recognizable themes, such as genre or price level, though they are not labeled.
• New users and items have no vectors, which is the cold-start problem.

Cold start happens when a new user or a new item has no interaction history, so collaborative methods have nothing to work with. Bridge the gap with information that exists from day one, and learn quickly from the first interactions.

For new users:

• Popular and trending items, ideally by region, device or acquisition channel.
• Onboarding questions: a few preferred topics or example items, chosen to be informative.
• Context: the page they arrived on, their search query or time of day.
• Fast updates: use session behavior within minutes, rather than waiting for a nightly retrain.

For new items:

• Content features: text, category, price and image embeddings map the new item near similar existing ones.
• Controlled exposure: reserve a small share of recommendation slots to show new items and gather feedback.
• Creator or brand history: a new item from a popular seller can borrow some of its predecessors' signal.

Exploration: bandit-style strategies balance showing what is known to work with trying new items and new interests. Track cold-start performance separately, since overall metrics are dominated by established users and items.

For a single user, precision at k is the share of the top k recommendations that are relevant, and recall at k is the share of all the user's relevant items that appear in the top k. Average both across users to evaluate the system.

• precision@k = relevant items in the top k / k
• recall@k = relevant items in the top k / total relevant items

Key points:

• Relevance comes from held-out behavior, such as items the user bought or rated highly in a later time period.
• Choose k to match the interface: if the page shows 5 slots, measure at 5.
• Precision and recall trade off: larger k usually raises recall and lowers precision.
• They ignore order within the top k; use ranking metrics that reward placing relevant items higher when position matters.
• Users with no relevant items make recall undefined, so exclude them or report them separately.

Offline metrics only approximate value. Confirm improvements with an online experiment on clicks, purchases or retention.

In time series, observations are ordered and depend on their own past, so the order itself carries information and the future must never leak into training. Ordinary regression assumes rows are independent and can be shuffled freely.

Key differences:

• Temporal dependence: today's value is strongly related to yesterday's, which is a feature, not a nuisance.
• Trend and seasonality: long-term movement and repeating cycles, such as daily, weekly or yearly patterns, must be modeled explicitly.
• Validation: splits must respect time; random cross-validation overstates accuracy.
• Features must exist at prediction time: lagged values and rolling statistics, not values from the forecast period.
• Multi-step horizons: forecasting 30 days ahead means either predicting each day separately or feeding predictions back in, and errors grow with the horizon.
• Changing patterns: promotions, holidays and structural breaks shift behavior, so models need retraining and event features.
• Uncertainty matters: planning decisions need prediction intervals, not just a single number.

Many forecasting problems are best solved by turning them into supervised learning with carefully built lag features. That works only if the time-based rules above are followed strictly.

A series is stationary when its statistical properties, such as mean, variance and the relationship between a value and its recent past, stay the same over time. Many classical forecasting models assume it, because patterns learned from the past then stay valid in the future.

Signs of non-stationarity:

• A trend: the mean rises or falls over time.
• Seasonality: regular cycles in the mean.
• Changing variance: fluctuations grow as the level grows, common with sales and revenue.

How to make a series stationary:

1. Difference it: model the change from one period to the next, y_t − y_(t−1), which removes a trend.
2. Seasonal differencing: y_t − y_(t−s), such as s = 7 for daily data with a weekly cycle.
3. Transform it: take logarithms to stabilize variance that grows with the level.
4. Remove components: subtract an estimated trend and seasonal pattern.
5. Check: plot the result, and use a statistical unit-root test as supporting evidence.

Tree-based and neural forecasting models do not strictly require stationarity. Even so, differencing or relative-change targets usually help them extrapolate a trend they would otherwise flatten.

Train only on data from before each validation period, and evaluate on the period that follows, repeating the process across several points in time. This is called walk-forward or rolling-origin validation.

flowchart LR
  A["Train on weeks 1 to 20"] --> B["Test on weeks 21 to 24"]
  B --> C["Train on weeks 1 to 24"]
  C --> D["Test on weeks 25 to 28"]
  D --> E["Train on weeks 1 to 28"]
  E --> F["Test on weeks 29 to 32"]

1. Pick several cut-off dates spread over the history.
2. Train on data before each cut-off, and forecast the next horizon, matching the real use, such as 4 weeks ahead.
3. Average errors across folds, and look at their spread.
4. Keep the final period untouched as a test of the chosen model.

Leaks to watch for:

• Features built with future data, such as rolling means centered on the date or normalization fitted on the full series.
• Late-arriving data: if sales are finalized three days late, the model must not use the last three days at prediction time.
• Random shuffling in any step, including hyperparameter search.
• Choosing the model on the test period by trying many variants against it.

Expanding windows use all available history; sliding windows keep a fixed length. Sliding windows suit series where old behavior no longer applies.

For each prediction date, create features from values strictly before that date: lagged values, such as the same day last week, and rolling statistics over recent windows. Each row then becomes an ordinary supervised learning example.

Key points:

• Choose lags from the data's cycles: lag 1 for momentum, lag 7 for weekly patterns, lag 365 for yearly.
• Rolling windows such as means, minimums, maximums and standard deviations over the last 3, 7 or 28 periods summarize recent behavior.
• End every window before the prediction date. A window that includes the target day leaks the answer.
• Match the forecast horizon: if you forecast 7 days ahead, the most recent usable value is 7 days old, so lag 1 is not available.
• Drop or handle early rows that do not have enough history.
• Group by series when you have many, such as one per store, so lags never cross from one store's data into another's.

Add calendar features and known future events, such as holidays and planned promotions, alongside lags. These are known in advance and are safe to use.