AI/ML Engineer Top 100 Interview Questions

Hardware processes batches as rectangular arrays, so sequences of different lengths must be padded to a common length, with masks that make the model and the loss ignore the padding. Group sequences by length to keep padding waste low.

1. Pad shorter sequences with a special padding value up to the longest in the batch, not the longest in the dataset.
2. Mask attention so real tokens never attend to padding positions.
3. Mask the loss so padding positions add nothing to the loss or the metrics.
4. Bucket by length: sort or group examples of similar length into the same batch to reduce wasted computation.
5. Batch by token budget, not by a fixed number of sequences, so batches of short sequences hold more examples.
6. Truncate or split very long sequences, with sliding windows when context across the cut matters.
7. For recurrent networks, use packed sequences so computation stops at each sequence's real end.

Forgetting a mask is a silent bug: the model trains and gives plausible results, but learns from padding and behaves differently when the batch composition changes.

An autoencoder is a neural network trained to reproduce its input after squeezing it through a narrow middle layer, the bottleneck. To reconstruct well, the encoder must learn a compact representation that keeps the most important information.

flowchart LR
  A["Input"] --> B["Encoder"]
  B --> C["Bottleneck code"]
  C --> D["Decoder"]
  D --> E["Reconstruction"]
  E -->|reconstruction error| F["Training loss"]

Common uses:

• Dimensionality reduction: the bottleneck code is a learned, nonlinear compression of the data.
• Anomaly detection: trained on normal data only, it reconstructs unusual inputs poorly, so high reconstruction error flags them.
• Denoising: trained to map corrupted inputs to clean ones, it learns to remove noise.
• Pretraining: the encoder becomes a feature extractor for a downstream task with few labels.
• Generation: variational autoencoders learn a smooth code space that can be sampled to create new data.

Pitfalls:

• Without a real bottleneck or other constraint, it can learn to copy the input, which is useless.
• For anomaly detection, any anomalies in the training data teach it to reconstruct them well, so clean the training set.

A generative adversarial network trains a generator to fool a discriminator that tries to tell real from generated data. A variational autoencoder learns to encode data into a smooth probability distribution over codes and decode samples from it, trained by reconstruction plus a penalty that keeps codes well organized.

When to use which:

• Adversarial networks when realism of individual samples matters most, such as image synthesis or super-resolution.
• Variational autoencoders when you need a structured latent space, an encoder, stable training or anomaly scores.

For high-quality image generation, diffusion models, which learn to remove noise step by step, have largely replaced both. Their samples are realistic, they cover the data well and they train stably, at the cost of slower generation.

Intersection over union measures how well a predicted box overlaps the true box: the area they share divided by the area they cover together. A prediction counts as correct only if its overlap with an unmatched true box of the same class passes a threshold.

• IoU = area of overlap / (area of box A + area of box B − area of overlap)

What each term means:

• Overlap: the intersection rectangle, with width max(0, min(right edges) − max(left edges)), and height computed the same way.
• Union: both areas added together, minus the overlap that was counted twice.

How detection metrics use it:

1. Each predicted box is matched to the true box it overlaps most, in descending order of confidence.
2. A match with IoU at or above the threshold, commonly 0.5, is a true positive; otherwise it is a false positive. Unmatched true boxes are false negatives.
3. Precision and recall across confidence levels give average precision per class, and its mean over classes summarizes the detector.

Stricter thresholds, such as 0.75, reward precise localization. Averaging over thresholds from 0.5 to 0.95 rewards both finding objects and drawing tight boxes.

Detectors usually predict several overlapping boxes for the same object. Non-maximum suppression keeps the highest-scoring box, removes the others that overlap it too much, and repeats with the remaining boxes.

1. Sort boxes by confidence, highest first.
2. Take the top box and keep it.
3. Remove every remaining box whose intersection over union with the kept box is at or above a threshold, such as 0.5.
4. Repeat until no boxes remain.

Key points:

• Run it per class, so a person box does not suppress an overlapping bicycle box.
• Filter low scores first, which removes most boxes cheaply before the pairwise comparisons.
• Threshold trade-off: a low threshold can merge genuinely adjacent objects, such as people in a crowd; a high one leaves duplicates.
• Soft variants lower the scores of overlapping boxes instead of deleting them, which helps in crowded scenes.

The simple version compares pairs of boxes, so its cost grows with the square of the box count. That is fine after score filtering leaves a few hundred boxes.

Data parallelism copies the whole model to every device and splits each batch between them, combining gradients after each step. Model parallelism splits the model itself across devices, because it is too large for one, and passes activations between them.

Variations used in practice:

• Sharded data parallelism: each device stores only a slice of the optimizer state, gradients and even weights, gathering them when needed. It trains much larger models with data-parallel simplicity.
• Pipeline parallelism: consecutive groups of layers sit on different devices, and micro-batches flow through them like an assembly line to keep every device busy.
• Tensor parallelism: single large matrix operations are split across devices, which needs fast links between them.

Large models usually combine all three. Data parallelism is always the first tool, because it is the simplest and scales well until the model no longer fits.

Mixed-precision training does most computation in 16-bit floating point while keeping a 32-bit master copy of the weights for updates. Modern accelerators run 16-bit matrix operations several times faster than 32-bit ones, and 16-bit activations take half the memory.

How it works:

1. Keep 32-bit master weights.
2. Each step, run the forward and backward passes in 16-bit, using a 16-bit copy of the weights.
3. Scale the loss up before the backward pass so small gradients do not round to zero in 16-bit, then scale them back down. A 16-bit format with the full 32-bit exponent range avoids most of this need.
4. Apply the update to the 32-bit master weights.
5. Keep numerically sensitive operations, such as large sums and normalization statistics, in 32-bit.

Where the gains come from:

• Faster matrix math on specialized hardware units.
• Half the activation memory, which allows larger batches or longer sequences.
• Less data to move between memory and compute, and between devices.

Weight and optimizer memory does not shrink much, because the master weights and optimizer state stay in 32-bit. The big memory saving is in activations.

Work from the simplest possible check upward: first prove the model can memorize a tiny dataset, then check data, loss and gradients, and only then tune hyperparameters. Most failures are bugs, not bad architecture choices.

flowchart TD
  A["Loss not improving"] --> B{"Can it overfit a tiny batch?"}
  B -->|no| C["Check data, labels, loss and output layer"]
  B -->|yes| D{"Gradients healthy?"}
  D -->|no| E["Fix initialization, normalization or learning rate"]
  D -->|yes| F{"Validation also flat?"}
  F -->|yes| G["Check features, leakage and model capacity"]
  F -->|no| H["Tune regularization and schedule"]

1. Overfit one small batch: a working setup reaches near-zero loss on 10 to 50 examples. If not, there is a bug.
2. Inspect the data the model actually receives: shapes, value ranges, label alignment and augmentation output.
3. Check the loss and output layer match: for example, applying softmax twice, or using a loss that expects raw scores on probabilities.
4. Check the initial loss: it should match random guessing, such as ln(10) ≈ 2.3 for ten balanced classes.
5. Watch gradients and activations: zeros, values that are not numbers, or exploding norms point to the learning rate, initialization or normalization.
6. Run a learning rate range test to find a workable rate.
7. Compare with a simple baseline to confirm that the data contains a learnable signal at all.

Choose the loss that matches the type of output, the noise in the data and the cost of different errors, and make sure it moves in the same direction as the metric you report.

Points that matter:

• Asymmetric costs can be built into the loss with weights, or handled later with the decision threshold.
• Losses that must be differentiable cannot be metrics such as accuracy directly, so use a smooth proxy and report the real metric.
• Check the scale: combining several losses needs weights, or one will dominate the gradients.

Contrastive learning trains an encoder so that two different views of the same example land close together in embedding space, while views of different examples land far apart. The views come from data augmentation or from naturally paired data, so no human labels are needed.

flowchart LR
  A["Image"] --> B["View 1: crop and recolor"]
  A --> C["View 2: different crop"]
  B --> D["Encoder"]
  C --> D
  D --> E["Pull matching views together"]
  D --> F["Push other examples apart"]

How it works:

1. Make two augmented views of each example in a batch.
2. Encode all views into vectors.
3. For each view, treat its partner as the positive and every other example in the batch as a negative.
4. Use a softmax over similarities, with a temperature, so the loss is lowest when the positive is the most similar.

Why it learns useful features: to identify its partner among many others, the model must keep what makes an example distinctive and ignore what the augmentations change, such as color or cropping.

Practical points:

• Augmentation choices define what is ignored; heavy color changes are wrong if color matters to the task.
• More negatives help, which favors large batches or memory banks of past embeddings.
• The same idea aligns paired data, such as images with captions, into one shared embedding space.

MLOps applies DevOps practices, such as automation, version control, testing and monitoring, to machine learning systems. It extends them because model behavior depends on data as well as code, and models can degrade silently without any code change.

Core MLOps practices:

• Reproducible training pipelines that record exactly which data, code and parameters produced each model.
• Automated validation gates for data and models before deployment.
• A model registry with versions, lineage and approval status.
• Safe deployment, such as shadow or canary releases, plus a fast rollback.
• Monitoring of data, predictions and business outcomes, feeding retraining decisions.

Maturity usually grows in steps: manual notebooks, then automated training pipelines, then automated retraining and deployment with monitoring that closes the loop.

An automated pipeline takes raw data to a deployed, monitored model through repeatable stages, each of which can stop the run if quality checks fail.

flowchart LR
  A["Ingest data"] --> B["Validate data"]
  B --> C["Build features"]
  C --> D["Train"]
  D --> E["Evaluate against current model"]
  E --> F{"Passes gates?"}
  F -->|no| G["Stop and alert"]
  F -->|yes| H["Register model"]
  H --> I["Deploy gradually"]
  I --> J["Monitor and trigger retraining"]

1. Ingest: pull a defined snapshot of data, so the run is reproducible.
2. Validate data: schema, ranges, missing values and distribution shifts, against expectations.
3. Build features with the same code used in serving.
4. Train with recorded parameters, code version and data version.
5. Evaluate on held-out data, per important slice, and against the current production model.
6. Gate: promote only if the new model beats or matches the current one without regressions in key slices or fairness checks.
7. Register the model with its lineage and evaluation report.
8. Deploy with a shadow run, canary or experiment.
9. Monitor inputs, outputs and outcomes, and trigger retraining when needed.

Each stage should be idempotent and independently re-runnable, so a failure at evaluation does not mean reloading all the data.

A feature store is a shared system for defining, computing, storing and serving machine learning features. Each feature is defined once and used for both training data and live predictions, which prevents the two from drifting apart.

flowchart LR
  A["Batch and stream sources"] --> B["Feature pipelines"]
  B --> C["Offline store with full history"]
  B --> D["Online store with latest values"]
  C -->|point-in-time join| E["Training datasets"]
  D -->|low-latency reads| F["Live model serving"]

What it provides:

• One definition per feature, so training and serving use the same logic instead of two separately written implementations.
• An offline store with historical values and timestamps, used to build training sets with point-in-time joins.
• An online store with the latest value per entity, readable in milliseconds during live predictions.
• A registry where teams discover and reuse existing features, with owners and documentation.
• Freshness monitoring and backfills when definitions change.

When it is worth it: several models share features, real-time serving needs the same features as training, or training-serving mismatches have already caused incidents. A single batch model may not need one.

Point-in-time correctness means every feature in a training row uses only the information that was available at the moment the prediction would have been made. Joining today's feature values onto past labels leaks the future and makes offline results look far better than production will be.

How leaks happen:

• Joining the latest value of a feature, such as the current account balance, onto labels from months ago.
• Aggregates computed over the whole history, including data after the prediction time.
• Records that were updated later, such as a corrected address, overwriting what was known at the time.
• Ingestion delays: data stamped at 10:00 that arrived at 14:00 was not available for a prediction at 11:00.

How to guarantee it:

1. Store feature history with validity timestamps, rather than overwriting values.
2. Join "as of" each label's prediction time: take the latest value at or before that time.
3. Use availability time, not event time, when data arrives late.
4. Generate training data with the same code the feature store uses, rather than ad hoc queries.
5. Check for leakage: a single feature with suspiciously high importance often signals future information.

A model registry is the system of record for trained models: it stores each version together with everything needed to understand, reproduce, approve, deploy and roll it back.

How it is used:

• Deployment pipelines pull a specific registered version, never a file from someone's laptop.
• Promotion between stages requires passing automated checks and, for high-risk models, a human approval.
• Rollback is switching production back to the previous registered version.
• Audits can answer which model made a given decision, trained on which data.

Keep the registry the single source of truth. A model running in production that is not in the registry is an untracked risk.

Treat every dataset used for training as an immutable, identifiable snapshot, and record its identifier in the model's lineage together with the code commit and parameters. Any model can then be traced to, and rebuilt from, exactly the data it learned from.

Practical techniques:

• Immutable snapshots: write each training dataset once, to a versioned path or table version, and never edit it in place.
• Table formats with time travel, which let you query a table as it was at a past version or timestamp.
• Content hashes of files or partitions, so identical data gets the same identifier and silent changes are detected.
• Pointers, not copies: store large data once and version the metadata that says which files make up a dataset.
• Version labels separately: labeling guidelines and label sets change over time, and they matter as much as the raw data.
• Version schemas and feature definitions, since a changed transformation is a changed dataset.
• Link it all in the registry: model version, data version, code commit, environment and parameters.

Complications to plan for:

• Deletion requests and retention rules may require removing a person's data from old snapshots, so record which models used which data in order to retrain them.
• Storage costs grow with snapshots, so set retention for datasets no production model depends on.

Retrain when there is evidence that the model has gone stale or that new data will improve it: on a schedule suited to how fast the domain changes, when monitoring detects drift or a drop in performance, or when the business or data sources change.

Guardrails:

• A retrain is not a release. Every retrained model passes the same validation gates and comparison against the current model before deployment.
• Investigate before retraining on drift: a data pipeline bug looks like drift, and retraining on broken data bakes the bug in.
• Track how much each retrain helps, which tells you the right cadence. If weekly retrains rarely change anything, monthly may be enough.

Test the data, the code that transforms it, the model's quality and behavior, and the serving system. Ordinary software tests catch only a fraction of machine learning failures.

How to run them: put cheap checks, such as data validation and unit tests, on every pipeline run, and expensive ones, such as full evaluation and load tests, as gates before promotion. Record every result with the model version in the registry.

A failed gate should stop the release automatically and say exactly which check failed. Teams then fix the cause instead of arguing about whether a drop was acceptable.

An A/B test splits traffic in fixed proportions for a set period to measure the difference between options accurately. A multi-armed bandit shifts traffic toward better-performing options while the test runs, earning more along the way at the cost of a less precise measurement.

When to use which:

• A/B tests for launching a new model or feature where you need a trustworthy estimate of the effect, including on guardrail metrics.
• Bandits for headline or creative selection, short promotions, and continuously choosing among many similar options where only reward matters.
• Contextual bandits pick the best option per user context, and work like a lightweight, continuously learning recommender.

Bandits rely on quick feedback. If the outcome takes weeks to observe, such as retention, the adaptation comes too late to help.

Combine a lean model, efficient serving, and enough horizontally scaled capacity sized from measured per-request costs, keeping everything that is not prediction out of the hot path.

Techniques:

• Make the model cheaper: smaller architectures, distillation into a compact model, quantization and pruning.
• Precompute features: read them from a low-latency online store instead of computing aggregates per request.
• Batch dynamically: group requests that arrive within a few milliseconds, which multiplies throughput on accelerators.
• Cache predictions for repeated inputs, such as popular items or anonymous pages.
• Use an optimized runtime with compiled graphs and fused operations.
• Scale horizontally behind a load balancer, with autoscaling on request rate or queue time.
• Set a latency budget per stage: network, feature lookup, inference and post-processing, and measure the 99th percentile, not the average.
• Plan a fallback, such as a simpler model or a default answer, for when the budget is exceeded.

A useful sizing rule: requests in flight = request rate × latency. It tells you how much concurrency the serving fleet must support.