AI/ML Engineer Top 100 Interview Questions

Choose an error measure that matches how the forecast is used, and check how it behaves on small and zero values. Percentage errors are easy to explain but explode when actual values are small.

When to use which:

• Inventory and planning across many products: weighted percentage error, so big sellers count more.
• Comparing models across very different series: scaled errors.
• When large misses are costly: root mean squared error, or a cost-weighted measure.

Also check bias, meaning whether forecasts run consistently high or low, because a small average error can hide systematic overstocking.

TF-IDF scores a word in a document by how often it appears there (term frequency), discounted by how common it is across all documents (inverse document frequency). Words that are frequent in one document but rare overall get the highest weights.

• tf(word, doc) = count of word in doc / words in doc
• idf(word) = log(number of documents / documents containing word)
• tf_idf = tf × idf

What it achieves:

• Words in every document, such as "the", get an IDF of zero, so they carry no weight.
• Distinctive words, such as "screen" in a review about displays, stand out.

Practical points:

• Implementations vary: smoothed IDF, logarithmic term frequency and row normalization are common, so scores differ slightly between tools.
• Combine it with stop word removal and n-grams, such as "not good", which carry meaning single words miss.
• The resulting sparse vectors work very well with linear classifiers and for keyword search.
• It ignores word order and meaning, so "cheap" and "inexpensive" are unrelated, which embeddings fix.

Bag-of-words features represent a document by counting which words it contains, with one dimension per vocabulary word. Word embeddings represent each word as a dense vector learned from context, so words used in similar ways get similar vectors.

When to use which:

• Bag-of-words for strong, fast baselines, small datasets, keyword-heavy tasks and when explanations by word are needed.
• Embeddings when meaning matters more than exact words, such as semantic search, short texts and multilingual data.

Static word embeddings give "bank" one vector whether it means a river bank or a money bank. Contextual models produce a different vector for each occurrence, which resolves such ambiguity.

Build it in stages, from clear labels to a strong simple baseline, then improve with richer representations only where the baseline falls short, and evaluate each stage on a held-out set that reflects real traffic.

flowchart LR
  A["Define labels and collect data"] --> B["Clean and split"]
  B --> C["Baseline: TF-IDF and linear model"]
  C --> D["Evaluate per class"]
  D --> E{"Good enough?"}
  E -->|no| F["Embeddings or fine-tuned model"]
  F --> D
  E -->|yes| G["Deploy with threshold and monitoring"]

1. Define labels precisely, with written guidelines and examples, and measure agreement between labelers.
2. Clean carefully: normalize whitespace and encoding, but keep signals such as negations, emoji or capitalization if they matter.
3. Split by time or source to avoid near-duplicates leaking between training and test sets.
4. Baseline: TF-IDF features with a regularized linear classifier, which is often surprisingly strong.
5. Evaluate per class, with precision, recall and a confusion matrix, since rare classes hide in averages.
6. Improve: pretrained embeddings or fine-tuning a pretrained language model, if the gain justifies the cost.
7. Deploy with thresholds per class, a route for low-confidence cases, and monitoring for new topics.

Error analysis drives most improvements. Read misclassified examples; they often reveal labeling problems rather than model problems.

Named entity recognition finds spans of text that refer to things such as people, organizations, places or dates. It is framed as labeling each token with a tag that says whether it begins an entity, is inside one, or is outside any entity, together with the entity type.

The tagging scheme:

• B-TYPE marks the first token of an entity, such as B-LOC.
• I-TYPE marks the following tokens of the same entity.
• O marks tokens outside any entity.
• The B tag lets the model separate two adjacent entities of the same type.

How models handle it:

• A model reads the whole sentence and predicts a tag per token, using context on both sides.
• A structured output layer can forbid invalid sequences, such as I-LOC directly after O.
• Subword tokenizers split words into pieces, so labels must be aligned, usually by tagging the first piece of each word.

Evaluation: score at the entity level, where an entity counts only if both its boundaries and its type are exactly right. Token accuracy looks high simply because most tokens are O.

The area under the ROC curve equals the probability that a randomly chosen positive example is scored higher than a randomly chosen negative one. You can compute it directly from ranks, without drawing the curve.

• AUC = (sum of the positives' ranks − n_pos × (n_pos + 1) / 2) / (n_pos × n_neg)

What each term means:

• Ranks: sort all scores in ascending order and number them from 1; tied scores get the average of their ranks.
• n_pos and n_neg: the number of positive and negative examples.
• The subtracted term removes the rank sum the positives would have if they were all ranked lowest.

How to read it:

• 0.5 is random ordering; 1.0 means every positive outranks every negative.
• It depends only on the order of scores, so it ignores calibration and is unchanged by any monotonic rescaling.
• With heavy class imbalance, a high value can coexist with poor precision, so also check precision-recall measures.

When to use it: to compare how well models rank cases, independent of any threshold. Choose the operating threshold separately, from business costs.

Compare the two models on the same test examples and estimate how much the difference could vary by chance, using a paired bootstrap or a paired statistical test. A difference that is smaller than its own uncertainty is not evidence of improvement.

How to do it:

1. Score both models on identical test rows, so the comparison is paired and noise from different samples cancels out.
2. Bootstrap the difference: resample the test rows with replacement thousands of times, recompute the metric difference each time, and take the 2.5th and 97.5th percentiles as a 95% interval.
3. Read the interval: if it includes zero, the improvement is not established.
4. Repeat across random seeds for models whose training is random, since seed-to-seed variation can be as large as the gain.
5. Correct for many comparisons: testing 20 variants will produce a "significant" winner by chance alone.

Beyond significance: ask whether the gain is practically meaningful, whether it holds across important subgroups, and whether it survives an online experiment, where the real effect is measured.

Multiply the number of cases in each cell of the confusion matrix by the cost of that outcome, add them up, and divide by the number of cases. This turns model evaluation into the business terms that actually drive decisions.

• expected cost per case = (TP × c_TP + FP × c_FP + FN × c_FN + TN × c_TN) / N

What each term means:

• TP, FP, FN, TN: counts of true positives, false positives, false negatives and true negatives.
• c_...: the cost of each outcome, such as the loss from a missed fraud case or the review cost of a flagged transaction. Benefits can be entered as negative costs.
• N: the total number of cases.

When to use it:

• To choose between models whose standard metrics point in different directions.
• To pick a decision threshold: compute expected cost at many thresholds and take the lowest.
• To compare against not using a model at all, which is the real baseline.

Get the costs from the business, and test how sensitive the decision is to them. If the best threshold changes a lot when one cost moves by 20%, say so.

Decide the smallest effect worth detecting, then calculate how many users each group needs so that a real effect of that size is very likely to show up as statistically significant. Running a test with too few users wastes it: it can only find huge effects.

• n per group ≈ (z_α/2 + z_β)² × [p1 × (1 − p1) + p2 × (1 − p2)] / (p1 − p2)²

What each term means:

• p1: the current conversion rate; p2: the rate you hope the new model achieves.
• z_α/2: 1.96 for a 5% significance level, two-sided.
• z_β: 0.84 for 80% power, the chance of detecting a real effect of this size.

Practical points:

• Smaller effects need far more users: halving the detectable difference roughly quadruples the sample.
• Fix the duration in advance and run whole weeks to cover weekly patterns. Stopping early when results look good inflates false positives.
• Randomize by user, not by request, so one person sees one experience.
• Pick one primary metric plus guardrail metrics, and decide them before starting.

An overall metric is a weighted average across subgroups, so a small group can perform badly without moving it, and a change in the mix of groups can move it even when every group stays the same or improves.

How to guard against it:

1. Define important slices in advance: customer type, region, device, language, new versus returning, and protected groups where fairness matters.
2. Report metrics per slice with sample sizes and confidence intervals, not just one number.
3. Set slice-level release criteria, such as "no slice may get more than 2 points worse".
4. Track the mix of slices over time, to separate real degradation from population change.

Momentum optimizers keep a running average of past gradients, so updates build speed along consistent directions and damp oscillations. Adaptive optimizers also scale each parameter's step by the history of its own gradient sizes, so rarely updated or noisy parameters get suitably sized steps automatically.

When to use which:

• Adaptive optimizers for fast, reliable progress on most deep learning tasks, especially attention-based models.
• Momentum with a tuned schedule is still competitive for image models, and sometimes generalizes better.
• Whatever the optimizer, the learning rate and its schedule matter more than the choice between close alternatives.

Adaptive methods store two running averages per parameter, roughly tripling optimizer memory over plain gradient descent. That matters for very large models.

A learning rate schedule changes the step size during training. Warmup starts small and ramps up, which keeps early updates stable while statistics and weights are still poorly set. Decay lowers the rate later, so training can settle into a good minimum instead of bouncing around it.

• Linear warmup: lr = peak × step / warmup_steps for the first steps.
• Cosine decay: lr = floor + 0.5 × (peak − floor) × (1 + cos(π × progress)), where progress runs from 0 to 1 after warmup.

Common schedules:

• Step decay: cut the rate by a factor, such as 10, at set epochs.
• Cosine decay: a smooth fall from the peak to near zero; a popular default.
• Reduce on plateau: lower the rate when the validation metric stops improving.
• One-cycle: rise then fall within a single run, which often trains quickly.

Why warmup helps: large models and large batches can diverge in the first few hundred steps. Adaptive optimizers start with unreliable gradient statistics, and a small initial rate protects against both problems.

Use a learning rate range test to pick the peak: increase the rate steadily over a short run and choose a value somewhat below where the loss starts to blow up.

Initialization sets the starting point of training. Weights that are too small make signals and gradients shrink layer after layer until they vanish; weights that are too large make them grow until they explode. Good initialization keeps the scale of activations and gradients roughly constant through the depth of the network.

Key ideas:

• Break symmetry: if all weights start equal, every neuron in a layer computes the same thing and receives the same update, so they never differentiate. Random initialization fixes this.
• Variance scaling: draw weights with a variance based on the number of inputs to each neuron, the fan-in, so that summing many inputs does not change the overall scale.
• Match the activation: rectified activations zero out half their inputs, so they need about twice the variance of tanh-like activations, commonly 2 / fan_in versus about 1 / fan_in.
• Biases usually start at zero. For a rare positive class, starting the output bias at the log-odds of the base rate speeds up early training.

How modern architectures reduce the risk: normalization layers and residual connections make training much less sensitive to initialization, and pretrained weights replace random starts entirely in transfer learning.

If a deep network's loss does not move at all from the first step, check the scale of activations at each layer. Initialization or a missing normalization is a common cause.

Early stopping ends training when the validation metric stops improving for a set number of evaluations, the patience. Checkpointing saves the model during training. Together they stop wasted training and let you restore the best model rather than the last one.

flowchart TD
  A["Train one epoch"] --> B["Evaluate on validation set"]
  B --> C{"Better than best so far?"}
  C -->|yes| D["Save best checkpoint, reset patience"]
  C -->|no| E["Reduce remaining patience"]
  D --> A
  E --> F{"Patience used up?"}
  F -->|no| A
  F -->|yes| G["Stop and restore best checkpoint"]

Practical points:

• Monitor the metric you care about, not just validation loss, if they disagree.
• Use a minimum improvement so tiny random gains do not reset patience forever.
• Set patience to match noise: noisy validation scores need more patience, or smoothing.
• Save full training state in periodic checkpoints, including optimizer state, learning rate schedule and random seeds, so an interrupted run can resume exactly.
• Keep the best and the latest checkpoints, and delete the rest to save storage.

The validation set used for early stopping has influenced the model, so report the final result on a separate test set.

Each output position needs the whole kernel to fit inside the padded input, and positions are spaced by the stride. That gives one formula per spatial dimension.

• output = floor((input + 2 × padding − dilation × (kernel − 1) − 1) / stride) + 1

What each term means:

• input: the height or width of the input feature map.
• kernel: the kernel size along that dimension, such as 3.
• padding: zeros added on each side; "same" padding with stride 1 uses (kernel − 1) / 2 so the size is unchanged.
• stride: how far the kernel moves between output positions; a stride of 2 roughly halves the size.
• dilation: spacing between kernel elements; 1 means a normal, dense kernel.

Related numbers worth knowing:

• Output channels equal the number of filters, independent of the formula above.
• Parameters: kernel_h × kernel_w × input_channels × output_channels, plus one bias per output channel.

When to use it: designing architectures, debugging shape mismatches, and checking memory use, since activation memory scales with output height × width × channels.

Both reduce the spatial size of feature maps. That lowers compute and memory, widens what each later neuron can see, and makes the network less sensitive to small shifts in the input.

Effects on the network:

• Fewer computations: halving height and width cuts the work of the next layer by about four.
• Larger receptive fields: each later neuron covers more of the original image.
• Some translation tolerance: a feature that moves by a pixel often produces the same pooled output.
• Lost precise location: a problem for tasks such as segmentation or keypoint detection, which add skip connections or upsampling to recover detail.

Global average pooling also lets one network accept different input sizes, because it always produces one value per channel.

A neuron's receptive field is the region of the original input that can influence its value. Early layers see small patches; deeper layers see larger regions, which is how a network combines edges into textures, parts and whole objects.

• r_l = r_(l−1) + (kernel_l − 1) × jump_(l−1)
• jump_l = jump_(l−1) × stride_l, starting from r_0 = 1 and jump_0 = 1

What each term means:

• r_l: the receptive field size after layer l, in input pixels.
• jump: how many input pixels one step in the current feature map corresponds to; every stride multiplies it.

Why it matters:

• An object larger than the receptive field cannot be recognized as a whole by any single neuron.
• Strides and pooling grow the field fastest, because they multiply the jump for every later layer.
• Stacked small kernels give large fields with fewer parameters than one big kernel.
• The effective receptive field is smaller than the theoretical one, since central pixels influence a neuron much more than those at the edges.

A residual connection adds a block's input directly to its output, output = x + F(x), so the block only has to learn a correction to its input rather than a whole new transformation. Very deep networks with these connections train reliably, where plain networks of the same depth train worse than shallower ones.

flowchart LR
  A["Input x"] --> B["Layers compute F of x"]
  A -->|identity shortcut| C["Add"]
  B --> C
  C --> D["Output x plus F of x"]

Why it works:

• Gradient flow: the identity path carries gradients straight back to early layers, so they do not vanish through dozens of multiplications.
• Easy identity: if a block is not useful, its weights can go to near zero and the block passes its input through. Extra depth then cannot make the network worse.
• Smoother optimization: the loss surface becomes easier to descend, so standard optimizers converge.

Where they appear: deep image networks, attention-based language models, where every attention and feed-forward block is wrapped in a residual connection, and many generative models.

Shapes must match to add. When a block changes the number of channels or the resolution, the shortcut uses a small projection, such as a 1×1 convolution.

Data augmentation creates modified copies of training examples that keep their labels, teaching the model which variations do not matter. It acts as regularization and is often the cheapest way to improve a model trained on limited data.

When it hurts:

• The transformation changes the label: flipping a "6" into a "9", mirroring text, or flipping a medical scan where left and right matter.
• Unrealistic variations: heavy distortions the model will never see in production waste capacity.
• Augmenting the validation or test set, which inflates scores.
• Leakage between splits: augmented copies of one original landing in both training and test data.
• Imbalanced effects: augmentations that suit one class but distort another.

Choose augmentations from how real inputs vary, such as lighting and camera angles in a warehouse. Then confirm the benefit on an untouched validation set.

A long short-term memory cell keeps a cell state that runs through time with only small, controlled changes, and uses gates, which are small learned layers with outputs between 0 and 1, to decide what to forget, what to add and what to output at each step.

Why it matters:

• The cell state is updated additively, so gradients can flow across many time steps without vanishing as quickly as in simple recurrent networks.
• The gates learn when to hold information, such as remembering an opening quote until the closing one.

In practice, gated recurrent networks are still used for small, streaming or on-device sequence tasks. Attention-based models have replaced them for most large-scale language work, because they process sequences in parallel and handle long contexts better.