Data Scientist Top 100 Interview Questions

Selection bias occurs when the data you analyze is not representative of the population you want to draw conclusions about, because of how cases entered the dataset. Survivorship bias is a common form: analyzing only cases that made it through some filter.

Examples in business data:

• Survivorship: studying the features used by long-term customers, then concluding those features cause retention, while ignoring churned customers who used them too.
• Self-selection: users who opt into a beta are more engaged, so the beta looks better than it would for everyone.
• Response bias: surveys answered mainly by very happy or very unhappy customers.
• Filtering: analyzing only completed orders, which hides problems that stopped orders from completing.

How to avoid it:

1. Define the population first, and check how each record got into the data.
2. Compare the characteristics of included and excluded cases.
3. Start cohorts at a shared starting point, such as signup, and follow everyone, including those who leave.
4. Use randomization when you need to compare treatments.

Collecting more data from the same biased process does not reduce selection bias; it only makes the wrong answer more precise.

It compares the change over time in a group that received a treatment with the change over the same period in a group that did not. Subtracting the control group's change removes trends that affect both groups, such as seasonality.

effect = (treated_after − treated_before) − (control_after − control_before)

The key assumption is parallel trends: without the treatment, both groups would have changed by the same amount. Check it by plotting several periods before the treatment; diverging pre-trends make the estimate unreliable.

Practical choices:

• Scale: decide in advance whether trends should be parallel in absolute terms or in percentages, since the two can give different answers.
• Several periods: use regression with group and time effects when there are many periods or groups.
• Timing: watch for other changes that hit only one group at the same time.
• Anticipation: if behavior changed before the official start, because the change was announced, the estimate is biased.

A propensity score is the estimated probability that a unit receives the treatment, given its observed characteristics. Matching or weighting on that score creates a comparison group whose observed characteristics resemble the treated group, which mimics a randomized comparison for those variables.

1. Fit a model predicting treatment from pre-treatment variables, such as usage, tenure and plan.
2. Match each treated unit with untreated units that have similar scores, or weight units by the inverse of their probability of the treatment they actually received.
3. Check balance: after matching, the distributions of each variable should be similar between groups.
4. Compare outcomes between the matched groups.

Limits:

• It only balances observed variables; an unmeasured confounder, such as motivation, still biases the result.
• It needs overlap: if some treated units have no comparable untreated units, the estimate does not apply to them.
• Extreme weights from scores near 0 or 1 make estimates unstable; trimming or capping helps.
• Variables measured after treatment must never go into the model.

Use it when an experiment is impossible, and present it with clear caveats about unmeasured confounding.

An instrumental variable is something that shifts the chance of receiving a treatment but affects the outcome only through that treatment. It isolates part of the treatment's variation that is free of confounding, even when confounders are unmeasured.

flowchart LR
  Z["Random email reminder"] --> T["Uses the new feature"]
  T --> Y["Retention"]
  U["Unmeasured motivation"] --> T
  U --> Y

Requirements for a valid instrument:

1. Relevance: it must noticeably change treatment uptake; weak instruments give very noisy and biased estimates.
2. Exclusion: it affects the outcome only through the treatment, which must be argued, since it cannot be fully tested.
3. Independence: it is unrelated to the confounders, which randomization of the instrument guarantees.

Common uses:

• Encouragement designs: randomly encouraging some users to try a feature, when the feature itself cannot be forced.
• Non-compliance in experiments: estimating the effect on those who actually used the treatment, not just those offered it.

The estimate applies to units whose treatment changed because of the instrument, which may differ from the whole population.

When treatment is assigned by a cutoff on a continuous score, units just above and just below the cutoff are nearly identical except for the treatment. Comparing them estimates the treatment's effect at the cutoff.

Typical situations:

• Loyalty tiers granted at a points threshold.
• Free shipping above an order value.
• Credit decisions based on a score cutoff.

How to do it well:

1. Plot the outcome against the score, and look for a jump at the cutoff.
2. Fit trends on each side within a narrow window, rather than comparing raw averages.
3. Test several window widths to check the result is stable.
4. Check for manipulation: if units can push themselves just over the cutoff, the groups are no longer comparable, which shows up as a bunching of units just above it.

Limits: the effect is local to units near the cutoff, and it needs many observations close to the threshold.

Choose a quasi-experimental method based on how the feature was rolled out and what comparison data exists. Every method relies on an assumption that should be stated and checked.

Making the result credible:

• Run placebo tests, such as pretending the launch happened earlier, which should show no effect.
• Use more than one method when possible, and check whether they agree.
• Report a range of estimates, with the assumptions behind each.

For future launches, recommend a staged rollout with random order, which makes the next analysis far easier.

An overall rate is a weighted average of segment rates. If the share of traffic in each segment shifts, the overall rate moves even when every segment performs exactly as before. In extreme cases, every segment can improve while the overall rate falls.

Where mix shift appears:

• More traffic from mobile, where conversion is naturally lower.
• A marketing campaign bringing many new users, who convert less than returning ones.
• Growth in a low-price region, lowering the average order value.

How to diagnose it:

1. Break the metric down by the main segments, such as device, channel, region and user tenure.
2. Separate the change into a rate effect, from changes within segments, and a mix effect, from changes in segment shares.
3. Report both, so the team does not chase a problem that does not exist, or miss one hidden by a favorable mix.

Estimate the effect within each segment, test whether the differences between segments are larger than chance would produce, and treat surprising segment results as hypotheses to confirm.

1. Pre-specify a few segments with a reason to expect differences, such as new versus returning users or platform.
2. Test the interaction, meaning the difference between segment effects, not whether each segment's effect is significant on its own. One significant segment and one non-significant segment do not prove the effects differ.
3. Correct for multiple comparisons when many segments are examined.
4. Check sample sizes, since small segments produce noisy, extreme estimates.
5. Confirm with a new experiment targeted at the segment before acting on an unexpected finding.

Why it matters:

• A feature that helps new users but annoys experienced users may deserve a targeted launch.
• Averages can hide harm to an important minority, such as users with accessibility needs.

Interview tip: mentioning that segment searches after the fact are a classic source of false discoveries signals rigor without dismissing segment analysis entirely.

A long-term holdout keeps a small random share of users on the old experience for months, while everyone else receives each new launch. Comparing the two groups measures the combined, lasting effect of all launches, which individual short experiments cannot.

flowchart LR
  A["All eligible users"] --> B["95% receive every launch"]
  A --> C["5% long-term holdout"]
  B --> D["Compare after several months"]
  C --> D
  D --> E["Cumulative lasting impact"]

What it reveals:

• Whether short-term gains persist, fade or reverse over time.
• Whether individual wins add up, or overlap and cancel each other out.
• Long-term effects on retention and revenue that short tests are too brief to measure.

Costs and cautions:

• Holdout users miss improvements, which has a business cost and sometimes a fairness cost.
• Keep the holdout small, stable and randomly chosen, and refresh it periodically.
• Some changes, such as security fixes or legal requirements, must reach everyone and cannot be held back.

A/B testing is powerful but not always appropriate. Skip it, or choose a different method, when the test cannot produce a trustworthy answer or the answer would not change the decision.

When an A/B test is the wrong tool:

• Too little traffic: the required sample would take months; use qualitative research, a bolder change or a quasi-experiment.
• Required changes: legal, security and accessibility fixes must ship regardless of metrics.
• Strong interference: marketplaces and social features where users affect each other need cluster designs instead.
• Long-term outcomes: effects that take a year to appear, such as brand trust, need holdouts or other evidence.
• Ethical limits: withholding something clearly beneficial, or exposing users to harm, is not acceptable.
• One-time decisions: a single big launch, such as a rebrand, offers no stable comparison and nothing to repeat.
• No decision depends on it: if the team will ship regardless, the test only spends time.

What to do instead: user research, staged rollouts with careful monitoring, before-and-after analysis with comparison groups, or simulations based on past data. State clearly how much uncertainty each alternative leaves.

A north star metric captures the core value customers get from the product, in a form that predicts long-term business success. Input metrics are the few levers that teams can move directly, which together drive the north star.

flowchart TD
  A["North star: orders delivered on time per week"] --> B["Active customers"]
  A --> C["Orders per active customer"]
  A --> D["On-time delivery rate"]
  B --> E["New customer activation"]
  B --> F["Returning customer retention"]
  C --> G["Repeat ordering features"]
  D --> H["Delivery partner availability"]

What makes a good north star:

• It reflects value delivered to customers, not just activity or revenue alone.
• It moves within weeks, so teams can see the impact of their work.
• It is hard to game: raising it should require genuinely helping customers.

Using input metrics:

1. Break the north star into a few multiplicative or additive components.
2. Assign each team the inputs it can influence.
3. Pair each input with guardrails, such as complaint rates, so teams do not push one number at the expense of experience.
4. Revisit the tree when the strategy changes, since a metric that fit early growth may not fit a mature product.

Work from the cheapest explanations to the most expensive: first confirm the drop is real, then find where it is concentrated, then find what changed at that time.

flowchart TD
  A["Metric dropped"] --> B{"Data issue? Logging, pipeline delay, definition change"}
  B -->|Yes| C["Fix the data and annotate the dashboard"]
  B -->|No| D["Split by segment: platform, region, channel, user type"]
  D --> E["Locate where the drop is concentrated"]
  E --> F["Match against changes: releases, campaigns, outages, competitors"]
  F --> G["Confirm the cause and size its impact"]

1. Check the data: compare with a second source, look for missing partitions, and check whether the metric definition or tracking changed.
2. Check the calendar: holidays, paydays, weather and seasonality often explain sharp moves.
3. Decompose: split the metric into components, such as traffic times conversion, and by segment, to find where the change lives.
4. Correlate with events: look at releases, experiments, marketing changes, pricing and outages around the start time.
5. Communicate early: share what you have ruled out, with an estimate of when the cause will be confirmed.

Interview tip: interviewers value a structured answer that starts with "is the data right?" before diving into business explanations.

Group users by when they started, then count how many of each cohort are active in each later period, divided by the cohort's size. The result is a table with one row per cohort and period, which shows whether retention is improving for newer cohorts.

1. Define the cohort: usually the week or month of signup or first purchase.
2. Define activity: a meaningful action, such as an order, not just an app open.
3. Compute the period index: the number of periods since the cohort start.
4. Divide by the full cohort size, including users who never returned, which is why the cohort size is counted separately from activity.

Reading the table:

• Compare the same period across cohorts to see whether product changes improved retention.
• Look for the curve flattening, which shows the share of users who become long-term.
• Recent cohorts have fewer observed periods; do not treat missing cells as zero retention.

Find the first time each user reached each step, then count users who reached every step in order. Counting step events independently overstates the funnel, because users can trigger later steps without the earlier ones or in the wrong order.

1. One row per user: use conditional aggregation, taking the minimum timestamp per step.
2. Enforce order: count a step only if its time is after the previous step's time.
3. Compute step rates: divide each step's count by the previous step's count, and the final step by the first for overall conversion.

Decisions to state explicitly:

• Time window: for example, purchase within seven days of the first view.
• Repeated attempts: whether users who abandon and return count once.
• Entry definition: which users enter the funnel, such as visitors from a specific campaign.

Comparing funnels across segments, such as device or channel, usually shows exactly which step is the bottleneck.

Daily active users count distinct users with a qualifying action each day; monthly active users count distinct users over the month. Stickiness divides the average daily count by the monthly count, showing what share of the month's users come back on a typical day.

• Define "active" carefully: a meaningful action, such as viewing content or placing an order, not a background app refresh.
• Count distinct users per day, then average those counts across the days of the period.
• Include zero days: join to a calendar table, so a day with no activity counts as zero rather than disappearing from the average.
• Interpretation: stickiness of 50% means the average user is active about 15 days a month; 10% means about 3 days.

Cautions:

• Stickiness can rise because monthly users fall, for example when casual users leave, so always show both counts.
• Compare stickiness only between products with similar natural usage frequency, since a tax app and a messaging app should differ.

Number the rows in sorted order with a window function, count the rows, then average the middle one or two values. The same idea extends to any percentile and to medians per group.

1. Rank: ROW_NUMBER() OVER (ORDER BY value) gives each row its position.
2. Count: COUNT(*) OVER () gives the total in the same query.
3. Pick the middle: positions (n + 1) / 2 and (n + 2) / 2, using integer division, are the same row for an odd count and the two middle rows for an even count.
4. Average them, which handles both cases.

Variations:

• Per group: add PARTITION BY group_column to both window functions and group the final average by that column.
• Other percentiles: pick the row at position CEIL(p × n) for a simple nearest-rank percentile.
• Large data: many analytical databases provide exact or approximate percentile functions, which are faster; approximate versions trade a little accuracy for speed.

Medians are more useful than means for skewed values such as order amounts and delivery times.

Use the LAG window function to bring the previous period's value onto the current row, then compute the percentage change. Ordering the window by the period makes the comparison explicit and avoids a self-join.

• Formula: (current − previous) / previous × 100.
• Division by zero: wrap the denominator in NULLIF(previous, 0), so a zero previous value gives a null instead of an error.
• First period: has no previous value, so the growth is null, which is correct.
• Gaps: if a week has no rows, LAG compares with the last week present, not the actual previous week; join to a calendar of weeks first.

Analysis habits:

• Compare with the same week last year as well, since week-over-week changes are dominated by seasonality.
• Watch for incomplete current weeks, which always look like a decline.
• Growth rates of small numbers swing wildly, so show the absolute values next to the percentages.

Join each conversion to the user's marketing touches that happened before it, rank those touches by time, and keep the first. Touches after the conversion must be excluded, and conversions with no prior touch should be kept as unattributed rather than silently dropped.

1. Join on user and time: touch_time <= conversion_time.
2. Use a left join, so conversions without any qualifying touch remain.
3. Rank: ROW_NUMBER() OVER (PARTITION BY conversion ORDER BY touch_time) and keep rank 1.
4. Aggregate conversions and revenue by channel.

Limits of first-touch attribution:

• It credits awareness channels and ignores everything that happened later.
• Last-touch attribution has the opposite bias, favoring channels close to purchase.
• Neither shows causal impact; channels may get credit for users who would have converted anyway.

Use attribution for directional reporting, and use holdout experiments, such as pausing a channel in some regions, to measure what each channel actually adds.

Start by understanding what each row represents and how the data was produced, then check quality, then explore distributions and relationships with specific questions in mind.

flowchart LR
  A["Grain: what is one row?"] --> B["Source and logging rules"]
  B --> C["Quality: missing, duplicates, ranges"]
  C --> D["Single-variable distributions"]
  D --> E["Relationships and segments"]
  E --> F["Questions and hypotheses to test"]

1. Grain and keys: confirm what one row means and whether the key is unique.
2. Coverage: check the time range, row counts per day and any gaps.
3. Quality: count missing values, duplicates, impossible values and sudden shifts in distributions.
4. Distributions: look at histograms and percentiles for numbers, and frequency tables for categories.
5. Relationships: plot key variables against each other and against time, split by important segments.
6. Reconcile: compare totals with a trusted source, such as finance reports.

Write down findings and open questions as you go. The most valuable output of exploration is often a list of data problems and a sharper question for the real analysis.

The reason data is missing decides which fixes are safe. You cannot prove the mechanism from the data alone, but you can gather evidence by comparing rows with and without the missing value.

How to gather evidence:

• Compare the observed characteristics of rows with and without the value; big differences rule out completely random missingness.
• Check how and when the data is collected, since logging rules usually explain missingness better than statistics.
• Track whether missingness changed over time, which often signals a pipeline change.

Good practice: report how much data is missing, add a missing indicator when modeling, and show how conclusions change under different assumptions.