Qm
Statistics

Simpson's paradox: which ad creative truly converts better?

simpsons-paradoxconfoundingconversion

You ran two ad creatives and measured conversion rate (conversions / impressions), split by device:

MobileDesktopOverall
Creative A100/1000 (10%)40/100 (40%)140/1100 (12.7%)
Creative B8/100 (8%)380/1000 (38%)388/1100 (35.3%)

Creative A converts better on mobile (10% vs 8%) and on desktop (40% vs 38%), yet B's overall rate is far higher (35.3% vs 12.7%).

Which creative should you serve, and why does the aggregate disagree with both segments?

Your answer

Solving needs a free account

Answers, streaks and solutions unlock when you are signed in. Reading the question and the hint stays free.

Create a free accountSign in

Discussion

Sign in to join the discussion · reading is open to everyone

💡 Discussion rules

  1. No full solutions here. Hints and approaches only.
  2. Complexity, edge cases and intuition are the point.
  3. Interview experiences are welcome. Respect your NDAs.

Loading discussion…

Learn the concepts

The theory behind this question.

Bad Controls and Post-Treatment BiasConfounding and the Backdoor CriterionConfounding in Financial DataDirected Acyclic Graphs for CausalityDouble Machine Learning

Related questions

The strategy that wins in every regime but loses overallThe batter with the higher average each year, but not overallThe ad channel that converts better on every deviceThe school that passes more students in every subjectDid the university admit men at a higher rate?
All questions →