Qm
Statistics

New users versus veterans is not a fair diff-in-diff

difference-in-differencesparallel-trendscohort-effects

A product analyst measures a new feature with difference-in-differences. The treated group is a cohort of brand-new users who received the feature; the control is a cohort of long-tenured users who did not. Engagement rose more in the treated cohort, and the analyst reports a positive feature effect.

Explain why comparing new users to veterans breaks parallel trends, and how you would redesign the comparison.

Your answer

Solving needs a free account

Answers, streaks and solutions unlock when you are signed in. Reading the question and the hint stays free.

Create a free accountSign in

Discussion

Sign in to join the discussion · reading is open to everyone

💡 Discussion rules

  1. No full solutions here. Hints and approaches only.
  2. Complexity, edge cases and intuition are the point.
  3. Interview experiences are welcome. Respect your NDAs.

Loading discussion…

Learn the concepts

The theory behind this question.

Difference-In-Differences In Microstructure ResearchEvent Study DesignsParallel Trends and Pre-Trend TestsStaggered Adoption and Two-Way Fixed Effects BiasThe Will Rogers Phenomenon

Related questions

The hidden assumption that makes diff-in-diff workA new subway opened the same year as the fare changePeople join the program right after a bad yearThe treated store was in a fast-growing suburbEstablishing causation: minimum wage and employment
All questions →