Qm

A clears the bar, B just misses, are they different?

Strategy A has an estimated edge of 3.03.0 bp with standard error 1.01.0 bp (zA=3.0z_A = 3.0, p0.003p \approx 0.003, significant). Strategy B has an estimated edge of 1.91.9 bp with standard error 1.01.0 bp (zB=1.9z_B = 1.9, p0.057p \approx 0.057, just not significant). Someone argues A is clearly the better strategy because it is significant and B is not.

Test whether A's edge significantly exceeds B's, and explain why "one passed, one failed" is not evidence of a difference.

Your answer

Solving needs a free account

Answers, streaks and solutions unlock when you are signed in. Reading the question and the hint stays free.

Discussion

Sign in to join the discussion · reading is open to everyone

💡 Discussion rules

  1. No full solutions here. Hints and approaches only.
  2. Complexity, edge cases and intuition are the point.
  3. Interview experiences are welcome. Respect your NDAs.

Loading discussion…

Learn the concepts

The theory behind this question.

Related questions

One significant, one not, so A beats B?When the difference actually is significantWhat a p-value actually isReading significance straight off a confidence intervalWhen even the wide interval straddles zero
All questions →