Quant Memo
Statistics/●●●●

An SRM that shows up in just one country

An A/B test passes the overall SRM check, so at first glance the 50/50 split looks healthy. But when the team slices by country, one large market has a severe SRM (treatment there got far fewer users than control), while other markets look fine. That one market happens to have very different conversion behavior from the rest. The overall result shows treatment up +1.5%.

Explain why a segment-level SRM matters even when the overall check passes, and what the team should do.

Your answer

This one is open-ended. Work it through, then check your reasoning against the full solution.

More Statistics questions