Why randomize? Onboarding tutorial and client trading

Question

Observational data show that clients who completed your brokerage's optional onboarding tutorial go on to trade more than clients who skipped it. Leadership wants to make the tutorial mandatory to boost activity.

Explain why this correlation cannot establish that the tutorial causes more trading, and design a test that can. Why is randomization the key ingredient?

Accepted Answer

Why the correlation is not enough. The tutorial was optional, so who completed it was not random. Motivated, engaged clients are exactly the ones who both finish optional tutorials and trade a lot anyway. Client engagement is a confounder: it drives tutorial completion and trading at the same time, so the two look linked even if the tutorial adds nothing. There may also be reverse selection (people already planning to trade actively bother with the tutorial). Making it mandatory forces it on the unmotivated clients, for whom it may do little, so the observed gap could evaporate. A test that c…

Why randomize? Onboarding tutorial and client trading

Your answer

More Statistics questions