A recommendation carousel that spikes then settles

Question

A streaming service adds a new "For You" recommendation carousel for a random half of users and measures clicks on recommended titles. Clicks are way up in the first few days, then the advantage over control shrinks each week and nearly disappears after a month.

What effect most likely explains this, and how would you confirm it?

Accepted Answer

The novelty effect. A new row on the home screen draws the eye and invites exploration simply because it is there and it is new. People click to see what it does. As it becomes a familiar part of the layout, they engage with it at a normal rate, so the early click surge fades toward the control level, the pattern described here. How to confirm it. 1. Keep the experiment running. Novelty shows up as a lift that decays and then flattens near zero; a durable feature holds its lift. 2. Split by tenure: compare users who signed up after the carousel launched (who never knew the old layout) with ex…

A recommendation carousel that spikes then settles

Your answer

More Statistics questions