A UI redesign's daily-active-user lift keeps shrinking

Question

A productivity app rolls out a bold visual redesign to a random half of its users and tracks daily active users (DAU). Treatment is up 12% the first week, 7% the second, 3% the third, and about 1% the fourth.

What is the most likely explanation for this shrinking lift, and how would you check it?

Accepted Answer

The novelty effect. A striking new look pulls people in to poke around and see what changed. That extra activity is driven by curiosity about the newness, not by the redesign being genuinely more useful day to day. As the look stops feeling new, usage settles back toward normal, so the DAU lift decays from 12% toward roughly zero, the pattern seen here. How to confirm it. 1. Extend the test. If the lift continues to fall and levels off near zero, that is novelty fading, not a lasting improvement. 2. Compare new users (who never saw the old design) with existing users (who did). A spike concen…

A UI redesign's daily-active-user lift keeps shrinking

Your answer

More Statistics questions