The ad channel that converts better on every device

Question

A marketing team compares two ad channels by conversion rate, split by the device visitors used.

Mobile: Channel A converts 60 of 1000 (6%); Channel B converts 25 of 500 (5%).
Desktop: Channel A converts 90 of 500 (18%); Channel B converts 160 of 1000 (16%).

Overall, Channel A converts 150 of 1500 (10%) and Channel B converts 185 of 1500 (about 12%), so B looks better in total.

Which channel actually converts better, and why is the overall rate misleading?

Accepted Answer

Channel A converts better. It beats B on mobile (6% versus 5%) and on desktop (18% versus 16%). On both device types, A wins. The pooled rate reverses this, which is Simpson's paradox, and the culprit is a confounder. The confounder is device mix. Desktop visitors convert far better than mobile visitors for both channels (mobile is single digits, desktop is mid-teens). But the two channels reached very different device mixes. Channel A's traffic was mostly mobile (1000 of its 1500 visits), the harder-converting device, while Channel B's traffic was mostly desktop (1000 of its 1500 visits), th…

The ad channel that converts better on every device

Your answer

Discussion

Learn the concepts

Related questions