The ad channel that converts better on every device

Question

A marketing team compares two ad channels by conversion rate, split by the device visitors used.

Mobile: Channel A converts 60 of 1000 (6%); Channel B converts 25 of 500 (5%).
Desktop: Channel A converts 90 of 500 (18%); Channel B converts 160 of 1000 (16%).

Overall, Channel A converts 150 of 1500 (10%) and Channel B converts 185 of 1500 (about 12%), so B looks better in total.

Which channel actually converts better, and why is the overall rate misleading?

Accepted Answer

Channel A converts better. It beats B on mobile (6% versus 5%) and on desktop (18% versus 16%). On both device types, A wins. The pooled rate reverses this, which is Simpson's paradox, and the culprit is a confounder. The confounder is device mix. Desktop visitors convert far better than mobile visitors for both channels (mobile is single digits, desktop is mid-teens). But the two channels reached very different device mixes. Channel A's traffic was mostly mobile (1000 of its 1500 visits), the harder-converting device, while Channel B's traffic was mostly desktop (1000 of its 1500 visits), th…

The ad channel that converts better on every device

Your answer

More Statistics questions