Treated advertisers bid more, raising the control's prices

Question

An ad platform randomly gives some advertisers a smart bidding tool and keeps others as controls, all competing in the same real-time ad auctions. Treated advertisers win more impressions at good prices; control advertisers win fewer at higher prices. The platform reports a big win for the tool.

Explain why this A/B comparison overstates the tool's value, and how you would run the experiment.

Accepted Answer

They all bid in the same auctions. Ad slots are sold in shared auctions, so what one advertiser bids affects the price and outcome for everyone else bidding on the same slots. When treated advertisers use the smart tool to bid more cleverly and aggressively, they win impressions that control advertisers would have won, and they push up the prices control advertisers must pay. The controls are worse off partly because the treatment reshaped the auction against them, not purely on their own merits. This is interference through a shared market mechanism. Why it overstates the tool. The treated-m…

Treated advertisers bid more, raising the control's prices

Your answer

Discussion

Learn the concepts

Related questions