Why ranking three signals by p-value misleads

Question

A researcher ranks three signals by p-value, C ( $p = 0.002$ ) best, D ( $p = 0.01$ ) second, E ( $p = 0.04$ ) worst, and proposes to allocate capital in that order of "strength."

Explain why ranking signals by p-value is not the same as ranking them by the size of their edge, and give a numeric example where the true edges run in the opposite order.

Accepted Answer

The decomposition. Each signal's z-statistic (and hence its p-value) is its edge divided by its standard error: z = 	ext{edge}/	ext{SE}. Two signals with the same edge but different sample sizes or volatilities will have different z-values, and two signals with different edges can share a z-value. So p-value order reflects a mix of edge and precision, not edge alone. A reversing example. Suppose: - Signal C: edge = 0.8 bp, SE = 0.26 bp, so z \approx 3.1, p \approx 0.002. - Signal D: edge = 1.5 bp, SE = 0.58 bp, so z \approx 2.6, p \approx 0.01. - Signal E: edge = 3.0 bp, SE = 1.46 bp, so z …

Why ranking three signals by p-value misleads

Your answer

Discussion

Learn the concepts

Related questions