How far can you compress data without losing a parameter

Question

Sufficiency lets you replace a large data set with a small summary without losing information about a parameter.

Explain how far this compression can go for the Poisson, the exponential, and the two-parameter normal, and why you generally cannot compress below the number of parameters.

Accepted Answer

Sufficiency is lossless compression. A sufficient statistic keeps every drop of information the data hold about the parameter while discarding the rest. So the question "how small can the summary be" is really "what is the most efficient lossless code for the parameter." One parameter, one number. For a Poisson rate, the total \sumi Xi is sufficient: a thousand minute-by-minute counts collapse to a single number with no loss about \lambda. Same story for the exponential rate (the total waiting time) and the Bernoulli probability (the count of successes). One unknown, one sufficient number. Tw…

How far can you compress data without losing a parameter

Your answer

Discussion

Learn the concepts

Related questions