Estimation Theory

Read these first

1. 01Consistency of Estimators

   An estimator is consistent if piling on more data drives it onto the true value and keeps it there. It is the minimum thing you should demand of a formula before you trust it, and it says nothing at all about whether the sample you actually have is big enough.

2. 02Generalized Method of Moments

   An estimation recipe for when you know some averages your model must reproduce but not the full distribution of the data. Write down the conditions, force the sample versions as close to zero as possible, and weight the reliable conditions more heavily.

3. 03The Delta Method

   A one-line rule for the error bar on a transformed estimate: multiply the original standard error by how steeply the transformation is rising at that point. It is how you get a standard error for a volatility, a ratio, or a Sharpe ratio without simulating anything.

4. 04The EM Algorithm

   A way out of the chicken-and-egg problem in models with hidden labels or missing values. Guess the fit, use it to fill in what you cannot see, refit on the filled-in data, and repeat until nothing moves.

Then the rest

1. 05Huber Robust Estimation
2. 06Influence Functions and the Breakdown Point
3. 07Peaks Over Threshold and the Generalized Pareto Distribution
4. 08The Hill Estimator for the Tail Index
5. 09Asymptotic Normality of Estimators
6. 10Exponential Family Distributions
7. 11Identifiability of a Model
8. 12M-Estimation
9. 13Modes of Convergence: In Probability and In Distribution
10. 14Quasi-Maximum Likelihood
11. 15Statistical Decision Theory and Loss Functions
12. 16The Jackknife
13. 17The James-Stein Estimator
14. 18The Rao-Blackwell Theorem
15. 19The Score Function and Likelihood Equations