Target Encoding for Categorical Features

One-hot encoding a categorical feature with hundreds of distinct values — a ZIP code, a stock ticker, a merchant ID — produces hundreds of new sparse columns, which slows training and can overwhelm the actual signal. Target encoding takes a different approach: replace each category with a single number, the average value of the target variable among rows sharing that category in the training data.

$$\text{encode}(c) = \frac{n_c \cdot \bar{y}_c + m \cdot \bar{y}}{n_c + m}$$

In words: the category's own average outcome, blended toward the overall (global) average by a smoothing weight $m$ — categories with lots of observations ($n_c$ large) rely mostly on their own average, while rare categories lean more on the global average, which prevents a category seen only twice from getting an extreme, unreliable encoding.

Worked example. A "merchant" feature has a category seen in only 3 training rows, all fraudulent (target average 1.0). Raw target encoding would assign that merchant a 1.0 — an overconfident signal from 3 data points. With smoothing weight $m=20$ and a global fraud rate of 0.02: $\text{encode} = \frac{3(1.0) + 20(0.02)}{3+20} = \frac{3.4}{23} \approx 0.148$ — pulled far back toward the base rate, reflecting how little evidence 3 observations really provide.