Weight of Evidence and Information Value

Feeding a categorical feature into a linear model — say, a borrower's employment type, or a loan's region — usually means one-hot encoding it into a set of 0/1 columns, one per category. This works, but treats every category as an unrelated dummy variable and can produce dozens of extra columns for a high-cardinality feature. Weight of evidence (WOE) is an older, still widely used credit-scoring alternative that converts each category into a single number reflecting how strongly it's associated with the outcome.

Weight of evidence, in words

For each category (or bin, if the feature is continuous and has been bucketed) of a feature, weight of evidence is the log of the ratio between the proportion of "good" outcomes in that category and the proportion of "bad" outcomes in that category, each measured relative to the totals across the whole dataset:

$$WOE_i = \ln\!\left(\frac{\%\text{ good}_i}{\%\text{ bad}_i}\right) .$$

In plain English: WOE asks, for category $i$, "does this category contain more than its fair share of good outcomes, or of bad outcomes, relative to the overall base rates?" A WOE of 0 means the category is exactly average. A large positive WOE means the category is disproportionately associated with good outcomes; a large negative WOE means disproportionately bad. Because WOE is a difference of log-odds, replacing a categorical feature with its WOE values makes it behave almost like a natural input to a logistic regression, whose own output is built from log-odds.

Worked example: employment type and loan default

Suppose 1,000 loans split by employment type show: salaried borrowers have 400 good / 40 bad; self-employed have 300 good / 90 bad; retired have 150 good / 20 bad. Overall totals are 850 good, 150 bad. For salaried: %good = 47.1%, %bad = 26.7%, so $WOE_{\text{salaried}} = \ln(47.1/26.7) \approx 0.57$ — over-represented among good outcomes. For self-employed: %good = 35.3%, %bad = 60.0%, so $WOE_{\text{self-employed}} \approx -0.53$ — disproportionately associated with defaults. These numbers turn a 3-level categorical feature into a single numeric column that already encodes its risk ordering.

Information value: how predictive is the whole feature?

Information value (IV) rolls up the WOE calculation across all categories of a feature into a single summary number, weighting each category's WOE by the difference between its good and bad proportions:

$$IV = \sum_i (\%\text{good}_i - \%\text{bad}_i) \times WOE_i .$$

In plain English: IV rewards categories that are both strongly skewed toward good-or-bad and have a large WOE in that direction — a category that's 90% one way but tiny in size contributes less than a similarly skewed category that represents a large share of the data. A widely used (if informal) rule of thumb in credit scoring treats $IV < 0.02$ as essentially useless, $0.1$–$0.3$ as moderately predictive, and $> 0.3$ as strongly predictive — a quick, standardized way to rank candidate features before deciding which to include in a scorecard model.

WOE and IV are built entirely from good/bad counts within categories — the same binomial-style good/bad split shown in the explorer above, just computed separately for each bin of a feature and then compared across bins.

What this means in practice

WOE encoding and IV screening remain standard in regulated credit-scoring environments because they produce monotonic, log-odds-scale features that plug directly into an interpretable logistic regression scorecard, and IV gives a fast, standardized way to rank many candidate features before modeling. Outside credit scoring, the same idea shows up more generally as a form of target encoding, and carries the same core risk as any target-derived encoding: it must be computed only on training data (using cross-validation folds for the training set itself) to avoid leaking target information into the encoded feature.