Concept Bottleneck Models

A standard neural network maps raw inputs straight to a prediction, and the intermediate layers that get it there are just numbers with no inherent meaning. A concept bottleneck model breaks that pipeline into two explicit stages: first predict a set of human-understandable concepts from the input (for a credit application, things like "high debt-to-income," "recent late payment," "long employment history"), then predict the final outcome using only those named concepts as inputs. The narrow, labeled layer in the middle is the "bottleneck."

The payoff is twofold. First, a human can read off exactly which concepts drove a given decision, rather than guessing at what a hidden layer represents. Second, and more unusually, a person can intervene: if the concept layer says "high debt-to-income" but a reviewer knows that's wrong, they can manually flip it and watch the final prediction update accordingly, which is not possible with an ordinary end-to-end model. The tradeoff is that building one requires concept labels — someone has to annotate training examples with those intermediate concept values — and if the chosen concepts don't fully capture what drives the outcome, accuracy can lag a plain black-box model trained on the same data.