Prediction Lineage and Auditability
The practice of recording exactly which model version, feature values, and code produced a given prediction, so that any past output can later be reproduced or explained.
When a model makes a prediction that later turns out to be wrong, or gets challenged by a regulator, someone needs to answer a very specific question: exactly what inputs, code, and model weights produced that particular number, at that particular moment? Prediction lineage is the recorded trail that answers this — a log linking each prediction back to the model version, the feature values used, and the pipeline code that ran, so the prediction can be reconstructed or explained after the fact.
Prediction lineage means logging enough detail — model version, exact feature values, code version — at prediction time that any past prediction can later be reproduced or audited, rather than trusting memory or approximate reconstruction.
What gets logged
A robust lineage system typically records, for every prediction: a model version identifier (which training run, which hyperparameters), the exact feature vector fed in (not just feature names, but the actual values, since a feature store can change retroactively), a timestamp, and a reference to the exact code commit that executed the pipeline. Without all four pieces, "reproducing" a prediction six months later can silently use different feature values or a patched pipeline, giving a different answer.
This matters most in regulated settings like credit and insurance, where a firm may need to explain a specific past decision years later, but it is good practice in any production ML system where debugging a model regression requires knowing exactly what changed.
Related concepts
Practice in interviews
Further reading
- Practitioner note: MLOps lineage and audit trails