Why good models fail validation (and what to do about it)
Most models that stall in validation are not badly built. They are badly evidenced. Here is where the gaps actually come from.
Here is a pattern anyone who has worked in model risk will recognize. A data science team spends four months building a model. It performs well. The business is excited. Then it goes to validation — and sits there for six months, sometimes longer, while validators and developers trade documents that neither side finds satisfying.
The instinct is to blame the validators for being slow or the developers for being sloppy. Usually neither is true. The model is fine. The problem is that the evidence a validator needs was never produced, because nobody treated evidence as a deliverable.
The three gaps that stall reviews
First: data lineage. A validator asks a simple question — where did this training data come from, and what happened to it along the way? If the answer involves someone re-running notebooks to remember, the review has already stalled. Lineage has to be captured while the work happens. Reconstructing it later is expensive, and reconstructed lineage is inherently less credible than recorded lineage.
Second: decision history. Validators are not only assessing the model; they are assessing the judgment behind it. Why this feature set? Why was that segment excluded? Why this threshold? When these decisions are documented at the moment they are made — even in two sentences — the review moves. When the answers are "we tried a lot of things and this worked," it does not.
Third: fairness methodology. Teams often run disparate impact analysis after the model is built, find results they can live with, and write the methodology to match. Validators can tell. A fairness testing plan written before results exist, with pass criteria committed in advance, is worth more than any number of favorable post-hoc analyses.
Evidence as a by-product, not a project
The fix is not more documentation effort at the end — it is restructuring development so evidence accumulates as the work progresses. Versioned data and seeded training runs make reproducibility a property of the pipeline rather than a promise. Lightweight decision logs and pre-agreed validation plans can prevent avoidable disputes later.
This approach may add work before the first candidate model is ready, but it reduces the amount of history teams have to reconstruct during review. In regulated settings, proving how a model was built is part of shipping it.