Acceptance Test

An acceptance test is a specific, defined criterion, agreed in advance rather than improvised after a result is already known, that a model, a dataset, or a system component has to satisfy before it is accepted into production use. Acceptance tests convert a judgment call, is this model good enough to price capital against, into a checkable pass-or-fail criterion set before anyone has seen how the candidate model actually performs, which is what prevents the threshold for acceptance from being quietly relaxed to accommodate a result the institution wants to approve. A well-designed acceptance test for an outcome-probability model should specify the minimum sample size behind a calibration claim, the required out-of-sample and out-of-time validation structure, and the specific reliability curve shape the model has to demonstrate at the segment level relevant to its intended use, not only in aggregate. An institution's diligence process, whether internal or conducted by a counterparty, should be able to point to the specific acceptance test a production model passed, and the date it passed it, as a documented artifact, rather than accepting a general assurance that the model met the institution's standards without a specific, checkable test behind that assurance, since the assurance alone tells a reviewer nothing about what threshold was actually cleared.