Out-of-Sample Validation

Out-of-sample validation means evaluating a model's performance against data that played no role in training that model, a basic requirement without which a performance claim measures the model's ability to remember its own training data rather than its ability to generalize to new, unseen matters. A model checked only against the data it was trained on will look calibrated and accurate almost by construction, because the model has, in effect, already seen the answer key for every question the check asks it. Out-of-sample validation alone is a necessary but not sufficient condition for a trustworthy performance claim, because it does not, by itself, address whether the held-out data came from the same time period as the training data, which introduces a related requirement, out-of-time validation, addressing a different way a check can still be circular. This platform treats out-of-sample validation as a baseline requirement for any calibration or accuracy claim, disclosed alongside the specific split methodology used to separate training data from the held-out population, so a counterparty can confirm the separation was genuine rather than assumed, and can distinguish a rigorous validation from one that only appears rigorous on the surface without actually protecting against a model that has simply memorized its own training set.