AUC

AUC, the area under the receiver operating characteristic curve, measures how well a model separates cases where an event occurs from cases where it does not, independent of any threshold. It describes ranking ability, not whether stated probabilities are right.

Read as a probability, it is the chance that the model assigns a higher score to a randomly chosen positive case than to a randomly chosen negative one. It is useful for comparing models and for detecting a model with no discrimination.

It says nothing about calibration, so a high AUC alone does not support pricing. It is also sensitive to the mix of the evaluation population, so it is reported with the population and the period.

An unusually high AUC is treated as a warning. It more often indicates that information about the outcome leaked into the inputs than that the model is exceptional.