Investigating Causes of Model Instability: Properties of the Prediction Accuracy Index

Autor: Ross Taplin
Jazyk: angličtina
Rok vydání: 2023
Předmět:
Zdroj: Risks, Vol 11, Iss 6, p 110 (2023)
Druh dokumentu: article
ISSN: 2227-9091
DOI: 10.3390/risks11060110
Popis: The Prediction Accuracy Index (PAI) monitors stability, defined as whether the predictive power of a model has deteriorated due to a change in the distribution of the explanatory variables since its development. This paper shows how the PAI is related to the Mahalanobis distance, an established statistic for examining high leverage observations in data. This relationship is used to explore properties of the PAI, including tools for how the PAI can be decomposed into effects due to (a) individual observations/cases, (b) individual variables, and (c) shifts in the mean of variables. Not only are these tools useful for practitioners to help determine why models fail stability, but they also have implications for auditors and regulators. In particular, reasons why models containing econometric variables may fail stability are explored and suggestions to improve model development are discussed.
Databáze: Directory of Open Access Journals
Nepřihlášeným uživatelům se plný text nezobrazuje