Estimating Prediction Qualities without Ground Truth: A Revisit of the Reverse Testing Framework
| Autor: | Dheeraj Bhaskaruni, Chao Lan, Fiona Patricia Moss |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Ground truth
business.industry Computer science Sample (statistics) 02 engineering and technology Machine learning computer.software_genre 030218 nuclear medicine & medical imaging 03 medical and health sciences 0302 clinical medicine 020204 information systems 0202 electrical engineering electronic engineering information engineering Task analysis Quality (philosophy) Anomaly detection Artificial intelligence business computer |
| Zdroj: | ICPR |
| DOI: | 10.1109/icpr.2018.8545706 |
| Popis: | To evaluate prediction qualities of machine learning models, it is typically assumed testing samples are labeled. However, testing labels are not always available in practice. A traditional solution is to approximate prediction qualities on testing samples by the qualities on labeled training samples. But this may be limited in that it completely ignores testing samples. In this paper, we present a new approach to estimate prediction qualities on unlabeled testing sample, based on the reverse testing framework [1]. We evaluate the approach with various quality metrics in classification and anomaly detection tasks, and over numerous real-world data sets. Experimental results show the proposed approach gives a more accurate estimate of prediction qualities on testing sample than those on training samples. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|