Positive-Congruent Training: Towards Regression-Free Model Updates
| Autor: | Wei Xia, Stefano Soatto, Shuo Yang, Sijie Yan, Siqi Deng, Yuanjun Xiong, Meng Wang, Kaustav Kundu |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Contextual image classification Computer science business.industry Computer Vision and Pattern Recognition (cs.CV) Deep learning Computer Science - Computer Vision and Pattern Recognition Word error rate Pattern recognition Regression Machine Learning (cs.LG) Congruence (geometry) Simple (abstract algebra) Pattern recognition (psychology) Artificial intelligence business Reference model |
| Zdroj: | CVPR |
| DOI: | 10.1109/cvpr46437.2021.01407 |
| Popis: | Reducing inconsistencies in the behavior of different versions of an AI system can be as important in practice as reducing its overall error. In image classification, sample-wise inconsistencies appear as "negative flips": A new model incorrectly predicts the output for a test sample that was correctly classified by the old (reference) model. Positive-congruent (PC) training aims at reducing error rate while at the same time reducing negative flips, thus maximizing congruency with the reference model only on positive predictions, unlike model distillation. We propose a simple approach for PC training, Focal Distillation, which enforces congruence with the reference model by giving more weights to samples that were correctly classified. We also found that, if the reference model itself can be chosen as an ensemble of multiple deep neural networks, negative flips can be further reduced without affecting the new model's accuracy. Accepted to CVPR 2021 (oral) |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|