When resampling/reweighting improves feature learning in imbalanced classification?: A toy-model study

Autor:	Obuchi, Tomoyuki, Tanaka, Toshiyuki
Rok vydání:	2024
Předmět:	Statistics - Machine Learning Condensed Matter - Disordered Systems and Neural Networks Computer Science - Information Theory Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	A toy model of binary classification is studied with the aim of clarifying the class-wise resampling/reweighting effect on the feature learning performance under the presence of class imbalance. In the analysis, a high-dimensional limit of the feature is taken while keeping the dataset size ratio against the feature dimension finite and the non-rigorous replica method from statistical mechanics is employed. The result shows that there exists a case in which the no resampling/reweighting situation gives the best feature learning performance irrespectively of the choice of losses or classifiers, supporting recent findings in Cao et al. (2019); Kang et al. (2019). It is also revealed that the key of the result is the symmetry of the loss and the problem setting. Inspired by this, we propose a further simplified model exhibiting the same property for the multiclass setting. These clarify when the class-wise resampling/reweighting becomes effective in imbalanced classification. Comment: 30 pages, 14 figures
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2409.05598 Zobrazit plný text záznamu View this record from Arxiv