Dataset Condensation with Latent Quantile Matching

Autor:	Wei, Wei, De Schepper, Tom, Mets, Kevin
Rok vydání:	2024
Předmět:	Computer Science - Machine Learning Computer Science - Artificial Intelligence Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	Dataset condensation (DC) methods aim to learn a smaller synthesized dataset with informative data records to accelerate the training of machine learning models. Current distribution matching (DM) based DC methods learn a synthesized dataset by matching the mean of the latent embeddings between the synthetic and the real dataset. However two distributions with the same mean can still be vastly different. In this work we demonstrate the shortcomings of using Maximum Mean Discrepancy to match latent distributions i.e. the weak matching power and lack of outlier regularization. To alleviate these shortcomings we propose our new method: Latent Quantile Matching (LQM) which matches the quantiles of the latent embeddings to minimize the goodness of fit test statistic between two distributions. Empirical experiments on both image and graph-structured datasets show that LQM matches or outperforms previous state of the art in distribution matching based DC. Moreover we show that LQM improves the performance in continual graph learning (CGL) setting where memory efficiency and privacy can be important. Our work sheds light on the application of DM based DC for CGL. Comment: Accepted by CVPR Workshop 2024: 1st Workshop on Dataset Distillation for Computer Vision
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2406.09860 Zobrazit plný text záznamu View this record from Arxiv