Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluation

Autor:	Jesus, Sérgio, Pombal, José, Alves, Duarte, Cruz, André, Saleiro, Pedro, Ribeiro, Rita P., Gama, João, Bizarro, Pedro
Rok vydání:	2022
Předmět:	Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Evaluating new techniques on realistic datasets plays a crucial role in the development of ML research and its broader adoption by practitioners. In recent years, there has been a significant increase of publicly available unstructured data resources for computer vision and NLP tasks. However, tabular data -- which is prevalent in many high-stakes domains -- has been lagging behind. To bridge this gap, we present Bank Account Fraud (BAF), the first publicly available privacy-preserving, large-scale, realistic suite of tabular datasets. The suite was generated by applying state-of-the-art tabular data generation techniques on an anonymized,real-world bank account opening fraud detection dataset. This setting carries a set of challenges that are commonplace in real-world applications, including temporal dynamics and significant class imbalance. Additionally, to allow practitioners to stress test both performance and fairness of ML methods, each dataset variant of BAF contains specific types of data bias. With this resource, we aim to provide the research community with a more realistic, complete, and robust test bed to evaluate novel and existing methods. Comment: Accepted at NeurIPS 2022. https://openreview.net/forum?id=UrAYT2QwOX8
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2211.13358 Zobrazit plný text záznamu View this record from Arxiv