An Empirical Comparison of Bias Reduction Methods on Real-World Problems in High-Stakes Policy Settings
| Autor: | Kit T. Rodolfa, Hemank Lamba, Rayid Ghani |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Empirical comparison Social work Computer science business.industry Geography Planning and Development Public policy 02 engineering and technology Data science Pipeline (software) Bias reduction Machine Learning (cs.LG) Computer Science - Computers and Society Policy decision 020204 information systems Computers and Society (cs.CY) Health care 0202 electrical engineering electronic engineering information engineering General Earth and Planetary Sciences 020201 artificial intelligence & image processing business Water Science and Technology Criminal justice |
| Zdroj: | ACM SIGKDD Explorations Newsletter. 23:69-85 |
| ISSN: | 1931-0153 1931-0145 |
| Popis: | Applications of machine learning (ML) to high-stakes policy settings -- such as education, criminal justice, healthcare, and social service delivery -- have grown rapidly in recent years, sparking important conversations about how to ensure fair outcomes from these systems. The machine learning research community has responded to this challenge with a wide array of proposed fairness-enhancing strategies for ML models, but despite the large number of methods that have been developed, little empirical work exists evaluating these methods in real-world settings. Here, we seek to fill this research gap by investigating the performance of several methods that operate at different points in the ML pipeline across four real-world public policy and social good problems. Across these problems, we find a wide degree of variability and inconsistency in the ability of many of these methods to improve model fairness, but post-processing by choosing group-specific score thresholds consistently removes disparities, with important implications for both the ML research community and practitioners deploying machine learning to inform consequential policy decisions. 17 pages, 9 figures, 2 tables |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|