Beyond NDCG: behavioral testing of recommender systems with RecList

Autor:	Chia, Patrick John, Tagliabue, Jacopo, Bianchi, Federico, He, Chloe, Ko, Brian
Rok vydání:	2021
Předmět:	Computer Science - Information Retrieval Computer Science - Artificial Intelligence Computer Science - Machine Learning
Druh dokumentu:	Working Paper
DOI:	10.1145/3487553.3524215
Popis:	As with most Machine Learning systems, recommender systems are typically evaluated through performance metrics computed over held-out data points. However, real-world behavior is undoubtedly nuanced: ad hoc error analysis and deployment-specific tests must be employed to ensure the desired quality in actual deployments. In this paper, we propose RecList, a behavioral-based testing methodology. RecList organizes recommender systems by use case and introduces a general plug-and-play procedure to scale up behavioral testing. We demonstrate its capabilities by analyzing known algorithms and black-box commercial systems, and we release RecList as an open source, extensible package for the community. Comment: Paper accepted to the WebConf 2022
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2111.09963 Zobrazit plný text záznamu View this record from Arxiv