Relative confidence sampling for efficient on-line ranker evaluation
| Autor: | Rémi Munos, Masrour Zoghi, Shimon Whiteson, Maarten de Rijke |
|---|---|
| Přispěvatelé: | Information and Language Processing Syst (IVI, FNWI), Amsterdam Machine Learning lab (IVI, FNWI) |
| Jazyk: | angličtina |
| Rok vydání: | 2014 |
| Předmět: |
Basis (linear algebra)
business.industry Computer science InformationSystems_INFORMATIONSTORAGEANDRETRIEVAL Rankers Sampling (statistics) Regret Machine learning computer.software_genre Line (geometry) Key (cryptography) Learning to rank Artificial intelligence Data mining business Finite set computer |
| Zdroj: | WSDM '14: proceedings of the 7th ACM International Conference on Web Search and Data Mining: February 24-28, 2014, New York, New York, USA, 73-82 STARTPAGE=73;ENDPAGE=82;TITLE=WSDM '14: proceedings of the 7th ACM International Conference on Web Search and Data Mining: February 24-28, 2014, New York, New York, USA WSDM |
| Popis: | A key challenge in information retrieval is that of on-line ranker evaluation: determining which one of a finite set of rankers performs the best in expectation on the basis of user clicks on presented document lists. When the presented lists are constructed using interleaved comparison methods, which interleave lists proposed by two different candidate rankers, then the problem of minimizing the total regret accumulated while evaluating the rankers can be formalized as a K-armed dueling bandits problem. In this paper, we propose a new method called relative confidence sampling (RCS) that aims to reduce cumulative regret by being less conservative than existing methods in eliminating rankers from contention. In addition, we present an empirical comparison between RCS and two state-of-the-art methods, relative upper confidence bound and SAVAGE. The results demonstrate that RCS can substantially outperform these alternatives on several large learning to rank datasets. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|