An Enhanced Evaluation Framework for Query Performance Prediction
| Autor: | Falk Scholer, J. Shane Culpepper, Oleg Zendel, Guglielmo Faggioli, Nicola Ferro |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Measure (data warehouse)
Computer science 05 social sciences 02 engineering and technology computer.software_genre Multiple factors 020204 information systems 0202 electrical engineering electronic engineering information engineering Performance prediction Key (cryptography) Statistical analysis Point estimation Data mining 0509 other social sciences 050904 information & library sciences computer |
| Zdroj: | Lecture Notes in Computer Science ISBN: 9783030721121 ECIR (1) |
| DOI: | 10.1007/978-3-030-72113-8_8 |
| Popis: | Query Performance Prediction (QPP) has been studied extensively in the IR community over the last two decades. A by-product of this research is a methodology to evaluate the effectiveness of QPP techniques. In this paper, we re-examine the existing evaluation methodology commonly used for QPP, and propose a new approach. Our key idea is to model QPP performance as a distribution instead of relying on point estimates. Our work demonstrates important statistical implications, and overcomes key limitations imposed by the currently used correlation-based point-estimate evaluation approaches. We also explore the potential benefits of using multiple query formulations and ANalysis Of VAriance (ANOVA) modeling in order to measure interactions between multiple factors. The resulting statistical analysis combined with a novel evaluation framework demonstrates the merits of modeling QPP performance as distributions, and enables detailed statistical ANOVA models for comparative analyses to be created. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|