Topic Difficulty: Collection and Query Formulation Effects
| Autor: | FerroNicola, KurlandOren, CulpepperJ. Shane, FaggioliGuglielmo |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
retrieval effectiveness
Information retrieval Topic difficulty Computer science effect size query formulation 05 social sciences Query formulation 02 engineering and technology General Business Management and Accounting Computer Science Applications 020204 information systems 0202 electrical engineering electronic engineering information engineering 0509 other social sciences 050904 information & library sciences Information Systems |
| Zdroj: | ACM Transactions on Information Systems. 40:1-36 |
| ISSN: | 1558-2868 1046-8188 |
| Popis: | Several recent studies have explored the interaction effects between topics, systems, corpora, and components when measuring retrieval effectiveness. However, all of these previous studies assume that a topic or information need is represented by a single query. In reality, users routinely reformulate queries to satisfy an information need. In recent years, there has been renewed interest in the notion of “query variations” which are essentially multiple user formulations for an information need. Like many retrieval models, some queries are highly effective while others are not. This is often an artifact of the collection being searched which might be more or less sensitive to word choice. Users rarely have perfect knowledge about the underlying collection, and so finding queries that work is often a trial-and-error process. In this work, we explore the fundamental problem of system interaction effects between collections, ranking models, and queries. To answer this important question, we formalize the analysis using ANalysis Of VAriance (ANOVA) models to measure multiple components effects across collections and topics by nesting multiple query variations within each topic. Our findings show that query formulations have a comparable effect size of the topic factor itself, which is known to be the factor with the greatest effect size in prior ANOVA studies. Both topic and formulation have a substantially larger effect size than any other factor, including the ranking algorithms and, surprisingly, even query expansion. This finding reinforces the importance of further research in understanding the role of query rewriting in IR related tasks. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|