Zobrazeno 1 - 10
of 142
pro vyhledávání: '"Soboroff, Ian"'
Autor:
Soboroff, Ian
Making the relevance judgments for a TREC-style test collection can be complex and expensive. A typical TREC track usually involves a team of six contractors working for 2-4 weeks. Those contractors need to be trained and monitored. Software has to b
Externí odkaz:
http://arxiv.org/abs/2409.15133
Autor:
Kandula, Hemanth, Karakos, Damianos, Qiu, Haoling, Rozonoyer, Benjamin, Soboroff, Ian, Tarlin, Lee, Min, Bonan
Frequently, users of an Information Retrieval (IR) system start with an overarching information need (a.k.a., an analytic task) and proceed to define finer-grained queries covering various important aspects (i.e., sub-topics) of that analytic task. W
Externí odkaz:
http://arxiv.org/abs/2409.04667
Autor:
Banerjee, Somnath, Halder, Avik, Mandal, Rajarshi, Layek, Sayan, Soboroff, Ian, Hazra, Rima, Mukherjee, Animesh
The integration of pretrained language models (PLMs) like BERT and GPT has revolutionized NLP, particularly for English, but it has also created linguistic imbalances. This paper strategically identifies the need for linguistic equity by examining se
Externí odkaz:
http://arxiv.org/abs/2406.11139
Autor:
Mayfield, James, Yang, Eugene, Lawrie, Dawn, MacAvaney, Sean, McNamee, Paul, Oard, Douglas W., Soldaini, Luca, Soboroff, Ian, Weller, Orion, Kayi, Efsun, Sanders, Kate, Mason, Marc, Hibbler, Noah
Large Language Models (LLMs) have enabled new ways to satisfy information needs. Although great strides have been made in applying them to settings like document ranking and short-form text generation, they still struggle to compose complete, accurat
Externí odkaz:
http://arxiv.org/abs/2405.00982
Autor:
Sakai, Tetsuya, Tao, Sijie, Maistro, Maria, Chu, Zhumin, Li, Yujing, Chen, Nuo, Ferro, Nicola, Wang, Junjie, Soboroff, Ian, Liu, Yiqun
Unfortunately, the official English (sub)task results reported in the NTCIR-14 WWW-2, NTCIR-15 WWW-3, and NTCIR-16 WWW-4 overview papers are incorrect due to noise in the official qrels files; this paper reports results based on the corrected qrels f
Externí odkaz:
http://arxiv.org/abs/2210.10266
Neural retrieval models are generally regarded as fundamentally different from the retrieval techniques used in the late 1990's when the TREC ad hoc test collections were constructed. They thus provide the opportunity to empirically test the claim th
Externí odkaz:
http://arxiv.org/abs/2201.11086
Autor:
Carterette, Ben, Jones, Rosie, Jones, Gareth F., Eskevich, Maria, Reddy, Sravana, Clifton, Ann, Yu, Yongze, Karlgren, Jussi, Soboroff, Ian
Rapidly growing online podcast archives contain diverse content on a wide range of topics. These archives form an important resource for entertainment and professional use, but their value can only be realized if users can rapidly and reliably locate
Externí odkaz:
http://arxiv.org/abs/2108.11460
Autor:
Roberts, Kirk, Alam, Tasmeer, Bedrick, Steven, Demner-Fushman, Dina, Lo, Kyle, Soboroff, Ian, Voorhees, Ellen, Wang, Lucy Lu, Hersh, William R
We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19. The goals of TREC-COVID include the construction of a pandemic search test collection and t
Externí odkaz:
http://arxiv.org/abs/2104.09632
Autor:
Craswell, Nick, Mitra, Bhaskar, Yilmaz, Emine, Campos, Daniel, Voorhees, Ellen M., Soboroff, Ian
The TREC Deep Learning (DL) Track studies ad hoc search in the large data regime, meaning that a large set of human-labeled training data is available. Results so far indicate that the best models with large data may be deep neural networks. This pap
Externí odkaz:
http://arxiv.org/abs/2104.09399
Autor:
Breuer, Timo, Ferro, Nicola, Fuhr, Norbert, Maistro, Maria, Sakai, Tetsuya, Schaer, Philipp, Soboroff, Ian
Replicability and reproducibility of experimental results are primary concerns in all the areas of science and IR is not an exception. Besides the problem of moving the field towards more reproducible experimental practices and protocols, we also fac
Externí odkaz:
http://arxiv.org/abs/2010.13447