Confirmatory bias in peer review
Autor: | Rosa Rodriguez-Sánchez, Jose A. García, Joaquín Fdez-Valdivia |
---|---|
Rok vydání: | 2020 |
Předmět: |
Evaluation strategy
Information retrieval media_common.quotation_subject Information processing General Social Sciences Recommendation quality Library and Information Sciences Information overload Computer Science Applications Confirmation bias Chunking (psychology) ComputingMethodologies_DOCUMENTANDTEXTPROCESSING Psychology Cognitive load media_common |
Zdroj: | Scientometrics. 123:517-533 |
ISSN: | 1588-2861 0138-9130 |
DOI: | 10.1007/s11192-020-03357-0 |
Popis: | A reduction in reviewer’s recommendation quality may be caused by a limitation of time or cognitive overload that comes from the level of redundancy, contradiction and inconsistency in the research. Some adaptive mechanisms by reviewers who deal with situations of information overload may be chunking single pieces of manuscript information into generic terms, unsystematic omission of research details, queuing of information processing, and prematurely stop the manuscript evaluation. Then, how would a reviewer optimize attention to positive and negative attributes of a manuscript before making a recommendation? How a reviewer’s characteristics such as her prior belief about the manuscript quality and manuscript evaluation cost, affect her attention allocation and final recommendation? To answer these questions, we use a probabilistic model in which a reviewer chooses the optimal evaluation strategy by trading off the value and cost of review information about the manuscript quality. We find that a reviewer could exhibit a confirmatory behavior under which she pays more attention to the type of manuscript attributes that favor her prior belief about the manuscript quality. Then, confirmatory bias could be an optimal behavior of the reviewers that optimize attention to positive and negative manuscript attributes under information overload. We also show that reviewer’s manuscript evaluation cost plays a key role in determining whether she may exhibit confirmatory bias. Moreover, when the reviewer’s prior belief about the manuscript quality is low enough, the probability of obtaining a positive review signal decreases with reviewer’s manuscript evaluation cost, for a sufficiently high cost. |
Databáze: | OpenAIRE |
Externí odkaz: |