Simulation Experiments on (the Absence of) Ratings Bias in Reputation Systems
| Autor: | Loren Terveen, Brent Hecht, Jacob Thebault-Spieker, Daniel Kluver, Joseph A. Konstan, Maximilian Klein, Aaron Halfaker |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
Computer Networks and Communications
media_common.quotation_subject 05 social sciences Bayesian probability 02 engineering and technology Test (assessment) Human-Computer Interaction Bayesian statistics Race (biology) Manipulation checks 020204 information systems Scale (social sciences) 0202 electrical engineering electronic engineering information engineering Econometrics Gender bias 0501 psychology and cognitive sciences Psychology 050107 human factors Social Sciences (miscellaneous) Reputation media_common |
| Zdroj: | Proceedings of the ACM on Human-Computer Interaction. 1:1-25 |
| ISSN: | 2573-0142 |
| Popis: | As the gig economy continues to grow and freelance work moves online, five-star reputation systems are becoming more and more common. At the same time, there are increasing accounts of race and gender bias in evaluations of gig workers, with negative impacts for those workers. We report on a series of four Mechanical Turk-based studies in which participants who rated simulated gig work did not show race- or gender bias, while manipulation checks showed they reliably distinguished between low- and high-quality work. Given prior research, this was a striking result. To explore further, we used a Bayesian approach to verify absence of ratings bias (as opposed to merely not detecting bias). This Bayesian test let us identify an upper- bound: if any bias did exist in our studies, it was below an average of 0.2 stars on a five-star scale. We discuss possible interpretations of our results and outline future work to better understand the results. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|