Scoring vs. Ranking: An Experimental Study of Idea Evaluation Processes
| Autor: | Dilney Gonçalves, Zhijian Cui, Shijith Kumar Pm |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
business.industry
Computer science Process (engineering) media_common.quotation_subject 05 social sciences Explanatory model Innovation management Experimental data Context (language use) Management Science and Operations Research Machine learning computer.software_genre Industrial and Manufacturing Engineering Ranking (information retrieval) Test (assessment) Management of Technology and Innovation 0502 economics and business 050211 marketing Quality (business) Artificial intelligence business computer 050203 business & management media_common |
| Zdroj: | Production and Operations Management. 28:176-188 |
| ISSN: | 1059-1478 |
| DOI: | 10.1111/poms.12910 |
| Popis: | Using an online experiment, this study examines the efficacy of two idea evaluation processes: scoring vs. ranking in the context of innovation management. In the scoring process, the evaluators are asked to rate the quality of each idea by assigning it a score (e.g., from 0 to 10), while in the ranking process the evaluator simply orders all ideas according to their perceived qualities. Our results suggest that the scoring process strictly outperforms the ranking process in terms of the likelihood of selecting the highest‐quality ideas. This result remains robust, irrespective of the possibility of allowing ties in the ranking process. However, when the number of ideas to be evaluated is reduced from eight to three, the efficacies of the two idea evaluation processes become similar. Additionally, we find that the efficacy of the ranking process is significantly improved when additional information is provided, yet the efficacy of the scoring process does not change with more information. Based on the observations from the experimental data, we propose and test an explanatory model in which the information becomes a cue that directs the participants’ efforts to evaluate the ideas (i.e., the time taken for evaluation). |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Plný text