Effects of the Number of Open-Ended Probing Questions on Response Quality in Cognitive Online Pretests
| Autor: | Neuert, Cornelia, Lenzner, Timo |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Sozialwissenschaften
Soziologie Social sciences sociology anthropology online pretest open-ended questions probes questionnaire design web probing web survey Erhebungstechniken und Analysetechniken der Sozialwissenschaften Methods and Techniques of Data Collection and Data Analysis Statistical Methods Computer Methods Online-Befragung Pretest Fragebogen Umfrageforschung Antwortverhalten Datenqualität online survey pretest questionnaire survey research response behavior data quality |
| Zdroj: | Social Science Computer Review, 39, 3, 456-468 |
| Druh dokumentu: | Zeitschriftenartikel<br />journal article |
| ISSN: | 1552-8286 |
| DOI: | 10.1177/0894439319866397 |
| Popis: | Cognitive online pretests have, in recent years, become recognized as a promising tool for evaluating questions prior to their use in actual surveys. While existing research has shown that cognitive online pretests produce similar results to face-to-face cognitive interviews with regard to the problems detected and the item revisions suggested, little is known about the ideal design of a cognitive online pretest. This study examines whether the number of open-ended probing questions asked during a cognitive online pretest has an effect on the quality and depth of respondents' answers as well as on respondents' satisfaction with the survey. We conducted an experiment in which we varied the number of open-ended probing questions that respondents received during a cognitive online pretest. The questionnaire consisted of 26 survey questions, and respondents received either 13 probing questions (n = 120, short version) or 21 probing questions (n = 120, long version). The findings suggest that asking a greater number of open-ended probes in a cognitive online pretest does not undermine the quality of respondents' answers represented by the response quality indicators: (1) amount of probe nonresponse, (2) number of uninterpretable answers, (3) number of dropouts, (4) number of words, (5) response times, and (6) number and type of themes covered by the probes. Furthermore, the respondents' satisfaction with the survey is not affected by the number of probes being asked. |
| Databáze: | SSOAR – Social Science Open Access Repository |
| Externí odkaz: |
|