Perceived Burden, Focus of Attention, and the Urge to Justify: The Impact of the Number of Screens and Probe Order on the Response Behavior of Probing Questions
| Autor: | Meitinger, K.M., Toroslu, Adrian, Raiber, Klara, Braun, Michael, Leerstoel Schoot, Methodology and statistics for the behavioural and social sciences |
|---|---|
| Přispěvatelé: | Leerstoel Schoot, Methodology and statistics for the behavioural and social sciences |
| Jazyk: | angličtina |
| Rok vydání: | 2022 |
| Předmět: |
Statistics and Probability
media_common.quotation_subject question order effects Context (language use) Inequality cohesion and modernization Multiple Probes Cognition 0504 sociology Web probing Taverne 050602 political science & public administration Quality (business) Ongelijkheid cohesie en modernisering visual design media_common Focus (computing) Context effect Applied Mathematics 05 social sciences Comparability 050401 social sciences methods 0506 political science Test (assessment) Term (time) Order (business) Question order Statistics Probability and Uncertainty Psychology Social Sciences (miscellaneous) Cognitive psychology |
| Zdroj: | Journal of Survey Statistics and Methodology, 10, 923-944 Journal of Survey Statistics and Methodology, 10(4). Oxford University Press Journal of Survey Statistics and Methodology, 10, 4, pp. 923-944 |
| ISSN: | 2325-0984 2325-0992 |
| Popis: | Web probing is a valuable tool to assess the validity and comparability of survey items. It uses different probe types—such as category-selection probes and specific probes—to inquire about different aspects of an item. Previous web probing studies often asked one probe type per item, but research situations exist where it might be preferable to test potentially problematic items with multiple probes. However, the response behavior might be affected by two factors: question order and the visual presentation of probes on one screen versus multiple screens as well as their interaction. In this study, we report evidence from a web experiment that was conducted with 532 respondents from Germany in September 2013. Experimental groups varied by screen number (1 versus 2) and probe order (category-selection probe first versus specific probe first). We assessed the impact of these manipulations on several indicators of response quality, probe answer content, and the respondents’ motivation with logistic regressions and two-way ANOVAs. We reveal that multiple mechanisms push response behavior in this context: perceived response burden, the focus of attention, the need for justification, and verbal context effects. We find that response behavior in the condition with two screens and category-selection probe first outperforms all other experimental conditions. We recommend this implementation in all but one scenario: if the goal is to test an item that includes a key term with a potentially too large lexical scope, we recommend starting with a specific probe but on the same screen as the category-selection probe. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|