Gesture and reference to objects in the here-and-now: Listeners’ use of gesture cues in quiet and in noise
| Autor: | Craig G. Chambers, Raheleh Saryazdi |
|---|---|
| Rok vydání: | 2022 |
| Předmět: |
Linguistics and Language
Gestures InformationSystems_INFORMATIONINTERFACESANDPRESENTATION(e.g. HCI) Speech recognition 05 social sciences Experimental and Cognitive Psychology Context (language use) Gaze 050105 experimental psychology Language and Linguistics Comprehension QUIET Speech Perception Humans Speech Eye tracking 0501 psychology and cognitive sciences Noise (video) Cues Psychology Gesture Spoken language |
| Zdroj: | Journal of Experimental Psychology: Learning, Memory, and Cognition. 48:583-597 |
| ISSN: | 1939-1285 0278-7393 |
| DOI: | 10.1037/xlm0001035 |
| Popis: | In face-to-face interaction, speakers spontaneously produce manual gestures that can facilitate listeners' comprehension of spoken language. The present study explores the factors affecting the uptake and influence of gesture cues in situations where a speaker is referring to objects visible to the listener. In this context, the listener's attention must be distributed across various scene regions, potentially reducing the ability to draw on and apply gesture cues in real time. In two experiments, the instruction provided by a speaker (e.g., "pick up the candy") was accompanied by an iconic grasp gesture (produced alongside the verb) that reflected the size/shape of the intended target. Effects on listeners' comprehension were compared with a no-gesture condition. Experiment 1 (audiovisual gating task) showed that, under simplified processing circumstances, gesture cues allowed earlier identification of intended targets. Experiment 2 (eye tracking) explored whether this facilitation is found in real-time comprehension, and whether attention to gesture information is influenced by the acoustic environment (quiet vs. background noise). Measures of gaze position showed that although the speaker's gesturing hand was rarely fixated directly, gestures did facilitate comprehension, particularly when the target object was smaller relative to alternatives. The magnitude of the gesture effect was greater in quiet than in noise, suggesting that the latter did not provoke listeners to increase attention to gesture to compensate for the challenging auditory signal. Together, the findings clarify how situational factors influence listeners' attention to visual information during real-time comprehension. (PsycInfo Database Record (c) 2022 APA, all rights reserved). |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|