Metaphor processing is supramodal semantic processing: The role of the bilateral lateral temporal regions in multimodal communication.

Autor: Joue G; Human Technology Center, RWTH Aachen University, Theaterplatz 14, 52056 Aachen, Germany; Department of Psychiatry, Psychotherapy and Psychosomatics, School of Medicine, RWTH Aachen University, Pauwelsstraße 30, 52074 Aachen, Germany; Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany. Electronic address: joue@humtec.rwth-aachen.de., Boven L; School of Medicine, RWTH Aachen University, Pauwelsstraße 30, 52074 Aachen, Germany., Willmes K; Section Neuropsychology, Department of Neurology, School of Medicine, RWTH Aachen University, Pauwelsstraße 30, 52074 Aachen, Germany., Evola V; Human Technology Center, RWTH Aachen University, Theaterplatz 14, 52056 Aachen, Germany; Bonn-Aachen International Center for Information Technology, Dahlmannstraße 2, 53113 Bonn, Germany; Faculty of Social Sciences and Humanities, New University of Lisbon, Portugal., Demenescu LR; Department of Psychiatry, Psychotherapy and Psychosomatics, School of Medicine, RWTH Aachen University, Pauwelsstraße 30, 52074 Aachen, Germany., Hassemer J; Human Technology Center, RWTH Aachen University, Theaterplatz 14, 52056 Aachen, Germany., Mittelberg I; Human Technology Center, RWTH Aachen University, Theaterplatz 14, 52056 Aachen, Germany., Mathiak K; Department of Psychiatry, Psychotherapy and Psychosomatics, School of Medicine, RWTH Aachen University, Pauwelsstraße 30, 52074 Aachen, Germany; JARA, Translational Brain Medicine, 52425 Jülich, Germany; Institute of Neuroscience and Medicine (INM-1), Research Center Jülich, 52425 Jülich, Germany., Schneider F; Department of Psychiatry, Psychotherapy and Psychosomatics, School of Medicine, RWTH Aachen University, Pauwelsstraße 30, 52074 Aachen, Germany., Habel U; Department of Psychiatry, Psychotherapy and Psychosomatics, School of Medicine, RWTH Aachen University, Pauwelsstraße 30, 52074 Aachen, Germany.
Jazyk: angličtina
Zdroj: Brain and language [Brain Lang] 2020 Jun; Vol. 205, pp. 104772. Date of Electronic Publication: 2020 Feb 29.
DOI: 10.1016/j.bandl.2020.104772
Abstrakt: This paper presents an fMRI study on healthy adult understanding of metaphors in multimodal communication. We investigated metaphors expressed either only in coverbal gestures ("monomodal metaphors") or in speech with accompanying gestures ("multimodal metaphors"). Monomodal metaphoric gestures convey metaphoric information not expressed in the accompanying speech (e.g. saying the non-metaphoric utterance, "She felt bad" while dropping down the hand with palm facing up; here, the gesture alone indicates metaphoricity), whereas coverbal gestures in multimodal metaphors indicate metaphoricity redundant to the speech (e.g. saying the metaphoric utterance, "Her spirits fell" while dropping the hand with palm facing up). In other words, in monomodal metaphors, gestures add information not spoken, whereas the gestures in multimodal metaphors can be redundant to the spoken content. Understanding and integrating the information in each modality, here spoken and visual, is important in multimodal communication, but most prior studies have only considered multimodal metaphors where the gesture is redundant to what is spoken. Our participants watched audiovisual clips of an actor speaking while gesturing. We found that abstract metaphor comprehension recruited the lateral superior/middle temporal cortices, regardless of the modality in which the conceptual metaphor is expressed. These results suggest that abstract metaphors, regardless of modality, involve resources implicated in general semantic processing and are consistent with the role of these areas in supramodal semantic processing as well as the theory of embodied cognition.
(Copyright © 2020 Elsevier Inc. All rights reserved.)
Databáze: MEDLINE
Externí odkaz: