The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation

Autor: Marcos Alberto Trevisan, Alan Taitz, Jacobo D. Sitt, Natalia Gabriela Elisei, Laurent D. Cohen, M. Florencia Assaneo, Mónica Trípodi
Přispěvatelé: Institut du Cerveau et de la Moëlle Epinière = Brain and Spine Institute (ICM), Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP), Service de Neurologie [CHU Pitié-Salpêtrière], IFR70-CHU Pitié-Salpêtrière [AP-HP], Université Pierre et Marie Curie - Paris 6 (UPMC)-Institut National de la Santé et de la Recherche Médicale (INSERM)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Centre National de la Recherche Scientifique (CNRS), Service de neurologie 1 [CHU Pitié-Salpétrière], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-CHU Pitié-Salpêtrière [AP-HP], Sorbonne Université (SU)-Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Sorbonne Université (SU)
Jazyk: angličtina
Rok vydání: 2018
Předmět:
Male
Visual perception
Speech recognition
[SDV]Life Sciences [q-bio]
LANGUAGE
lcsh:Medicine
Social Sciences
purl.org/becyt/ford/1 [https]
Machine Learning
0302 clinical medicine
Medicine and Health Sciences
Psychology
lcsh:Science
media_common
Language
[PHYS]Physics [physics]
Grammar
Multidisciplinary
Physics
05 social sciences
Phonetics
Middle Aged
Sound
Physical Sciences
Onomatopoeia
Auditory Perception
Speech Perception
Visual Perception
Female
Sensory Perception
Anatomy
CIENCIAS NATURALES Y EXACTAS
Vocal tract
Research Article
Adult
Computer and Information Sciences
media_common.quotation_subject
Phonology
050105 experimental psychology
ONOMATOPOEIAS
Ciencias Biológicas
03 medical and health sciences
Young Adult
Tongue
Artificial Intelligence
Perception
Classifier (linguistics)
Humans
0501 psychology and cognitive sciences
Syntax
purl.org/becyt/ford/1.6 [https]
Vowels
Mouth
Syntax (programming languages)
lcsh:R
Phonemes
Biology and Life Sciences
Linguistics
Acoustics
Models
Theoretical

Biofísica
Manner of articulation
Voice
COGNITION
lcsh:Q
Digestive System
030217 neurology & neurosurgery
Neuroscience
Zdroj: PLoS ONE
PLoS ONE, Public Library of Science, 2018, 13 (3), pp.e0193466. ⟨10.1371/journal.pone.0193466⟩
PLoS ONE, Vol 13, Iss 3, p e0193466 (2018)
CONICET Digital (CONICET)
Consejo Nacional de Investigaciones Científicas y Técnicas
instacron:CONICET
ISSN: 1932-6203
Popis: Sound-symbolic word classes are found in different cultures and languages worldwide. These words are continuously produced to code complex information about events. Here we explore the capacity of creative language to transport complex multisensory information in a controlled experiment, where our participants improvised onomatopoeias from noisy moving objects in audio, visual and audiovisual formats. We found that consonants communicate movement types (slide, hit or ring) mainly through the manner of articulation in the vocal tract. Vowels communicate shapes in visual stimuli (spiky or rounded) and sound frequencies in auditory stimuli through the configuration of the lips and tongue. A machine learning model was trained to classify movement types and used to validate generalizations of our results across formats. We implemented the classifier with a list of cross-linguistic onomatopoeias simple actions were correctly classified, while different aspects were selected to build onomatopoeias of complex actions. These results show how the different aspects of complex sensory information are coded and how they interact in the creation of novel onomatopoeias. Fil: Taitz, Alan. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física de Buenos Aires. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física de Buenos Aires; Argentina Fil: Assaneo, María Florencia. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física de Buenos Aires. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física de Buenos Aires; Argentina Fil: Elisei, Natalia Gabriela. Universidad de Buenos Aires. Facultad de Medicina; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina Fil: Tripodi, Monica Noemi. Universidad de Buenos Aires; Argentina Fil: Cohen, Laurent. Centre National de la Recherche Scientifique; Francia. Universite Pierre et Marie Curie; Francia. Institut National de la Santé et de la Recherche Médicale; Francia Fil: Sitt, Jacobo Diego. Centre National de la Recherche Scientifique; Francia. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Institut National de la Santé et de la Recherche Médicale; Francia. Universite Pierre et Marie Curie; Francia Fil: Trevisan, Marcos Alberto. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física de Buenos Aires. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física de Buenos Aires; Argentina
Databáze: OpenAIRE