A statistical model from information theory to explain Zipf's law of abbreviation
| Autor: | Hernández-Fernández, Antoni, González Torres, Iván, Lacasa, Lucas, Kello, Christopher T., Luque Serrano, Bartolome |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: | |
| Zdroj: | International Conference on Interdisciplinary Advances in Statistical Learning | International Conference on Interdisciplinary Advances in Statistical Learning.. | 27-29 jun. | San-Sebastian, España Archivo Digital UPM Universidad Politécnica de Madrid |
| Popis: | Brevity and frequency are two crucial factors in the processes of statistical learning in language. The compression principle had already been used previously to explain the origin of Zipf’s law for the frequency of words. Here we use a model from information theory to also explain the Zipf’s law of abbreviation, or the statistical tendency of more frequent elements in language to be shorter (in characters in the case of written language, and in time durations for oral communication). As far as we know, we show for the first time that Zipf’s law of abbreviation is a global speech process that holds in words regardless of what are the linguistics units of study. In addition, the derived model from information theory allows us to fit empirically linguistic data in English (Buckeye Corpus), considering both acoustic elements (phonemes and words) and its transcripts. This raises that the processes measured in units of written text are a byproduct of spontaneous speech patterns. The more a word is used, the greatest effort in compression that will make it shorter; but also the shorter it is, the more times it will be used statistically. This work paves the way for new experimental approaches to the study of statistical learning in human language and other communication systems. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|