Zipf’s word frequency law in natural language: A critical review and future directions
Autor: | Steven Piantadosi |
---|---|
Rok vydání: | 2014 |
Předmět: |
Scientific law
Structure (mathematical logic) Vocabulary Zipf's law media_common.quotation_subject Statistics as Topic Linguistics Experimental and Cognitive Psychology Models Theoretical Semantics Heaps' law Article Word lists by frequency Arts and Humanities (miscellaneous) Law Developmental and Educational Psychology Humans Psychology Natural language Language media_common |
Zdroj: | Psychonomic Bulletin & Review. 21:1112-1130 |
ISSN: | 1531-5320 1069-9384 |
Popis: | The frequency distribution of words has been a key object of study in statistical linguistics for the past 70 years. This distribution approximately follows a simple mathematical form known as Zipf ’ s law. This article first shows that human language has a highly complex, reliable structure in the frequency distribution over and above this classic law, although prior data visualization methods have obscured this fact. A number of empirical phenomena related to word frequencies are then reviewed. These facts are chosen to be informative about the mechanisms giving rise to Zipf’s law and are then used to evaluate many of the theoretical explanations of Zipf’s law in language. No prior account straightforwardly explains all the basic facts or is supported with independent evaluation of its underlying assumptions. To make progress at understanding why language obeys Zipf’s law, studies must seek evidence beyond the law itself, testing assumptions and evaluating novel predictions with new, independent data. |
Databáze: | OpenAIRE |
Externí odkaz: |