Feature-rich continuous language models for speech recognition
| Autor: | Piotr Mirowski, Sumit Chopra, Suhrid Balakrishnan, Srinivas Bangalore |
|---|---|
| Rok vydání: | 2010 |
| Předmět: |
Text corpus
Perplexity Computer science business.industry Speech recognition Computer Science::Computation and Language (Computational Linguistics and Natural Language and Speech Processing) Context (language use) computer.software_genre Cache language model Feature (machine learning) Artificial intelligence Language model business computer Natural language Natural language processing Word (computer architecture) |
| Zdroj: | SLT |
| Popis: | State-of-the-art probabilistic models of text such as n-grams require an exponential number of examples as the size of the context grows, a problem that is due to the discrete word representation. We propose to solve this problem by learning a continuous-valued and low-dimensional mapping of words, and base our predictions for the probabilities of the target word on non-linear dynamics of the latent space representation of the words in context window. We build on neural networks-based language models; by expressing them as energy-based models, we can further enrich the models with additional inputs such as part-of-speech tags, topic information and graphs of word similarity. We demonstrate a significantly lower perplexity on different text corpora, as well as improved word accuracy rate on speech recognition tasks, as compared to Kneser-Ney back-off n-gram-based language models. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|