Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
| Autor: | Liqun Chen, Dinghan Shen, Asli Celikyilmaz, Lawrence Carin, Jianfeng Gao, Yizhe Zhang, Xin Wang |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Theoretical computer science Computer Science - Computation and Language Computer science 05 social sciences Latent variable 010501 environmental sciences 01 natural sciences Machine Learning (cs.LG) 0502 economics and business Leverage (statistics) 050207 economics Encoder Computation and Language (cs.CL) Sentence 0105 earth and related environmental sciences |
| Zdroj: | Scopus-Elsevier ACL (1) |
| DOI: | 10.48550/arxiv.1902.00154 |
| Popis: | Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables. In this paper, we investigate several multi-level structures to learn a VAE model to generate long, and coherent text. In particular, we use a hierarchy of stochastic layers between the encoder and decoder networks to generate more informative latent codes. We also investigate a multi-level decoder structure to learn a coherent long-term structure by generating intermediate sentence representations as high-level plan vectors. Empirical results demonstrate that a multi-level VAE model produces more coherent and less repetitive long text compared to the standard VAE models and can further mitigate the posterior-collapse issue. Comment: To appear at ACL 2019 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|