$\textit{latent}$-GLAT: Glancing at Latent Variables for Parallel Text Generation

Autor:	Bao, Yu, Zhou, Hao, Huang, Shujian, Wang, Dongqi, Qian, Lihua, Dai, Xinyu, Chen, Jiajun, Li, Lei
Rok vydání:	2022
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence
Druh dokumentu:	Working Paper
Popis:	Recently, parallel text generation has received widespread attention due to its success in generation efficiency. Although many advanced techniques are proposed to improve its generation quality, they still need the help of an autoregressive model for training to overcome the one-to-many multi-modal phenomenon in the dataset, limiting their applications. In this paper, we propose $\textit{latent}$-GLAT, which employs the discrete latent variables to capture word categorical information and invoke an advanced curriculum learning technique, alleviating the multi-modality problem. Experiment results show that our method outperforms strong baselines without the help of an autoregressive model, which further broadens the application scenarios of the parallel decoding paradigm. Comment: 12 pages, 5 figures, 6 tables. Accepted as a long paper in the main conference of ACL-2022
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2204.02030 Zobrazit plný text záznamu View this record from Arxiv