Syntax-driven iterative expansion language models for controllable text generation
| Autor: | Casas Manzanares, Noé, Rodríguez Fonollosa, José Adrián, Ruiz Costa-Jussà, Marta |
|---|---|
| Přispěvatelé: | Universitat Politècnica de Catalunya. Doctorat en Teoria del Senyal i Comunicacions, Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions, Universitat Politècnica de Catalunya. Departament de Ciències de la Computació, Universitat Politècnica de Catalunya. VEU - Grup de Tractament de la Parla |
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
Grammars
Natural language processing (Computer science) Enginyeria de la telecomunicació::Processament del senyal::Processament de la parla i del senyal acústic [Àrees temàtiques de la UPC] Computational linguistics Lingüística computacional Text analysis Tractament del llenguatge natural (Informàtica) Informàtica::Intel·ligència artificial::Llenguatge natural [Àrees temàtiques de la UPC] Trees (mathematics) Vocabulary |
| Zdroj: | UPCommons. Portal del coneixement obert de la UPC Universitat Politècnica de Catalunya (UPC) |
| Popis: | The dominant language modeling paradigm handles text as a sequence of discrete tokens. While that approach can capture the latent structure of the text, it is inherently constrained to sequential dynamics for text generation. We propose a new paradigm for introducing a syntactic inductive bias into neural text generation, where the dependency parse tree is used to drive the Transformer model to generate sentences iteratively. Our experiments show that this paradigm is effective at text generation, with quality between LSTMs and Transformers, and comparable diversity, requiring less than half their decoding steps, and its generation process allows direct control over the syntactic constructions of the generated text, enabling the induction of stylistic variations. This work is partially supported by Lucy Software / United Language Group (ULG) and the Catalan Agency for Management of University and Research Grants (AGAUR) through an Industrial Ph.D. Grant. This work also is supported in part by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund through the postdoctoral senior grant Ramón y Cajal and by the Agencia Estatal de Investigación through the projects EUR2019-103819, PCIN2017-079 and PID2019-107579RB-I00 / AEI / 10.13039/501100011033 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|