Inference Strategies for Machine Translation with Conditional Masking
| Autor: | Julia Kreutzer, Colin Cherry, George Foster |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Computation and Language Machine translation business.industry Computer science Inference Conditional probability Machine learning computer.software_genre Thresholding Masking (Electronic Health Record) Language model Artificial intelligence Heuristics business Computation and Language (cs.CL) computer |
| Zdroj: | EMNLP (1) |
| Popis: | Conditional masked language model (CMLM) training has proven successful for non-autoregressive and semi-autoregressive sequence generation tasks, such as machine translation. Given a trained CMLM, however, it is not clear what the best inference strategy is. We formulate masked inference as a factorization of conditional probabilities of partial sequences, show that this does not harm performance, and investigate a number of simple heuristics motivated by this perspective. We identify a thresholding strategy that has advantages over the standard "mask-predict" algorithm, and provide analyses of its behavior on machine translation tasks. EMNLP 2020, updated Fig 3 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|