STD: An Automatic Evaluation Metric for Machine Translation Based on Word Embeddings
| Autor: | Wujie Zheng, Zibin Zheng, Fanghua Ye, Yuetang Deng, Pairui Li, Chuan Chen |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
Matching (statistics)
Acoustics and Ultrasonics Machine translation Computer science business.industry computer.software_genre Computational Mathematics Range (mathematics) Semantic similarity Metric (mathematics) Computer Science (miscellaneous) NIST Artificial intelligence Electrical and Electronic Engineering business computer Natural language processing Word (computer architecture) Word order |
| Zdroj: | IEEE/ACM Transactions on Audio, Speech, and Language Processing. 27:1497-1506 |
| ISSN: | 2329-9304 2329-9290 |
| DOI: | 10.1109/taslp.2019.2922845 |
| Popis: | Lexical-based metrics such as BLEU, NIST, and WER have been widely used in machine translation MT evaluation. However, these metrics badly represent semantic relationships and impose strict identity matching, leading to moderate correlation with human judgments. In this paper, we propose a novel MT automatic evaluation metric Semantic Travel Distance STD based on word embeddings. STD incorporates both semantic and lexical features word embeddings and n-gram and word order into one metric. It measures the semantic distance between the hypothesis and reference by calculating the minimum cumulative cost that the embedded n-grams of the hypothesis need to “travel” to reach the embedded n-grams of the reference. Experiment results show that STD has a better and more robust performance than a range of state-of-the-art metrics for both the segment-level and system-level evaluation. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|