Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization
| Autor: | Dongyub Lee, Taesun Whang, Myeong Cheol Shin, Seungwoo Cho, Daniel Lee, Byeongil Ko, Jaechoon Jo, EungGyun Kim |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Agglutinative language Computer Science - Machine Learning Computer science Process (engineering) Machine Learning (stat.ML) 02 engineering and technology 010501 environmental sciences computer.software_genre 01 natural sciences Machine Learning (cs.LG) Statistics - Machine Learning Morpheme 0202 electrical engineering electronic engineering information engineering 0105 earth and related environmental sciences Computer Science - Computation and Language business.industry Automatic summarization Salient 020201 artificial intelligence & image processing Artificial intelligence business Computation and Language (cs.CL) computer Natural language processing Word (computer architecture) Meaning (linguistics) |
| Zdroj: | COLING |
| DOI: | 10.18653/v1/2020.coling-main.491 |
| Popis: | Text summarization refers to the process that generates a shorter form of text from the source document preserving salient information. Many existing works for text summarization are generally evaluated by using recall-oriented understudy for gisting evaluation (ROUGE) scores. However, as ROUGE scores are computed based on n-gram overlap, they do not reflect semantic meaning correspondences between generated and reference summaries. Because Korean is an agglutinative language that combines various morphemes into a word that express several meanings, ROUGE is not suitable for Korean summarization. In this paper, we propose evaluation metrics that reflect semantic meanings of a reference summary and the original document, Reference and Document Aware Semantic Score (RDASS). We then propose a method for improving the correlation of the metrics with human judgment. Evaluation results show that the correlation with human judgment is significantly higher for our evaluation metrics than for ROUGE scores. Comment: COLING 2020 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|