A Dataset and Evaluation Metrics for Abstractive Compression of Sentences and Short Paragraphs
| Autor: | Toutanova, K., Brockett, C., Tran, K.M., Amershi, S., Su, J., Duh, K., Carreras, X. |
|---|---|
| Přispěvatelé: | Faculty of Science, IvI Research (FNWI), Information and Language Processing Syst (IVI, FNWI) |
| Jazyk: | angličtina |
| Rok vydání: | 2016 |
| Předmět: |
business.industry
Computer science media_common.quotation_subject Context (language use) 02 engineering and technology computer.software_genre Task (project management) 020204 information systems 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Quality (business) Artificial intelligence Paragraph business computer Natural language processing Sentence media_common Meaning (linguistics) |
| Zdroj: | EMNLP EMNLP 2016 : Conference on Empirical Methods in Natural Language Processing: November 1-5, 2016 Austin, Texas, USA : conference proceedings, 340-350 STARTPAGE=340;ENDPAGE=350;TITLE=EMNLP 2016 : Conference on Empirical Methods in Natural Language Processing |
| Popis: | We introduce a manually-created, multi-reference dataset for abstractive sentence and short paragraph compression. First, we examine the impact of single- and multi-sentence level editing operations on human compression quality as found in this corpus. We observe that substitution and rephrasing operations are more meaning preserving than other operations, and that compressing in context improves quality. Second, we systematically explore the correlations between automatic evaluation metrics and human judgments of meaning preservation and grammaticality in the compression task, and analyze the impact of the linguistic units used and precision versus recall measures on the quality of the metrics. Multi-reference evaluation metrics are shown to offer significant advantage over single reference-based metrics. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|