Reassessing the Goals of Grammatical Error Correction: Fluency Instead of Grammaticality
| Autor: | Courtney Napoles, Matt Post, Joel Tetreault, Keisuke Sakaguchi |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
Linguistics and Language
Computer science business.industry Communication Speech recognition 02 engineering and technology computer.software_genre Grammatical error Computer Science Applications Human-Computer Interaction Correlation 03 medical and health sciences Fluency Annotation 0302 clinical medicine Artificial Intelligence 030221 ophthalmology & optometry 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Grammaticality Artificial intelligence business computer Natural language processing Coding (social sciences) |
| Zdroj: | Transactions of the Association for Computational Linguistics. 4:169-182 |
| ISSN: | 2307-387X |
| DOI: | 10.1162/tacl_a_00091 |
| Popis: | The field of grammatical error correction (GEC) has grown substantially in recent years, with research directed at both evaluation metrics and improved system performance against those metrics. One unvisited assumption, however, is the reliance of GEC evaluation on error-coded corpora, which contain specific labeled corrections. We examine current practices and show that GEC’s reliance on such corpora unnaturally constrains annotation and automatic evaluation, resulting in (a) sentences that do not sound acceptable to native speakers and (b) system rankings that do not correlate with human judgments. In light of this, we propose an alternate approach that jettisons costly error coding in favor of unannotated, whole-sentence rewrites. We compare the performance of existing metrics over different gold-standard annotations, and show that automatic evaluation with our new annotation scheme has very strong correlation with expert rankings (ρ = 0.82). As a result, we advocate for a fundamental and necessary shift in the goal of GEC, from correcting small, labeled error types, to producing text that has native fluency. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|