Multiple Instance Learning for Content Feedback Localization without Annotation
| Autor: | William Murray, Mark Rosenstein, Marcia A. Derr, Peter W. Foltz, Adam Wiemerslage, Scott Hellman, Lee Becker |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Computer science
business.industry Process (engineering) 05 social sciences 050301 education Rubric 02 engineering and technology Automated essay scoring computer.software_genre Task (project management) Formative assessment Annotation Classifier (linguistics) 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Artificial intelligence Natural approach business 0503 education computer Natural language processing |
| Zdroj: | BEA@ACL |
| DOI: | 10.18653/v1/2020.bea-1.3 |
| Popis: | Automated Essay Scoring (AES) can be used to automatically generate holistic scores with reliability comparable to human scoring. In addition, AES systems can provide formative feedback to learners, typically at the essay level. In contrast, we are interested in providing feedback specialized to the content of the essay, and specifically for the content areas required by the rubric. A key objective is that the feedback should be localized alongside the relevant essay text. An important step in this process is determining where in the essay the rubric designated points and topics are discussed. A natural approach to this task is to train a classifier using manually annotated data; however, collecting such data is extremely resource intensive. Instead, we propose a method to predict these annotation spans without requiring any labeled annotation data. Our approach is to consider AES as a Multiple Instance Learning (MIL) task. We show that such models can both predict content scores and localize content by leveraging their sentence-level score predictions. This capability arises despite never having access to annotation training data. Implications are discussed for improving formative feedback and explainable AES models. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|