TagRuler: Interactive Tool for Span-Level Data Programming by Demonstration
| Autor: | Dongjin Choi, Sara Evensen, Çağatay Demiralp, Estevam R. Hruschka |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Computer Science - Computation and Language business.industry Computer science Active learning (machine learning) Document classification Programming by demonstration Supervised learning Computer Science - Human-Computer Interaction Databases (cs.DB) Machine learning computer.software_genre Bottleneck Field (computer science) Human-Computer Interaction (cs.HC) Machine Learning (cs.LG) Annotation Computer Science - Databases Artificial intelligence business F1 score Computation and Language (cs.CL) computer |
| Zdroj: | WWW (Companion Volume) |
| Popis: | Despite rapid developments in the field of machine learning research, collecting high-quality labels for supervised learning remains a bottleneck for many applications. This difficulty is exacerbated by the fact that state-of-the-art models for NLP tasks are becoming deeper and more complex, often increasing the amount of training data required even for fine-tuning. Weak supervision methods, including data programming, address this problem and reduce the cost of label collection by using noisy label sources for supervision. However, until recently, data programming was only accessible to users who knew how to program. To bridge this gap, the Data Programming by Demonstration framework was proposed to facilitate the automatic creation of labeling functions based on a few examples labeled by a domain expert. This framework has proven successful for generating high-accuracy labeling models for document classification. In this work, we extend the DPBD framework to span-level annotation tasks, arguably one of the most time-consuming NLP labeling tasks. We built a novel tool, TagRuler, that makes it easy for annotators to build span-level labeling functions without programming and encourages them to explore trade-offs between different labeling models and active learning strategies. We empirically demonstrated that an annotator could achieve a higher F1 score using the proposed tool compared to manual labeling for different span-level annotation tasks. WWW'21 Demo |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|