Keyphrase Extraction as Sequence Labeling Using Contextualized Embeddings
| Autor: | Yaman Kumar, Roger Zimmermann, Haimin Zhang, Amanda Stent, Mayank Kulkarni, Rakesh Gosangi, Dhruva Sahrawat, Rajiv Ratn Shah, Debanjan Mahata, Agniv Sharma |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
0303 health sciences
Word embedding business.industry Computer science 02 engineering and technology computer.software_genre Sequence labeling 03 medical and health sciences Error analysis 0202 electrical engineering electronic engineering information engineering Benchmark (computing) Embedding 020201 artificial intelligence & image processing Artificial intelligence business computer Natural language processing Word (computer architecture) 030304 developmental biology |
| Zdroj: | Lecture Notes in Computer Science ISBN: 9783030454418 ECIR (2) |
| Popis: | In this paper, we formulate keyphrase extraction from scholarly articles as a sequence labeling task solved using a BiLSTM-CRF, where the words in the input text are represented using deep contextualized embeddings. We evaluate the proposed architecture using both contextualized and fixed word embedding models on three different benchmark datasets, and compare with existing popular unsupervised and supervised techniques. Our results quantify the benefits of: (a) using contextualized embeddings over fixed word embeddings; (b) using a BiLSTM-CRF architecture with contextualized word embeddings over fine-tuning the contextualized embedding model directly; and (c) using domain-specific contextualized embeddings (SciBERT). Through error analysis, we also provide some insights into why particular models work better than the others. Lastly, we present a case study where we analyze different self-attention layers of the two best models (BERT and SciBERT) to better understand their predictions. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|