Unsupervised Sentence Embedding Using Document Structure-Based Context
| Autor: | Taesung Lee, Youngja Park |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Document Structure Description
Coreference Artificial neural network Computer science business.industry Context (language use) 010501 environmental sciences computer.software_genre 01 natural sciences Paraphrase Metadata 030507 speech-language pathology & audiology 03 medical and health sciences ComputingMethodologies_DOCUMENTANDTEXTPROCESSING Artificial intelligence 0305 other medical science business computer Word (computer architecture) Natural language processing Sentence 0105 earth and related environmental sciences |
| Zdroj: | Machine Learning and Knowledge Discovery in Databases ISBN: 9783030461461 ECML/PKDD (2) |
| DOI: | 10.1007/978-3-030-46147-8_38 |
| Popis: | We present a new unsupervised method for learning general-purpose sentence embeddings. Unlike existing methods which rely on local contexts, such as words inside the sentence or immediately neighboring sentences, our method selects, for each target sentence, influential sentences from the entire document based on the document structure. We identify a dependency structure of sentences using metadata and text styles. Additionally, we propose an out-of-vocabulary word handling technique for the neural network outputs to model many domain-specific terms which were mostly discarded by existing sentence embedding training methods. We empirically show that the model relies on the proposed dependencies more than the sequential dependency in many cases. We also validate our model on several NLP tasks showing 23% F1-score improvement in coreference resolution in a technical domain and 5% accuracy increase in paraphrase detection compared to baselines. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|