PAUSE: Positive and Annealed Unlabeled Sentence Embedding

Autor:	Cao, Lele, Larsson, Emil, von Ehrenheim, Vilhelm, Rocha, Dhiana Deva Cavalcanti, Martin, Anna, Horn, Sonja
Rok vydání:	2021
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Sentence embedding refers to a set of effective and versatile techniques for converting raw text into numerical vector representations that can be used in a wide range of natural language processing (NLP) applications. The majority of these techniques are either supervised or unsupervised. Compared to the unsupervised methods, the supervised ones make less assumptions about optimization objectives and usually achieve better results. However, the training requires a large amount of labeled sentence pairs, which is not available in many industrial scenarios. To that end, we propose a generic and end-to-end approach -- PAUSE (Positive and Annealed Unlabeled Sentence Embedding), capable of learning high-quality sentence embeddings from a partially labeled dataset. We experimentally show that PAUSE achieves, and sometimes surpasses, state-of-the-art results using only a small fraction of labeled sentence pairs on various benchmark tasks. When applied to a real industrial use case where labeled samples are scarce, PAUSE encourages us to extend our dataset without the liability of extensive manual annotation work. Comment: Accepted by EMNLP 2021 main conference as long paper (12 pages and 2 figures). For source code, see https://github.com/EQTPartners/pause
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2109.03155 Zobrazit plný text záznamu View this record from Arxiv