Non-contrastive sentence representations via self-supervision

Autor:	Farina, Marco, Pappadopulo, Duccio
Rok vydání:	2023
Předmět:	Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	Sample contrastive methods, typically referred to simply as contrastive are the foundation of most unsupervised methods to learn text and sentence embeddings. On the other hand, a different class of self-supervised loss functions and methods have been considered in the computer vision community and referred to as dimension contrastive. In this paper, we thoroughly compare this class of methods with the standard baseline for contrastive sentence embeddings, SimCSE. We find that self-supervised embeddings trained using dimension contrastive objectives can outperform SimCSE on downstream tasks without needing auxiliary loss functions. Comment: Submitted and rejected by EMNLP 2023. Contact the authors for a copy of the "reviews"
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2310.17690 Zobrazit plný text záznamu View this record from Arxiv