Zobrazeno 1 - 10
of 212
pro vyhledávání: '"Merlo, Paola"'
We investigate to what degree existing LLMs encode abstract linguistic information in Italian in a multi-task setting. We exploit curated synthetic data on a large scale -- several Blackbird Language Matrices (BLMs) problems in Italian -- and use the
Externí odkaz:
http://arxiv.org/abs/2409.06622
In this paper, our goal is to investigate to what degree multilingual pretrained language models capture cross-linguistically valid abstract linguistic representations. We take the approach of developing curated synthetic data on a large scale, with
Externí odkaz:
http://arxiv.org/abs/2409.06567
Autor:
Nastase, Vivi, Merlo, Paola
Analyses of transformer-based models have shown that they encode a variety of linguistic information from their textual input. While these analyses have shed a light on the relation between linguistic information on one side, and internal architectur
Externí odkaz:
http://arxiv.org/abs/2407.18119
Autor:
Nastase, Vivi, Merlo, Paola
Sentence embeddings from transformer models encode in a fixed length vector much linguistic information. We explore the hypothesis that these embeddings consist of overlapping layers of information that can be separated, and on which specific types o
Externí odkaz:
http://arxiv.org/abs/2406.16563
Autor:
Nastase, Vivi, Merlo, Paola
Sentence and word embeddings encode structural and semantic information in a distributed manner. Part of the information encoded -- particularly lexical information -- can be seen as continuous, whereas other -- like structural information -- is most
Externí odkaz:
http://arxiv.org/abs/2312.11272
Autor:
Nastase, Vivi, Merlo, Paola
Publikováno v:
Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP 2023)
Sentence embeddings induced with various transformer architectures encode much semantic and syntactic information in a distributed manner in a one-dimensional array. We investigate whether specific grammatical information can be accessed in these dis
Externí odkaz:
http://arxiv.org/abs/2312.09890
Autor:
Merlo, Paola
We motivate and formally define a new task for fine-tuning rule-like generalization in large language models. It is conjectured that the shortcomings of current LLMs are due to a lack of ability to generalize. It has been argued that, instead, humans
Externí odkaz:
http://arxiv.org/abs/2306.11444
Current successes of machine learning architectures are based on computationally expensive algorithms and prohibitively large amounts of data. We need to develop tasks and data to train networks to reach more complex and more compositional skills. In
Externí odkaz:
http://arxiv.org/abs/2205.10866
Generative adversarial networks (GANs) have succeeded in inducing cross-lingual word embeddings -- maps of matching words across languages -- without supervision. Despite these successes, GANs' performance for the difficult case of distant languages
Externí odkaz:
http://arxiv.org/abs/2010.08432
Distributed representations of words which map each word to a continuous vector have proven useful in capturing important linguistic information not only in a single language but also across different languages. Current unsupervised adversarial appro
Externí odkaz:
http://arxiv.org/abs/1904.09446