Zobrazeno 1 - 10
of 50
pro vyhledávání: '"Ponti, Edoardo Maria"'
Autor:
Ostapenko, Oleksiy, Su, Zhan, Ponti, Edoardo Maria, Charlin, Laurent, Roux, Nicolas Le, Pereira, Matheus, Caccia, Lucas, Sordoni, Alessandro
The growing number of parameter-efficient adaptations of a base large language model (LLM) calls for studying whether we can reuse such trained adapters to improve performance for new tasks. We study how to best build a library of adapters given mult
Externí odkaz:
http://arxiv.org/abs/2405.11157
Language models (LMs) are bound to their tokenizer, which maps raw text to a sequence of vocabulary items (tokens). This restricts their flexibility: for example, LMs trained primarily on English may still perform well in other natural and programmin
Externí odkaz:
http://arxiv.org/abs/2405.07883
Autor:
Ponti, Edoardo Maria, O’Horan, Helen, Berzak, Yevgeni, Vulić, Ivan, Reichart, Roi, Poibeau, Thierry, Shutova, Ekaterina, Korhonen, Anna
Publikováno v:
Computational Linguistics, Vol 45, Iss 3, Pp 559-601 (2019)
Linguistic typology aims to capture structural and semantic variation across the world’s languages. A large-scale typology could provide excellent guidance for multilingual Natural Language Processing (NLP), particularly for languages that suffer f
Externí odkaz:
https://doaj.org/article/e766c2c989b842e388991cd857e2c997
Autor:
Daheim, Nico, Möllenhoff, Thomas, Ponti, Edoardo Maria, Gurevych, Iryna, Khan, Mohammad Emtiyaz
Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail? Here, we connect the inaccuracy of weighted-averaging to mismatches in the gradients and propose a new uncertai
Externí odkaz:
http://arxiv.org/abs/2310.12808
Massively multilingual Transformers (MMTs), such as mBERT and XLM-R, are widely used for cross-lingual transfer learning. While these are pretrained to represent hundreds of languages, end users of NLP systems are often interested only in individual
Externí odkaz:
http://arxiv.org/abs/2306.01709
Transfer learning has recently become the dominant paradigm of machine learning. Pre-trained models fine-tuned for downstream tasks achieve better performance with fewer labelled examples. Nonetheless, it remains unclear how to develop models that sp
Externí odkaz:
http://arxiv.org/abs/2302.11529
Autor:
Vulić, Ivan, Glavaš, Goran, Liu, Fangyu, Collier, Nigel, Ponti, Edoardo Maria, Korhonen, Anna
Pretrained multilingual language models (LMs) can be successfully transformed into multilingual sentence encoders (SEs; e.g., LaBSE, xMPNet) via additional fine-tuning or model distillation with parallel data. However, it remains unclear how to best
Externí odkaz:
http://arxiv.org/abs/2205.00267
Multilingual task-oriented dialogue (ToD) facilitates access to services and information for many (communities of) speakers. Nevertheless, the potential of this technology is not fully realised, as current datasets for multilingual ToD - both for mod
Externí odkaz:
http://arxiv.org/abs/2201.13405
Autor:
Bugliarello, Emanuele, Liu, Fangyu, Pfeiffer, Jonas, Reddy, Siva, Elliott, Desmond, Ponti, Edoardo Maria, Vulić, Ivan
Reliable evaluation benchmarks designed for replicability and comprehensiveness have driven progress in machine learning. Due to the lack of a multilingual benchmark, however, vision-and-language research has mostly focused on English language tasks.
Externí odkaz:
http://arxiv.org/abs/2201.11732
Fine-tuning the entire set of parameters of a large pretrained model has become the mainstream approach for transfer learning. To increase its efficiency and prevent catastrophic forgetting and interference, techniques like adapters and sparse fine-t
Externí odkaz:
http://arxiv.org/abs/2110.07560