Zobrazeno 1 - 9
of 9
pro vyhledávání: '"Vitvitskyi, Alex"'
Autor:
Barbero, Federico, Vitvitskyi, Alex, Perivolaropoulos, Christos, Pascanu, Razvan, Veličković, Petar
Positional Encodings (PEs) are a critical component of Transformer-based Large Language Models (LLMs), providing the attention mechanism with important sequence-position information. One of the most popular types of encoding used today in LLMs are Ro
Externí odkaz:
http://arxiv.org/abs/2410.06205
Autor:
Bounsi, Wilfried, Ibarz, Borja, Dudzik, Andrew, Hamrick, Jessica B., Markeeva, Larisa, Vitvitskyi, Alex, Pascanu, Razvan, Veličković, Petar
Transformers have revolutionized machine learning with their simple yet effective architecture. Pre-training Transformers on massive text datasets from the Internet has led to unmatched generalization for natural language understanding (NLU) tasks. H
Externí odkaz:
http://arxiv.org/abs/2406.09308
Autor:
Barbero, Federico, Banino, Andrea, Kapturowski, Steven, Kumaran, Dharshan, Araújo, João G. M., Vitvitskyi, Alex, Pascanu, Razvan, Veličković, Petar
We study how information propagates in decoder-only Transformers, which are the architectural backbone of most existing frontier large language models (LLMs). We rely on a theoretical signal propagation analysis -- specifically, we analyse the repres
Externí odkaz:
http://arxiv.org/abs/2406.04267
Autor:
Markeeva, Larisa, McLeish, Sean, Ibarz, Borja, Bounsi, Wilfried, Kozlova, Olga, Vitvitskyi, Alex, Blundell, Charles, Goldstein, Tom, Schwarzschild, Avi, Veličković, Petar
Eliciting reasoning capabilities from language models (LMs) is a critical direction on the path towards building intelligent systems. Most recent studies dedicated to reasoning focus on out-of-distribution performance on procedurally-generated synthe
Externí odkaz:
http://arxiv.org/abs/2406.04229
Autor:
Ibarz, Borja, Kurin, Vitaly, Papamakarios, George, Nikiforou, Kyriacos, Bennani, Mehdi, Csordás, Róbert, Dudzik, Andrew, Bošnjak, Matko, Vitvitskyi, Alex, Rubanova, Yulia, Deac, Andreea, Bevilacqua, Beatrice, Ganin, Yaroslav, Blundell, Charles, Veličković, Petar
The cornerstone of neural algorithmic reasoning is the ability to solve algorithmic tasks, especially in a way that generalises out of distribution. While recent years have seen a surge in methodological improvements in this area, they mostly focused
Externí odkaz:
http://arxiv.org/abs/2209.11142
Autor:
Campos, Víctor, Sprechmann, Pablo, Hansen, Steven, Barreto, Andre, Kapturowski, Steven, Vitvitskyi, Alex, Badia, Adrià Puigdomènech, Blundell, Charles
Designing agents that acquire knowledge autonomously and use it to solve new tasks efficiently is an important challenge in reinforcement learning. Knowledge acquired during an unsupervised pre-training phase is often transferred by fine-tuning neura
Externí odkaz:
http://arxiv.org/abs/2102.13515
Autor:
Badia, Adrià Puigdomènech, Piot, Bilal, Kapturowski, Steven, Sprechmann, Pablo, Vitvitskyi, Alex, Guo, Daniel, Blundell, Charles
Atari games have been a long-standing benchmark in the reinforcement learning (RL) community for the past decade. This benchmark was proposed to test general competency of RL algorithms. Previous work has achieved good average performance by doing ou
Externí odkaz:
http://arxiv.org/abs/2003.13350
Autor:
Badia, Adrià Puigdomènech, Sprechmann, Pablo, Vitvitskyi, Alex, Guo, Daniel, Piot, Bilal, Kapturowski, Steven, Tieleman, Olivier, Arjovsky, Martín, Pritzel, Alexander, Bolt, Andew, Blundell, Charles
We propose a reinforcement learning agent to solve hard exploration games by learning a range of directed exploratory policies. We construct an episodic memory-based intrinsic reward using k-nearest neighbors over the agent's recent experience to tra
Externí odkaz:
http://arxiv.org/abs/2002.06038
Autor:
Campos, V��ctor, Sprechmann, Pablo, Hansen, Steven, Barreto, Andre, Kapturowski, Steven, Vitvitskyi, Alex, Badia, Adri�� Puigdom��nech, Blundell, Charles
Designing agents that acquire knowledge autonomously and use it to solve new tasks efficiently is an important challenge in reinforcement learning. Knowledge acquired during an unsupervised pre-training phase is often transferred by fine-tuning neura
Externí odkaz:
https://explore.openaire.eu/search/publication?articleId=doi_dedup___::32b5bd8f8f27081cf98d345123418f18
http://arxiv.org/abs/2102.13515
http://arxiv.org/abs/2102.13515