Zobrazeno 1 - 10
of 9 822
pro vyhledávání: '"Language modelling"'
Autor:
Walker, Nicholas
Causal decoder-only transformer models used for generative language modelling, such as Generative Pre-trained Transformers (GPT), are trained to predict the next token in a sequence based only on its previous tokens. Despite this simple training obje
Externí odkaz:
http://arxiv.org/abs/2410.18160
Autor:
Brunato, Dominique
This position paper investigates the potential of integrating insights from language impairment research and its clinical treatment to develop human-inspired learning strategies and evaluation frameworks for language models (LMs). We inspect the theo
Externí odkaz:
http://arxiv.org/abs/2412.15785
Autor:
Zhou, Yixuan, Qin, Xiaoyu, Jin, Zeyu, Zhou, Shuoyi, Lei, Shun, Zhou, Songtao, Wu, Zhiyong, Jia, Jia
Recent AIGC systems possess the capability to generate digital multimedia content based on human language instructions, such as text, image and video. However, when it comes to speech, existing methods related to human instruction-to-speech generatio
Externí odkaz:
http://arxiv.org/abs/2408.15676
Language Modelling has been a central part of Natural Language Processing for a very long time and in the past few years LSTM-based language models have been the go-to method for commercial language modeling. Recently, it has been shown that when loo
Externí odkaz:
http://arxiv.org/abs/2406.10256
Large Language Models (LLMs) have revolutionised the field of Natural Language Processing (NLP) and have achieved state-of-the-art performance in practically every task in this field. However, the prevalent approach used in text generation, Causal La
Externí odkaz:
http://arxiv.org/abs/2405.12630
Autor:
O'Brien, Hugh1 (AUTHOR), Salm, Max1 (AUTHOR) m.salm@achillestx.com, Morton, Laura T.1 (AUTHOR), Szukszto, Maciej2 (AUTHOR), O'Farrell, Felix1 (AUTHOR), Boulton, Charlotte2 (AUTHOR), King, Laurence1,2 (AUTHOR), Bola, Supreet Kaur2 (AUTHOR), Becker, Pablo D.1 (AUTHOR), Craig, Andrew1 (AUTHOR), Nielsen, Morten3 (AUTHOR), Samuels, Yardena4 (AUTHOR), Swanton, Charles5 (AUTHOR), Mansour, Marc R.2,6 (AUTHOR), Hadrup, Sine Reker3 (AUTHOR) m.salm@achillestx.com, Quezada, Sergio A.1,2 (AUTHOR) sirha@dtu.dk
Publikováno v:
PLoS Computational Biology. 11/11/2024, Vol. 20 Issue 11, p1-23. 23p.
In recent years, self-supervised pre-training methods have gained significant traction in learning high-level information from raw speech. Among these methods, HuBERT has demonstrated SOTA performance in automatic speech recognition (ASR). However, H
Externí odkaz:
http://arxiv.org/abs/2406.05661
Autor:
Wang, Shida
In this paper, we investigate the length-extension of state-space models (SSMs) in language modeling. Length extension involves training models on short sequences and testing them on longer ones. We show that state-space models trained with zero hidd
Externí odkaz:
http://arxiv.org/abs/2406.02080
The prediction of ship trajectories is a growing field of study in artificial intelligence. Traditional methods rely on the use of LSTM, GRU networks, and even Transformer architectures for the prediction of spatio-temporal series. This study propose
Externí odkaz:
http://arxiv.org/abs/2405.09596
Predicting legal judgments with reliable confidence is paramount for responsible legal AI applications. While transformer-based deep neural networks (DNNs) like BERT have demonstrated promise in legal tasks, accurately assessing their prediction conf
Externí odkaz:
http://arxiv.org/abs/2404.10481