Zobrazeno 1 - 10
of 449
pro vyhledávání: '"Pretraining and Finetuning"'
This study introduces an innovative approach to analyzing unlabeled data in high-energy physics (HEP) through the application of self-supervised learning (SSL). Faced with the increasing computational cost of producing high-quality labeled simulation
Externí odkaz:
http://arxiv.org/abs/2408.09343
Autor:
Chidlovskii, Boris, Antsfeld, Leonid
For the task of simultaneous monocular depth and visual odometry estimation, we propose learning self-supervised transformer-based models in two steps. Our first step consists in a generic pretraining to learn 3D geometry, using cross-view completion
Externí odkaz:
http://arxiv.org/abs/2406.11019
Large language models (LLMs) are computationally intensive. The computation workload and the memory footprint grow quadratically with the dimension (layer width). Most of LLMs' parameters come from the linear layers of the transformer structure and a
Externí odkaz:
http://arxiv.org/abs/2402.13533
Medical visual question answering (VQA) is a challenging multimodal task, where Vision-Language Pre-training (VLP) models can effectively improve the generalization performance. However, most methods in the medical field treat VQA as an answer classi
Externí odkaz:
http://arxiv.org/abs/2401.05163
Akademický článek
Tento výsledek nelze pro nepřihlášené uživatele zobrazit.
K zobrazení výsledku je třeba se přihlásit.
K zobrazení výsledku je třeba se přihlásit.
Autor:
Luong, Kha-Dinh, Singh, Ambuj
Property prediction on molecular graphs is an important application of Graph Neural Networks. Recently, unlabeled molecular data has become abundant, which facilitates the rapid development of self-supervised learning for GNNs in the chemical domain.
Externí odkaz:
http://arxiv.org/abs/2310.03274
Contrastive learning (CL) can learn generalizable feature representations and achieve the state-of-the-art performance of downstream tasks by finetuning a linear classifier on top of it. However, as adversarial robustness becomes vital in image class
Externí odkaz:
http://arxiv.org/abs/2111.01124
Relation extraction is used to populate knowledge bases that are important to many applications. Prior datasets used to train relation extraction models either suffer from noisy labels due to distant supervision, are limited to certain domains or are
Externí odkaz:
http://arxiv.org/abs/2102.09681
Autor:
Tang, Yuqing, Tran, Chau, Li, Xian, Chen, Peng-Jen, Goyal, Naman, Chaudhary, Vishrav, Gu, Jiatao, Fan, Angela
Recent work demonstrates the potential of multilingual pretraining of creating one model that can be used for various tasks in different languages. Previous work in multilingual pretraining has demonstrated that machine translation systems can be cre
Externí odkaz:
http://arxiv.org/abs/2008.00401
Autor:
Pierse, Nuo Wang, Lu, Jingwen
We demonstrate that explicitly aligning the pretraining objectives to the finetuning objectives in language model training significantly improves the finetuning task performance and reduces the minimum amount of finetuning examples required. The perf
Externí odkaz:
http://arxiv.org/abs/2002.02000