Zobrazeno 1 - 5
of 5
pro vyhledávání: '"Liangzhu Ge"'
The core of self-supervised learning for pre-training language models includes pre-training task design as well as appropriate data augmentation. Most data augmentations in language model pre-training are context-independent. A seminal contextualized
Externí odkaz:
https://explore.openaire.eu/search/publication?articleId=doi_dedup___::d7460534a7ebbb20b5e1db07b8dcbaa4
http://arxiv.org/abs/2106.07176
http://arxiv.org/abs/2106.07176
Publikováno v:
Mathematical Problems in Engineering, Vol 2021 (2021)
During the development of deep neural networks (DNNs), it is difficult to trade off the performance of fitting ability and generalization ability in training set and unknown data (such as test set). The current solution is to reduce the complexity of
Publikováno v:
CIKM
Deep Neural Networks (DNNs) with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Till now, many regularizers such as dropout, data augmentation have been proposed to
Publikováno v:
CIKM
Recently, recurrent neural networks (RNNs) based methods have achieved profitable performance on mining temporal characteristics in user behavior. However, user preferences are changing over time and have not been fully exploited in e-commerce scenar
Publikováno v:
AAAI
Despite their widespread applications, deep neural networks often tend to overfit the training data. Here, we propose a measure called VECA (Variance of Eigenvalues of Covariance matrix of Activation matrix) and demonstrate that VECA is a good predic