Zobrazeno 1 - 2
of 2
pro vyhledávání: '"Dai, Zihang"'
We revisit the design choices in Transformers, and propose methods to address their weaknesses in handling long sequences. First, we propose a simple layer named gated attention unit, which allows the use of a weaker single-head attention with minima
Externí odkaz:
https://explore.openaire.eu/search/publication?articleId=doi_dedup___::c37381123e5ac2b3015334b82374a5d6
Large Transformer models have been central to recent advances in natural language processing. The training and inference costs of these models, however, have grown rapidly and become prohibitively expensive. Here we aim to reduce the costs of Transfo
Externí odkaz:
https://explore.openaire.eu/search/publication?articleId=doi_dedup___::22a2f12c46d614f2cc6271e50eeb14fb