Zobrazeno 1 - 4
of 4
pro vyhledávání: '"Zuo, Chunsheng"'
Transformers with causal attention can solve tasks that require positional information without using positional encodings. In this work, we propose and investigate a new hypothesis about how positional information can be stored without using explicit
Externí odkaz:
http://arxiv.org/abs/2501.00073
Autor:
Zuo, Chunsheng, Guerzhoy, Michael
As we show in this paper, the prediction for output token $n+1$ of Transformer architectures without one of the mechanisms of positional encodings and causal attention is invariant to permutations of input tokens $1, 2, ..., n-1$. Usually, both mecha
Externí odkaz:
http://arxiv.org/abs/2402.05969
Publikováno v:
Transactions of the Chinese Society of Agricultural Engineering. 2017, Vol. 33 Issue 1, p8-16. 9p.
Autor:
Zhu Ruixiang1, Li Chengxin1, Cheng Yang1, Yan Xiaoli1 zrxjdxy2006@sohu.com, Li Jiang1, Shi Yanpeng1, Ge Shiqiang1
Publikováno v:
Transactions of the Chinese Society of Agricultural Engineering. Sep2014, Vol. 30 Issue 18, p47-54. 8p.