Zobrazeno 1 - 10
of 123
pro vyhledávání: '"Whittington, James"'
Autor:
Dorrell, Will, Hsu, Kyle, Hollingsworth, Luke, Lee, Jin Hwa, Wu, Jiajun, Finn, Chelsea, Latham, Peter E, Behrens, Tim EJ, Whittington, James CR
Why do biological and artificial neurons sometimes modularise, each encoding a single meaningful variable, and sometimes entangle their representation of many variables? In this work, we develop a theory of when biologically inspired representations
Externí odkaz:
http://arxiv.org/abs/2410.06232
Autor:
Tokpanov, Yury, Millidge, Beren, Glorioso, Paolo, Pilault, Jonathan, Ibrahim, Adam, Whittington, James, Anthony, Quentin
The size of large language models (LLMs) has scaled dramatically in recent years and their computational and data requirements have surged correspondingly. State-of-the-art language models, even at relatively smaller sizes, typically require training
Externí odkaz:
http://arxiv.org/abs/2406.01981
Autor:
Glorioso, Paolo, Anthony, Quentin, Tokpanov, Yury, Whittington, James, Pilault, Jonathan, Ibrahim, Adam, Millidge, Beren
In this technical report, we present Zamba, a novel 7B SSM-transformer hybrid model which achieves competitive performance against leading open-weight models at a comparable scale. Zamba is trained on 1T tokens from openly available datasets and is t
Externí odkaz:
http://arxiv.org/abs/2405.16712
In disentangled representation learning, a model is asked to tease apart a dataset's underlying sources of variation and represent them independently of one another. Since the model is provided with no ground truth information about these sources, in
Externí odkaz:
http://arxiv.org/abs/2305.18378
Neurons in the brain are often finely tuned for specific task variables. Moreover, such disentangled representations are highly sought after in machine learning. Here we mathematically prove that simple biological constraints on neurons, namely nonne
Externí odkaz:
http://arxiv.org/abs/2210.01768
To afford flexible behaviour, the brain must build internal representations that mirror the structure of variables in the external world. For example, 2D space obeys rules: the same set of actions combine in the same way everywhere (step north, then
Externí odkaz:
http://arxiv.org/abs/2209.15563
Learning and interpreting the structure of the environment is an innate feature of biological systems, and is integral to guiding flexible behaviours for evolutionary viability. The concept of a cognitive map has emerged as one of the leading metapho
Externí odkaz:
http://arxiv.org/abs/2202.01682
Many deep neural network architectures loosely based on brain networks have recently been shown to replicate neural firing patterns observed in the brain. One of the most exciting and promising novel architectures, the Transformer neural network, was
Externí odkaz:
http://arxiv.org/abs/2112.04035
Autor:
Whittington, James C. R., Kabra, Rishabh, Matthey, Loic, Burgess, Christopher P., Lerchner, Alexander
Learning structured representations of visual scenes is currently a major bottleneck to bridging perception with reasoning. While there has been exciting progress with slot-based models, which learn to segment scenes into sets of objects, learning co
Externí odkaz:
http://arxiv.org/abs/2107.11153
Autor:
Stevens, Philip W., Dutka-Gianelli, Jynessa, Nagid, Eric J., Trotter, Alexis A., Johnson, Kevin G., Tuten, Travis, Whittington, James A.
Publikováno v:
Estuaries and Coasts, 2020 Mar 01. 43(2), 396-439.
Externí odkaz:
https://www.jstor.org/stable/48728507
