Zobrazeno 1 - 10
of 30
pro vyhledávání: '"Khona, Mikail"'
Frontier AI systems are making transformative impacts across society, but such benefits are not without costs: models trained on web-scale datasets containing personal and private data raise profound concerns about data privacy and security. Language
Externí odkaz:
http://arxiv.org/abs/2406.14549
In-context learning is a powerful capability of certain machine learning models that arguably underpins the success of today's frontier AI models. However, in-context learning is critically limited to settings where the in-context distribution of int
Externí odkaz:
http://arxiv.org/abs/2406.12785
Autor:
Schaeffer, Rylan, Lecomte, Victor, Pai, Dhruv Bhandarkar, Carranza, Andres, Isik, Berivan, Unell, Alyssa, Khona, Mikail, Yerxa, Thomas, LeCun, Yann, Chung, SueYeon, Gromov, Andrey, Shwartz-Ziv, Ravid, Koyejo, Sanmi
Maximum Manifold Capacity Representations (MMCR) is a recent multi-view self-supervised learning (MVSSL) method that matches or surpasses other leading MVSSL methods. MMCR is intriguing because it does not fit neatly into any of the commonplace MVSSL
Externí odkaz:
http://arxiv.org/abs/2406.09366
Autor:
Luo, Xiaoliang, Rechardt, Akilles, Sun, Guangzhi, Nejad, Kevin K., Yáñez, Felipe, Yilmaz, Bati, Lee, Kangjoo, Cohen, Alexandra O., Borghesani, Valentina, Pashkov, Anton, Marinazzo, Daniele, Nicholas, Jonathan, Salatiello, Alessandro, Sucholutsky, Ilia, Minervini, Pasquale, Razavi, Sepehr, Rocca, Roberta, Yusifov, Elkhan, Okalova, Tereza, Gu, Nianlong, Ferianc, Martin, Khona, Mikail, Patil, Kaustubh R., Lee, Pui-Shee, Mata, Rui, Myers, Nicholas E., Bizley, Jennifer K, Musslick, Sebastian, Bilgin, Isil Poyraz, Niso, Guiomar, Ales, Justin M., Gaebler, Michael, Murty, N Apurva Ratan, Loued-Khenissi, Leyla, Behler, Anna, Hall, Chloe M., Dafflon, Jessica, Bao, Sherry Dongqi, Love, Bradley C.
Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offer a solution. LLMs trained on the vast scientific literature could pot
Externí odkaz:
http://arxiv.org/abs/2403.03230
Autor:
Schaeffer, Rylan, Zahedi, Nika, Khona, Mikail, Pai, Dhruv, Truong, Sang, Du, Yilun, Ostrow, Mitchell, Chandra, Sarthak, Carranza, Andres, Fiete, Ila Rani, Gromov, Andrey, Koyejo, Sanmi
Associative memory and probabilistic modeling are two fundamental topics in artificial intelligence. The first studies recurrent neural networks designed to denoise, complete and retrieve data, whereas the second studies learning and sampling from pr
Externí odkaz:
http://arxiv.org/abs/2402.10202
Autor:
Khona, Mikail, Okawa, Maya, Hula, Jan, Ramesh, Rahul, Nishi, Kento, Dick, Robert, Lubana, Ekdeep Singh, Tanaka, Hidenori
Stepwise inference protocols, such as scratchpads and chain-of-thought, help language models solve complex problems by decomposing them into a sequence of simpler subproblems. Despite the significant gain in performance achieved via these protocols,
Externí odkaz:
http://arxiv.org/abs/2402.07757
Work on deep learning-based models of grid cells suggests that grid cells generically and robustly arise from optimizing networks to path integrate, i.e., track one's spatial position by integrating self-velocity signals. In previous work, we challen
Externí odkaz:
http://arxiv.org/abs/2312.03954
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks
Transformers trained on huge text corpora exhibit a remarkable set of capabilities, e.g., performing basic arithmetic. Given the inherent compositional nature of language, one can expect the model to learn to compose these capabilities, potentially y
Externí odkaz:
http://arxiv.org/abs/2311.12997
Autor:
Schaeffer, Rylan, Khona, Mikail, Ma, Tzuhsuan, Eyzaguirre, Cristóbal, Koyejo, Sanmi, Fiete, Ila Rani
To solve the spatial problems of mapping, localization and navigation, the mammalian lineage has developed striking spatial representations. One important spatial representation is the Nobel-prize winning grid cells: neurons that represent self-locat
Externí odkaz:
http://arxiv.org/abs/2311.02316
Recurrent neural networks (RNNs) trained on compositional tasks can exhibit functional modularity, in which neurons can be clustered by activity similarity and participation in shared computational subtasks. Unlike brains, these RNNs do not exhibit a
Externí odkaz:
http://arxiv.org/abs/2310.07711