Zobrazeno 1 - 10
of 10
pro vyhledávání: '"Singh, Aaditya K"'
Autor:
Wang, Christopher, Yaari, Adam Uri, Singh, Aaditya K, Subramaniam, Vighnesh, Rosenfarb, Dana, DeWitt, Jan, Misra, Pranav, Madsen, Joseph R., Stone, Scellig, Kreiman, Gabriel, Katz, Boris, Cases, Ignacio, Barbu, Andrei
We present the Brain Treebank, a large-scale dataset of electrophysiological neural responses, recorded from intracranial probes while 10 subjects watched one or more Hollywood movies. Subjects watched on average 2.6 Hollywood movies, for an average
Externí odkaz:
http://arxiv.org/abs/2411.08343
Autor:
Singh, Aaditya K., Kocyigit, Muhammed Yusuf, Poulton, Andrew, Esiobu, David, Lomeli, Maria, Szilvasy, Gergely, Hupkes, Dieuwke
Hampering the interpretation of benchmark scores, evaluation data contamination has become a growing concern in the evaluation of LLMs, and an active area of research studies its effects. While evaluation data contamination is easily understood intui
Externí odkaz:
http://arxiv.org/abs/2411.03923
Data curation is commonly considered a "secret-sauce" for LLM training, with higher quality data usually leading to better LLM performance. Given the scale of internet-scraped corpora, data pruning has become a larger and larger focus. Specifically,
Externí odkaz:
http://arxiv.org/abs/2407.00434
Autor:
Madaan, Lovish, Singh, Aaditya K., Schaeffer, Rylan, Poulton, Andrew, Koyejo, Sanmi, Stenetorp, Pontus, Narang, Sharan, Hupkes, Dieuwke
Evaluation benchmarks are the cornerstone of measuring capabilities of large language models (LLMs), as well as driving progress in said capabilities. Originally designed to make claims about capabilities (or lack thereof) in fully pretrained models,
Externí odkaz:
http://arxiv.org/abs/2406.10229
In-context learning is a powerful emergent ability in transformer models. Prior work in mechanistic interpretability has identified a circuit element that may be critical for in-context learning -- the induction head (IH), which performs a match-and-
Externí odkaz:
http://arxiv.org/abs/2404.07129
Autor:
Singh, Aaditya K., Strouse, DJ
Tokenization, the division of input text into input tokens, is an often overlooked aspect of the large language model (LLM) pipeline and could be the source of useful or harmful inductive biases. Historically, LLMs have relied on byte pair encoding,
Externí odkaz:
http://arxiv.org/abs/2402.14903
Autor:
Yang, Yu, Singh, Aaditya K., Elhoushi, Mostafa, Mahmoud, Anas, Tirumala, Kushal, Gloeckle, Fabian, Rozière, Baptiste, Wu, Carole-Jean, Morcos, Ari S., Ardalani, Newsha
Code datasets, often collected from diverse and uncontrolled sources such as GitHub, potentially suffer from quality issues, thereby affecting the performance and training efficiency of Large Language Models (LLMs) optimized for code generation. Prev
Externí odkaz:
http://arxiv.org/abs/2312.02418
Autor:
Singh, Aaditya K., Chan, Stephanie C. Y., Moskovitz, Ted, Grant, Erin, Saxe, Andrew M., Hill, Felix
Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has provided a deeper understanding of how ICL emerges in transformers, e.g. through the lens of mecha
Externí odkaz:
http://arxiv.org/abs/2311.08360
Autor:
Moskovitz, Ted, Singh, Aaditya K., Strouse, DJ, Sandholm, Tuomas, Salakhutdinov, Ruslan, Dragan, Anca D., McAleer, Stephen
Large language models are typically aligned with human preferences by optimizing $\textit{reward models}$ (RMs) fitted to human feedback. However, human preferences are multi-faceted, and it is increasingly common to derive reward from a composition
Externí odkaz:
http://arxiv.org/abs/2310.04373
Effective communication requires adapting to the idiosyncrasies of each communicative context--such as the common ground shared with each partner. Humans demonstrate this ability to specialize to their audience in many contexts, such as the popular g
Externí odkaz:
http://arxiv.org/abs/2206.08349