Zobrazeno 1 - 4
of 4
pro vyhledávání: '"Guo, Zifan Carl"'
When a neural network can learn multiple distinct algorithms to solve a task, how does it "choose" between them during training? To approach this question, we take inspiration from ecology: when multiple species coexist, they eventually reach an equi
Externí odkaz:
http://arxiv.org/abs/2405.17420
Autor:
Ho, Anson, Besiroglu, Tamay, Erdil, Ege, Owen, David, Rahman, Robi, Guo, Zifan Carl, Atkinson, David, Thompson, Neil, Sevilla, Jaime
We investigate the rate at which algorithms for pre-training language models have improved since the advent of deep learning. Using a dataset of over 200 language model evaluations on Wikitext and Penn Treebank spanning 2012-2023, we find that the co
Externí odkaz:
http://arxiv.org/abs/2403.05812
Autor:
Michaud, Eric J., Liao, Isaac, Lad, Vedang, Liu, Ziming, Mudide, Anish, Loughridge, Chloe, Guo, Zifan Carl, Kheirkhah, Tara Rezaei, Vukelić, Mateja, Tegmark, Max
We present MIPS, a novel method for program synthesis based on automated mechanistic interpretability of neural networks trained to perform the desired task, auto-distilling the learned algorithm into Python code. We test MIPS on a benchmark of 62 al
Externí odkaz:
http://arxiv.org/abs/2402.05110
Autor:
Gurnee, Wes, Horsley, Theo, Guo, Zifan Carl, Kheirkhah, Tara Rezaei, Sun, Qinyi, Hathaway, Will, Nanda, Neel, Bertsimas, Dimitris
A basic question within the emerging field of mechanistic interpretability is the degree to which neural networks learn the same underlying mechanisms. In other words, are neural mechanisms universal across different models? In this work, we study th
Externí odkaz:
http://arxiv.org/abs/2401.12181