Zobrazeno 1 - 10
of 22
pro vyhledávání: '"Boopathy, Akhilan"'
Sequential learning in deep models often suffers from challenges such as catastrophic forgetting and loss of plasticity, largely due to the permutation dependence of gradient-based algorithms, where the order of training data impacts the learning out
Externí odkaz:
http://arxiv.org/abs/2410.22695
Autor:
Hwang, Jaedong, Cheung, Brian, Hong, Zhang-Wei, Boopathy, Akhilan, Agrawal, Pulkit, Fiete, Ila
Highly performant large-scale pre-trained models promise to also provide a valuable foundation for learning specialized tasks, by fine-tuning the model to the desired task. By starting from a good general-purpose model, the goal is to achieve both sp
Externí odkaz:
http://arxiv.org/abs/2410.21582
Autor:
Boopathy, Akhilan, Fiete, Ila
As neural networks continue to grow in size but datasets might not, it is vital to understand how much performance improvement can be expected: is it more important to scale network size or data volume? Thus, neural network scaling laws, which charac
Externí odkaz:
http://arxiv.org/abs/2409.05782
Modular neural networks outperform nonmodular neural networks on tasks ranging from visual question answering to robotics. These performance improvements are thought to be due to modular networks' superior ability to model the compositional and combi
Externí odkaz:
http://arxiv.org/abs/2409.05780
Much research in machine learning involves finding appropriate inductive biases (e.g. convolutional neural networks, momentum-based optimizers, transformers) to promote generalization on tasks. However, quantification of the amount of inductive bias
Externí odkaz:
http://arxiv.org/abs/2406.15941
State estimation is crucial for the performance and safety of numerous robotic applications. Among the suite of estimation techniques, particle filters have been identified as a powerful solution due to their non-parametric nature. Yet, in high-dimen
Externí odkaz:
http://arxiv.org/abs/2404.13698
Deep reinforcement learning methods exhibit impressive performance on a range of tasks but still struggle on hard exploration tasks in large environments with sparse rewards. To address this, intrinsic rewards can be generated using forward model pre
Externí odkaz:
http://arxiv.org/abs/2310.17537
Animals and robots navigate through environments by building and refining maps of space. These maps enable functions including navigation back to home, planning, search and foraging. Here, we use observations from neuroscience, specifically the obser
Externí odkaz:
http://arxiv.org/abs/2307.05793
The measure of a machine learning algorithm is the difficulty of the tasks it can perform, and sufficiently difficult tasks are critical drivers of strong machine learning models. However, quantifying the generalization difficulty of machine learning
Externí odkaz:
http://arxiv.org/abs/2305.01034
Autor:
Schaeffer, Rylan, Khona, Mikail, Robertson, Zachary, Boopathy, Akhilan, Pistunova, Kateryna, Rocks, Jason W., Fiete, Ila Rani, Koyejo, Oluwasanmi
Double descent is a surprising phenomenon in machine learning, in which as the number of model parameters grows relative to the number of data, test error drops as models grow ever larger into the highly overparameterized (data undersampled) regime.
Externí odkaz:
http://arxiv.org/abs/2303.14151