Zobrazeno 1 - 10
of 17 385
pro vyhledávání: '"Weight space"'
Weight space learning aims to extract information about a neural network, such as its training dataset or generalization error. Recent approaches learn directly from model weights, but this presents many challenges as weights are high-dimensional and
Externí odkaz:
http://arxiv.org/abs/2410.10811
Efficiently learning a sequence of related tasks, such as in continual learning, poses a significant challenge for neural nets due to the delicate trade-off between catastrophic forgetting and loss of plasticity. We address this challenge with a grou
Externí odkaz:
http://arxiv.org/abs/2410.06800
Autor:
Dravid, Amil, Gandelsman, Yossi, Wang, Kuan-Chieh, Abdal, Rameen, Wetzstein, Gordon, Efros, Alexei A., Aberman, Kfir
We investigate the space of weights spanned by a large collection of customized diffusion models. We populate this space by creating a dataset of over 60,000 models, each of which is a base model fine-tuned to insert a different person's visual ident
Externí odkaz:
http://arxiv.org/abs/2406.09413
Learning representations of well-trained neural network models holds the promise to provide an understanding of the inner workings of those models. However, previous work has either faced limitations when processing larger networks or was task-specif
Externí odkaz:
http://arxiv.org/abs/2406.09997
Autor:
Shamsian, Aviv, Navon, Aviv, Zhang, David W., Zhang, Yan, Fetaya, Ethan, Chechik, Gal, Maron, Haggai
Learning in deep weight spaces (DWS), where neural networks process the weights of other neural networks, is an emerging research direction, with applications to 2D and 3D neural fields (INRs, NeRFs), as well as making inferences about other types of
Externí odkaz:
http://arxiv.org/abs/2402.04081
Autor:
Koyun, Onur Can, Töreyin, Behçet Uğur
Deep learning models like Transformers and Convolutional Neural Networks (CNNs) have revolutionized various domains, but their parameter-intensive nature hampers deployment in resource-constrained settings. In this paper, we introduce a novel concept
Externí odkaz:
http://arxiv.org/abs/2401.16438
Permutation symmetries of deep networks make basic operations like model merging and similarity estimation challenging. In many cases, aligning the weights of the networks, i.e., finding optimal permutations between their weights, is necessary. Unfor
Externí odkaz:
http://arxiv.org/abs/2310.13397
Autor:
Ninalga, Dean
Recent work suggests that interpolating between the weights of two specialized language models can transfer knowledge between tasks in a way that multi-task learning cannot. However, very few have explored interpolation between more than two models,
Externí odkaz:
http://arxiv.org/abs/2307.03506
Research on neural networks has focused on understanding a single model trained on a single dataset. However, relatively little is known about the relationships between different models, particularly those trained or tested on different datasets. We
Externí odkaz:
http://arxiv.org/abs/2302.04863
Implicit neural fields, typically encoded by a multilayer perceptron (MLP) that maps from coordinates (e.g., xyz) to signals (e.g., signed distances), have shown remarkable promise as a high-fidelity and compact representation. However, the lack of a
Externí odkaz:
http://arxiv.org/abs/2303.17015