Zobrazeno 1 - 2
of 2
pro vyhledávání: '"Tuddenham, Mark"'
The optimisation of neural networks can be sped up by orthogonalising the gradients before the optimisation step, ensuring the diversification of the learned representations. We orthogonalise the gradients of the layer's components/filters with respe
Externí odkaz:
http://arxiv.org/abs/2202.07052
Publikováno v:
OPT2020: 12th Annual Workshop on Optimization for Machine Learning
Classification problems using deep learning have been shown to have a high-curvature subspace in the loss landscape equal in dimension to the number of classes. Moreover, this subspace corresponds to the subspace spanned by the logit gradients for ea
Externí odkaz:
http://arxiv.org/abs/2012.01938