Zobrazeno 1 - 10
of 69
pro vyhledávání: '"Karakida, Ryo"'
Autor:
Takase, Tomoumi, Karakida, Ryo
While data augmentation (DA) is generally applied to input data, several studies have reported that applying DA to hidden layers in neural networks, i.e., feature augmentation, can improve performance. However, in previous studies, the layers to whic
Externí odkaz:
http://arxiv.org/abs/2408.13426
Transformers have established themselves as the leading neural network model in natural language processing and are increasingly foundational in various domains. In vision, the MLP-Mixer model has demonstrated competitive performance, suggesting that
Externí odkaz:
http://arxiv.org/abs/2406.12220
The self-attention mechanism prevails in modern machine learning. It has an interesting functionality of adaptively selecting tokens from an input sequence by modulating the degree of attention localization, which many researchers speculate is the ba
Externí odkaz:
http://arxiv.org/abs/2402.02098
Autor:
Ishikawa, Satoki, Karakida, Ryo
Second-order optimization has been developed to accelerate the training of deep neural networks and it is being applied to increasingly larger-scale models. In this study, towards training on further larger scales, we identify a specific parameteriza
Externí odkaz:
http://arxiv.org/abs/2312.12226
Autor:
Hayase, Tomohiro, Karakida, Ryo
Multi-layer perceptron (MLP) is a fundamental component of deep learning, and recent MLP-based architectures, especially the MLP-Mixer, have achieved significant empirical success. Nevertheless, our understanding of why and how the MLP-Mixer outperfo
Externí odkaz:
http://arxiv.org/abs/2306.01470
Autor:
Ota, Toshihiro, Karakida, Ryo
Hopfield networks and Boltzmann machines (BMs) are fundamental energy-based neural network models. Recent studies on modern Hopfield networks have broaden the class of energy functions and led to a unified perspective on general Hopfield networks inc
Externí odkaz:
http://arxiv.org/abs/2212.04692
Gradient regularization (GR) is a method that penalizes the gradient norm of the training loss during training. While some studies have reported that GR can improve generalization performance, little attention has been paid to it from the algorithmic
Externí odkaz:
http://arxiv.org/abs/2210.02720
A biological neural network in the cortex forms a neural field. Neurons in the field have their own receptive fields, and connection weights between two neurons are random but highly correlated when they are in close proximity in receptive fields. In
Externí odkaz:
http://arxiv.org/abs/2202.05254
Autor:
Karakida, Ryo, Akaho, Shotaro
Publikováno v:
Published in ICLR 2022
Sequential training from task to task is becoming one of the major objects in deep learning applications such as continual learning and transfer learning. Nevertheless, it remains unclear under what conditions the trained model's performance improves
Externí odkaz:
http://arxiv.org/abs/2112.01653
Data augmentation is widely used for machine learning; however, an effective method to apply data augmentation has not been established even though it includes several factors that should be tuned carefully. One such factor is sample suitability, whi
Externí odkaz:
http://arxiv.org/abs/2010.15434