Limitation of capsule networks
| Autor: | Antonio Jose Rodríguez-Sánchez, David Peer, Sebastian Stabinger |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning business.industry Computer science Distributed computing Deep learning Machine Learning (stat.ML) 02 engineering and technology Object (computer science) 01 natural sciences Machine Learning (cs.LG) Artificial Intelligence Statistics - Machine Learning 0103 physical sciences Signal Processing 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Computer Vision and Pattern Recognition Artificial intelligence Routing (electronic design automation) 010306 general physics business Software |
| Zdroj: | Pattern Recognition Letters |
| ISSN: | 0167-8655 |
| DOI: | 10.1016/j.patrec.2021.01.017 |
| Popis: | A recently proposed method in deep learning groups multiple neurons to capsules such that each capsule represents an object or part of an object. Routing algorithms route the output of capsules from lower-level layers to upper-level layers. In this paper, we prove that state-of-the-art routing procedures decrease the expressivity of capsule networks. More precisely, it is shown that EM-routing and routing-by-agreement prevent capsule networks from distinguishing inputs and their negative counterpart. Therefore, only symmetric functions can be expressed by capsule networks, and it can be concluded that they are not universal approximators. We also theoretically motivate and empirically show that this limitation affects the training of deep capsule networks negatively. Therefore, we present an incremental improvement for state-of-the-art routing algorithms that solves the aforementioned limitation and stabilizes the training of capsule networks. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect