| Autor: |
Hafez, Muhammad Burhan, Immisch, Tilman, Weber, Tom, Wermter, Stefan |
| Rok vydání: |
2023 |
| Předmět: |
|
| Zdroj: |
Frontiers in Neurorobotics 17:1127642 (2023) |
| Druh dokumentu: |
Working Paper |
| DOI: |
10.3389/fnbot.2023.1127642 |
| Popis: |
Deep Reinforcement Learning agents often suffer from catastrophic forgetting, forgetting previously found solutions in parts of the input space when training on new data. Replay Memories are a common solution to the problem, decorrelating and shuffling old and new training samples. They naively store state transitions as they come in, without regard for redundancy. We introduce a novel cognitive-inspired replay memory approach based on the Grow-When-Required (GWR) self-organizing network, which resembles a map-based mental model of the world. Our approach organizes stored transitions into a concise environment-model-like network of state-nodes and transition-edges, merging similar samples to reduce the memory size and increase pair-wise distance among samples, which increases the relevancy of each sample. Overall, our paper shows that map-based experience replay allows for significant memory reduction with only small performance decreases. |
| Databáze: |
arXiv |
| Externí odkaz: |
|
