The asymmetric learning rates of murine exploratory behavior in sparse reward environments
| Autor: | Yuji Morimoto, Kuniaki Satori, Hiroyuki Ohta, Toshiaki Ishizuka, Masashi Arake, Yu Takarada, Tatsuji Takahashi |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
0209 industrial biotechnology
Exploit Computer science Cognitive Neuroscience Perseveration 02 engineering and technology Multi-armed bandit Mice 020901 industrial engineering & automation Reward Artificial Intelligence 0202 electrical engineering electronic engineering information engineering medicine Reinforcement learning Animals Animal behavior Probability Action (philosophy) Differential learning Exploratory Behavior 020201 artificial intelligence & image processing medicine.symptom Reinforcement Psychology psychological phenomena and processes Algorithms Cognitive psychology |
| Zdroj: | Neural networks : the official journal of the International Neural Network Society. 143 |
| ISSN: | 1879-2782 |
| Popis: | Goal-oriented behaviors of animals can be modeled by reinforcement learning algorithms. Such algorithms predict future outcomes of selected actions utilizing action values and updating those values in response to the positive and negative outcomes. In many models of animal behavior, the action values are updated symmetrically based on a common learning rate, that is, in the same way for both positive and negative outcomes. However, animals in environments with scarce rewards may have uneven learning rates. To investigate the asymmetry in learning rates in reward and non-reward, we analyzed the exploration behavior of mice in five-armed bandit tasks using a Q-learning model with differential learning rates for positive and negative outcomes. The positive learning rate was significantly higher in a scarce reward environment than in a rich reward environment, and conversely, the negative learning rate was significantly lower in the scarce environment. The positive to negative learning rate ratio was about 10 in the scarce environment and about 2 in the rich environment. This result suggests that when the reward probability was low, the mice tend to ignore failures and exploit the rare rewards. Computational modeling analysis revealed that the increased learning rates ratio could cause an overestimation of and perseveration on rare-rewarding events, increasing total reward acquisition in the scarce environment but disadvantaging impartial exploration. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect