Optimal structure of metaplasticity for adaptive learning
| Autor: | Alireza Soltani, Peyman Khorsand |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2017 |
| Předmět: |
0301 basic medicine
Statistical methods Physiology Computer science Distributed computing Social Sciences computer.software_genre Nervous System Synaptic Transmission Synapse Learning and Memory 0302 clinical medicine Medicine and Health Sciences Psychology Probability Estimation skin and connective tissue diseases lcsh:QH301-705.5 media_common Neuronal Plasticity Ecology Noise (signal processing) Simulation and Modeling Synaptic efficacy Brain Adaptation Physiological Electrophysiology Monte Carlo method Physical sciences Computational Theory and Mathematics Modeling and Simulation Engineering and Technology Adaptive learning Anatomy Research Article Optimization media_common.quotation_subject Models Neurological Neurophysiology Statistics (mathematics) Machine learning Adaptability Bottleneck 03 medical and health sciences Cellular and Molecular Neuroscience Developmental Neuroscience Reward Control theory Metaplasticity Genetics Learning Humans Computer Simulation Adaptation (computer science) Molecular Biology Ecology Evolution Behavior and Systematics Structure (mathematical logic) Models Statistical business.industry Mechanism (biology) fungi Cognitive Psychology Biology and Life Sciences Noise Reduction Research and analysis methods Range (mathematics) 030104 developmental biology lcsh:Biology (General) Cellular Neuroscience Synapses Signal Processing Synaptic plasticity Mathematical and statistical techniques Cognitive Science Artificial intelligence sense organs Nerve Net business computer Mathematics 030217 neurology & neurosurgery Neuroscience Synaptic Plasticity |
| Zdroj: | PLoS Computational Biology, Vol 13, Iss 6, p e1005630 (2017) PLoS Computational Biology |
| ISSN: | 1553-7358 |
| Popis: | Learning from reward feedback in a changing environment requires a high degree of adaptability, yet the precise estimation of reward information demands slow updates. In the framework of estimating reward probability, here we investigated how this tradeoff between adaptability and precision can be mitigated via metaplasticity, i.e. synaptic changes that do not always alter synaptic efficacy. Using the mean-field and Monte Carlo simulations we identified ‘superior’ metaplastic models that can substantially overcome the adaptability-precision tradeoff. These models can achieve both adaptability and precision by forming two separate sets of meta-states: reservoirs and buffers. Synapses in reservoir meta-states do not change their efficacy upon reward feedback, whereas those in buffer meta-states can change their efficacy. Rapid changes in efficacy are limited to synapses occupying buffers, creating a bottleneck that reduces noise without significantly decreasing adaptability. In contrast, more-populated reservoirs can generate a strong signal without manifesting any observable plasticity. By comparing the behavior of our model and a few competing models during a dynamic probability estimation task, we found that superior metaplastic models perform close to optimally for a wider range of model parameters. Finally, we found that metaplastic models are robust to changes in model parameters and that metaplastic transitions are crucial for adaptive learning since replacing them with graded plastic transitions (transitions that change synaptic efficacy) reduces the ability to overcome the adaptability-precision tradeoff. Overall, our results suggest that ubiquitous unreliability of synaptic changes evinces metaplasticity that can provide a robust mechanism for mitigating the tradeoff between adaptability and precision and thus adaptive learning. Author summary Successful learning from our experience and feedback from the environment requires that the reward value assigned to a given option or action to be updated by a precise amount after each feedback. In the standard model for reward-based learning known as reinforcement learning, the learning rates determine the strength of such update. A large learning rate allows fast update of values (large adaptability) but introduces noise (small precision), whereas a small learning rate does the opposite. Thus, learning seems to be bounded by a tradeoff between adaptability and precision. Here, we asked whether there are synaptic mechanisms that are capable of adjusting the brain’s level of plasticity according to reward statistics, and, therefore, allow the learning process to be adaptive. We showed that metaplasticity, changes in the synaptic state that shape future synaptic modifications without any observable changes in the strength of synapses, could provide such a mechanism and furthermore, identified the optimal structure of such metaplasticity. We propose that metaplasticity, which sometimes causes no observable changes in behavior and thus could be perceived as a lack of learning, can provide a robust mechanism for adaptive learning. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|