Hindsight Experience Replay Improves Reinforcement Learning for Control of a MIMO Musculoskeletal Model of the Human Arm
Autor: | Douglas C. Crowder, Robert F. Kirsch, Jessica Abreu |
---|---|
Rok vydání: | 2021 |
Předmět: |
Computer science
Movement 0206 medical engineering MIMO education Biomedical Engineering Electric Stimulation Therapy 02 engineering and technology Kinematics 010501 environmental sciences Degrees of freedom (mechanics) musculoskeletal model 01 natural sciences Article functional electrical stimulation Upper Extremity Control theory Reinforcement learning Internal Medicine Functional electrical stimulation Humans Simulation Functional movement 0105 earth and related environmental sciences General Neuroscience Rehabilitation 020601 biomedical engineering Biomechanical Phenomena hindsight experience replay Arm Quality of Life actor-critic Hindsight bias |
Zdroj: | IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society |
ISSN: | 1558-0210 |
Popis: | High-level spinal cord injuries often result in paralysis of all four limbs, leading to decreased patient independence and quality of life. Coordinated functional electrical stimulation (FES) of paralyzed muscles can be used to restore some motor function in the upper extremity. To coordinate functional movements, FES controllers should be developed to exploit the complex characteristics of human movement and produce the intended movement kinematics and/or kinetics. Here, we demonstrate the ability of a controller trained using reinforcement learning to generate desired movements of a horizontal planar musculoskeletal model of the human arm with 2 degrees of freedom and 6 actuators. The controller is given information about the kinematics of the arm, but not the internal state of the actuators. In particular, we demonstrate that a technique called “hindsight experience replay” can improve controller performance while also decreasing controller training time. |
Databáze: | OpenAIRE |
Externí odkaz: |