Controlling Level of Unconsciousness by Titrating Propofol with Deep Reinforcement Learning
| Autor: | Emery N. Brown, Marcus A. Badgeley, Gabriel Schamberg |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
0209 industrial biotechnology
Computer science Continuous infusion 02 engineering and technology Machine learning computer.software_genre 020901 industrial engineering & automation Pharmacokinetics Control theory Robustness (computer science) 0202 electrical engineering electronic engineering information engineering medicine Reinforcement learning Artificial neural network business.industry Deep learning Medication regimen Pharmacodynamics Pattern recognition (psychology) Anesthetic 020201 artificial intelligence & image processing Artificial intelligence business Propofol computer medicine.drug |
| Zdroj: | Artificial Intelligence in Medicine ISBN: 9783030591366 AIME |
| DOI: | 10.1007/978-3-030-59137-3_3 |
| Popis: | Reinforcement Learning (RL) can be used to fit a mapping from patient state to a medication regimen. Prior studies have used deterministic and value-based tabular learning to learn a propofol dose from an observed anesthetic state. Deep RL replaces the table with a deep neural network and has been used to learn medication regimens from registry databases. Here we perform the first application of deep RL to closed-loop control of anesthetic dosing in a simulated environment. We use the cross-entropy method to train a deep neural network to map an observed anesthetic state to a probability of infusing a fixed propofol dosage. During testing, we implement a deterministic policy that transforms the probability of infusion to a continuous infusion rate. The model is trained and tested on simulated pharmacokinetic/pharmacodynamic models with randomized parameters to ensure robustness to patient variability. The deep RL agent significantly outperformed a proportional-integral-derivative controller (median absolute performance error 1.7% ± 0.6 and 3.4% ± 1.2). Modeling continuous input variables instead of a table affords more robust pattern recognition and utilizes our prior domain knowledge. Deep RL learned a smooth policy with a natural interpretation to data scientists and anesthesia care providers alike. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|