A scalable species-based genetic algorithm for reinforcement learning problems
| Autor: | Anirudh Seth, Alexandros Nikou, Marios Daoutis |
|---|---|
| Rok vydání: | 2022 |
| Předmět: | |
| Zdroj: | The Knowledge Engineering Review. 37 |
| ISSN: | 1469-8005 0269-8889 |
| DOI: | 10.1017/s0269888922000042 |
| Popis: | Reinforcement Learning (RL) methods often rely on gradient estimates to learn an optimal policy for control problems. These expensive computations result in long training times, a poor rate of convergence, and sample inefficiency when applied to real-world problems with a large state and action space. Evolutionary Computation (EC)-based techniques offer a gradient-free apparatus to train a deep neural network for RL problems. In this work, we leverage the benefits of EC and propose a novel variant of genetic algorithm called SP-GA which utilizes a species-inspired weight initialization strategy and trains a population of deep neural networks, each estimating the Q-function for the RL problem. Efficient encoding of a neural network that utilizes less memory is also proposed which provides an intuitive mechanism to apply Gaussian mutations and single-point crossover. The results on Atari 2600 games outline comparable performance with gradient-based algorithms like Deep Q-Network (DQN), Asynchronous Advantage Actor Critic (A3C), and gradient-free algorithms like Evolution Strategy (ES) and simple Genetic Algorithm (GA) while requiring far fewer hyperparameters to train. The algorithm also improved certain Key Performance Indicators (KPIs) when applied to a Remote Electrical Tilt (RET) optimization task in the telecommunication domain. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|