Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimization

Autor:	Kim, Hyeonah, Kim, Minsu, Ahn, Sungsoo, Park, Jinkyoo
Rok vydání:	2023
Předmět:	Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Deep reinforcement learning (DRL) has significantly advanced the field of combinatorial optimization (CO). However, its practicality is hindered by the necessity for a large number of reward evaluations, especially in scenarios involving computationally intensive function assessments. To enhance the sample efficiency, we propose a simple but effective method, called symmetric replay training (SRT), which can be easily integrated into various DRL methods. Our method leverages high-reward samples to encourage exploration of the under-explored symmetric regions without additional online interactions - free. Through replay training, the policy is trained to maximize the likelihood of the symmetric trajectories of discovered high-rewarded samples. Experimental results demonstrate the consistent improvement of our method in sample efficiency across diverse DRL methods applied to real-world tasks, such as molecular optimization and hardware design. Comment: International Conference on Machine Learning
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2306.01276 Zobrazit plný text záznamu View this record from Arxiv