Hierarchies of Planning and Reinforcement Learning for Robot Navigation
| Autor: | Jan Wöhlke, Herke van Hoof, Felix Schmitt |
|---|---|
| Přispěvatelé: | Amsterdam Machine Learning lab (IVI, FNWI) |
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Computer science business.industry Floor plan Machine Learning (cs.LG) Data modeling Task (project management) Computer Science - Robotics Task analysis Robot Reinforcement learning Artificial intelligence Markov decision process Motion planning business Robotics (cs.RO) |
| Zdroj: | ICRA 2021 IEEE International Conference on Robotics and Automation (ICRA 2021): May 31-June 4, 2021, Xi'an, China, 10682-10688 STARTPAGE=10682;ENDPAGE=10688;TITLE=2021 IEEE International Conference on Robotics and Automation (ICRA 2021) |
| Popis: | Solving robotic navigation tasks via reinforcement learning (RL) is challenging due to their sparse reward and long decision horizon nature. However, in many navigation tasks, high-level (HL) task representations, like a rough floor plan, are available. Previous work has demonstrated efficient learning by hierarchal approaches consisting of path planning in the HL representation and using sub-goals derived from the plan to guide the RL policy in the source task. However, these approaches usually neglect the complex dynamics and sub-optimal sub-goal-reaching capabilities of the robot during planning. This work overcomes these limitations by proposing a novel hierarchical framework that utilizes a trainable planning policy for the HL representation. Thereby robot capabilities and environment conditions can be learned utilizing collected rollout data. We specifically introduce a planning policy based on value iteration with a learned transition model (VI-RL). In simulated robotic navigation tasks, VI-RL results in consistent strong improvement over vanilla RL, is on par with vanilla hierarchal RL on single layouts but more broadly applicable to multiple layouts, and is on par with trainable HL path planning baselines except for a parking task with difficult non-holonomic dynamics where it shows marked improvements. Comment: 7 pages, 5 figures, 2021 IEEE International Conference on Robotics and Automation (ICRA), v2: DOI number added |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|