ERRA: An Embodied Representation and Reasoning Architecture for Long-Horizon Language-Conditioned Manipulation Tasks
| Autor: | Chao Zhao, Shuai Yuan, Chunli Jiang, Junhao Cai, Hongyu Yu, Michael Yu Wang, Qifeng Chen |
|---|---|
| Rok vydání: | 2023 |
| Předmět: |
FOS: Computer and information sciences
Human-Computer Interaction Computer Science - Robotics Control and Optimization Artificial Intelligence Control and Systems Engineering Mechanical Engineering Biomedical Engineering Computer Vision and Pattern Recognition Robotics (cs.RO) Computer Science Applications |
| Zdroj: | IEEE Robotics and Automation Letters. 8:3230-3237 |
| ISSN: | 2377-3774 |
| DOI: | 10.1109/lra.2023.3265893 |
| Popis: | This letter introduces ERRA, an embodied learning architecture that enables robots to jointly obtain three fundamental capabilities (reasoning, planning, and interaction) for solving long-horizon language-conditioned manipulation tasks. ERRA is based on tightly-coupled probabilistic inferences at two granularity levels. Coarse-resolution inference is formulated as sequence generation through a large language model, which infers action language from natural language instruction and environment state. The robot then zooms to the fine-resolution inference part to perform the concrete action corresponding to the action language. Fine-resolution inference is constructed as a Markov decision process, which takes action language and environmental sensing as observations and outputs the action. The results of action execution in environments provide feedback for subsequent coarse-resolution reasoning. Such coarse-to-fine inference allows the robot to decompose and achieve long-horizon tasks interactively. In extensive experiments, we show that ERRA can complete various long-horizon manipulation tasks specified by abstract language instructions. We also demonstrate successful generalization to the novel but similar natural language instructions. Comment: Accepted to IEEE Robotics and Automation Letters (RA-L) |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|