Tell me Dave: Context-sensitive grounding of natural language to manipulation instructions
| Autor: | Ashutosh Saxena, Jaeyong Sung, Kevin Lee, Dipendra Misra |
|---|---|
| Rok vydání: | 2015 |
| Předmět: |
Conditional random field
0209 industrial biotechnology Computer science media_common.quotation_subject Natural language understanding Context (language use) 02 engineering and technology computer.software_genre Task (project management) 020901 industrial engineering & automation Artificial Intelligence 0202 electrical engineering electronic engineering information engineering Electrical and Electronic Engineering Function (engineering) media_common Sequence business.industry Applied Mathematics Mechanical Engineering Context (computing) Linguistics Modeling and Simulation Robot 020201 artificial intelligence & image processing Artificial intelligence business computer Natural language processing Software Natural language |
| Zdroj: | Robotics: Science and Systems |
| ISSN: | 1741-3176 0278-3649 |
| DOI: | 10.1177/0278364915602060 |
| Popis: | It is important for a robot to be able to interpret natural language commands given by a human. In this paper, we consider performing a sequence of mobile manipulation tasks with instructions described in natural language. Given a new environment, even a simple task such as boiling water would be performed quite differently depending on the presence, location and state of the objects. We start by collecting a dataset of task descriptions in free-form natural language and the corresponding grounded task-logs of the tasks performed in an online robot simulator. We then build a library of verb–environment instructions that represents the possible instructions for each verb in that environment, these may or may not be valid for a different environment and task context. We present a model that takes into account the variations in natural language and ambiguities in grounding them to robotic instructions with appropriate environment context and task constraints. Our model also handles incomplete or noisy natural language instructions. It is based on an energy function that encodes such properties in a form isomorphic to a conditional random field. We evaluate our model on tasks given in a robotic simulator and show that it successfully outperforms the state of the art with 61.8% accuracy. We also demonstrate a grounded robotic instruction sequence on a PR2 robot using the Learning from Demonstration approach. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|