Ego-Topo: Environment Affordances From Egocentric Video
| Autor: | Tushar Nagarajan, Kristen Grauman, Christoph Feichtenhofer, Yanghao Li |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
0209 industrial biotechnology business.industry Computer science Computer Vision and Pattern Recognition (cs.CV) Computer Science - Computer Vision and Pattern Recognition Representation (systemics) 02 engineering and technology Simultaneous localization and mapping Space (commercial competition) Visualization 020901 industrial engineering & automation Human–computer interaction Id ego and super-ego 0202 electrical engineering electronic engineering information engineering Task analysis 020201 artificial intelligence & image processing Artificial intelligence Topological map business Affordance |
| Zdroj: | CVPR |
| Popis: | First-person video naturally brings the use of a physical environment to the forefront, since it shows the camera wearer interacting fluidly in a space based on his intentions. However, current methods largely separate the observed actions from the persistent space itself. We introduce a model for environment affordances that is learned directly from egocentric video. The main idea is to gain a human-centric model of a physical space (such as a kitchen) that captures (1) the primary spatial zones of interaction and (2) the likely activities they support. Our approach decomposes a space into a topological map derived from first-person activity, organizing an ego-video into a series of visits to the different zones. Further, we show how to link zones across multiple related environments (e.g., from videos of multiple kitchens) to obtain a consolidated representation of environment functionality. On EPIC-Kitchens and EGTEA+, we demonstrate our approach for learning scene affordances and anticipating future actions in long-form video. Published in CVPR 2020, project page: http://vision.cs.utexas.edu/projects/ego-topo/ |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|