Visual Forecasting by Imitating Dynamics in Natural Sequences
| Autor: | William B. Shen, Kuo-Hao Zeng, De-An Huang, Min Sun, Juan Carlos Niebles |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2017 |
| Předmět: |
FOS: Computer and information sciences
Semantics (computer science) business.industry Computer science Computer Vision and Pattern Recognition (cs.CV) media_common.quotation_subject Frame (networking) Computer Science - Computer Vision and Pattern Recognition 02 engineering and technology 010501 environmental sciences 01 natural sciences Bottleneck Visualization Dynamic programming Feature (computer vision) 0202 electrical engineering electronic engineering information engineering Domain knowledge 020201 artificial intelligence & image processing Artificial intelligence business Imitation 0105 earth and related environmental sciences media_common |
| Zdroj: | ICCV |
| Popis: | We introduce a general framework for visual forecasting, which directly imitates visual sequences without additional supervision. As a result, our model can be applied at several semantic levels and does not require any domain knowledge or handcrafted features. We achieve this by formulating visual forecasting as an inverse reinforcement learning (IRL) problem, and directly imitate the dynamics in natural sequences from their raw pixel values. The key challenge is the high-dimensional and continuous state-action space that prohibits the application of previous IRL algorithms. We address this computational bottleneck by extending recent progress in model-free imitation with trainable deep feature representations, which (1) bypasses the exhaustive state-action pair visits in dynamic programming by using a dual formulation and (2) avoids explicit state sampling at gradient computation using a deep feature reparametrization. This allows us to apply IRL at scale and directly imitate the dynamics in high-dimensional continuous visual sequences from the raw pixel values. We evaluate our approach at three different level-of-abstraction, from low level pixels to higher level semantics: future frame generation, action anticipation, visual story forecasting. At all levels, our approach outperforms existing methods. 10 pages, 9 figures, accepted to ICCV 2017 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|