Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time
| Autor: | Emre Aksan, Gerard Pons-Moll, Otmar Hilliges, Manuel Kaufmann, Yinghao Huang, Michael J. Black |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2018 |
| Předmět: |
FOS: Computer and information sciences
Artificial neural network business.industry Computer science Deep learning Computer Vision and Pattern Recognition (cs.CV) Computer Science - Computer Vision and Pattern Recognition 020207 software engineering 02 engineering and technology Computer Graphics and Computer-Aided Design Motion capture Graphics (cs.GR) Computer Science - Graphics Inertial measurement unit 0202 electrical engineering electronic engineering information engineering Code (cryptography) 020201 artificial intelligence & image processing Computer vision Artificial intelligence business |
| Zdroj: | ACM Transactions on Graphics, 37 (6) |
| Popis: | We demonstrate a novel deep neural network capable of reconstructing human full body pose in real-time from 6 Inertial Measurement Units (IMUs) worn on the user's body. In doing so, we address several difficult challenges. First, the problem is severely under-constrained as multiple pose parameters produce the same IMU orientations. Second, capturing IMU data in conjunction with ground-truth poses is expensive and difficult to do in many target application scenarios (e.g., outdoors). Third, modeling temporal dependencies through non-linear optimization has proven effective in prior work but makes real-time prediction infeasible. To address this important limitation, we learn the temporal pose priors using deep learning. To learn from sufficient data, we synthesize IMU data from motion capture datasets. A bi-directional RNN architecture leverages past and future information that is available at training time. At test time, we deploy the network in a sliding window fashion, retaining real time capabilities. To evaluate our method, we recorded DIP-IMU, a dataset consisting of $10$ subjects wearing 17 IMUs for validation in $64$ sequences with $330\,000$ time instants; this constitutes the largest IMU dataset publicly available. We quantitatively evaluate our approach on multiple datasets and show results from a real-time implementation. DIP-IMU and the code are available for research purposes. Comment: SIGGRAPH Asia 2018. First two authors contributed equally to this work. Project page: http://dip.is.tue.mpg.de/ |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|