Novel View Synthesis of Humans using Differentiable Rendering
Autor: | Rochette, Guillaume, Russell, Chris, Bowden, Richard |
---|---|
Rok vydání: | 2023 |
Předmět: | |
Druh dokumentu: | Working Paper |
DOI: | 10.1109/TBIOM.2022.3218903 |
Popis: | We present a new approach for synthesizing novel views of people in new poses. Our novel differentiable renderer enables the synthesis of highly realistic images from any viewpoint. Rather than operating over mesh-based structures, our renderer makes use of diffuse Gaussian primitives that directly represent the underlying skeletal structure of a human. Rendering these primitives gives results in a high-dimensional latent image, which is then transformed into an RGB image by a decoder network. The formulation gives rise to a fully differentiable framework that can be trained end-to-end. We demonstrate the effectiveness of our approach to image reconstruction on both the Human3.6M and Panoptic Studio datasets. We show how our approach can be used for motion transfer between individuals; novel view synthesis of individuals captured from just a single camera; to synthesize individuals from any virtual viewpoint; and to re-render people in novel poses. Code and video results are available at https://github.com/GuillaumeRochette/HumanViewSynthesis. Comment: Accepted at IEEE transactions on Biometrics, Behavior, and Identity Science, 10 pages, 11 figures. arXiv admin note: substantial text overlap with arXiv:2111.12731 |
Databáze: | arXiv |
Externí odkaz: |