Practical Face Reconstruction via Differentiable Ray Tracing
| Autor: | Cedric Thebault, Junghyun Ahn, Gaurav Bharaj, Abdallah Dib, Philippe-Henri Gosselin, Louis Chevallier, Marco Romeo |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Monocular 65D19 68U05 Computer science business.industry Computer Vision and Pattern Recognition (cs.CV) I.4.5 I.3.7 I.4.8 Computer Science - Computer Vision and Pattern Recognition Parameterized complexity Computer Graphics and Computer-Aided Design Graphics (cs.GR) Computer Science - Graphics Face (geometry) Computer vision Ray tracing (graphics) Differentiable function Artificial intelligence Specular reflection business Light stage Parametric statistics |
| Popis: | We present a differentiable ray-tracing based novel face reconstruction approach where scene attributes - 3D geometry, reflectance (diffuse, specular and roughness), pose, camera parameters, and scene illumination - are estimated from unconstrained monocular images. The proposed method models scene illumination via a novel, parameterized virtual light stage, which in-conjunction with differentiable ray-tracing, introduces a coarse-to-fine optimization formulation for face reconstruction. Our method can not only handle unconstrained illumination and self-shadows conditions, but also estimates diffuse and specular albedos. To estimate the face attributes consistently and with practical semantics, a two-stage optimization strategy systematically uses a subset of parametric attributes, where subsequent attribute estimations factor those previously estimated. For example, self-shadows estimated during the first stage, later prevent its baking into the personalized diffuse and specular albedos in the second stage. We show the efficacy of our approach in several real-world scenarios, where face attributes can be estimated even under extreme illumination conditions. Ablation studies, analyses and comparisons against several recent state-of-the-art methods show improved accuracy and versatility of our approach. With consistent face attributes reconstruction, our method leads to several style -- illumination, albedo, self-shadow -- edit and transfer applications, as discussed in the paper. 16 pages, 14 figures |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|