Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination
| Autor: | Matthew Tancik, Noah Snavely, Jonathan T. Barron, Ben Mildenhall, Pratul P. Srinivasan, Richard Tucker |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
0209 industrial biotechnology Ground truth Panorama Global illumination business.industry Computer science Computer Vision and Pattern Recognition (cs.CV) Computer Science - Computer Vision and Pattern Recognition 020207 software engineering Volume rendering 02 engineering and technology Solid modeling Real image Graphics (cs.GR) 020901 industrial engineering & automation Computer Science - Graphics 0202 electrical engineering electronic engineering information engineering Computer vision Specular reflection Artificial intelligence business |
| Zdroj: | CVPR |
| DOI: | 10.48550/arxiv.2003.08367 |
| Popis: | We present a deep learning solution for estimating the incident illumination at any 3D location within a scene from an input narrow-baseline stereo image pair. Previous approaches for predicting global illumination from images either predict just a single illumination for the entire scene, or separately estimate the illumination at each 3D location without enforcing that the predictions are consistent with the same 3D scene. Instead, we propose a deep learning model that estimates a 3D volumetric RGBA model of a scene, including content outside the observed field of view, and then uses standard volume rendering to estimate the incident illumination at any 3D location within that volume. Our model is trained without any ground truth 3D data and only requires a held-out perspective view near the input stereo pair and a spherical panorama taken within each scene as supervision, as opposed to prior methods for spatially-varying lighting estimation, which require ground truth scene geometry for training. We demonstrate that our method can predict consistent spatially-varying lighting that is convincing enough to plausibly relight and insert highly specular virtual objects into real images. Comment: CVPR 2020. Project page: https://people.eecs.berkeley.edu/~pratul/lighthouse/ [Updates: typos corrected] |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|