Neural-PBIR Reconstruction of Shape, Material, and Illumination

Autor:	Sun, Cheng, Cai, Guangyan, Li, Zhengqin, Yan, Kai, Zhang, Cheng, Marshall, Carl, Huang, Jia-Bin, Zhao, Shuang, Dong, Zhao
Rok vydání:	2023
Předmět:	Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	Reconstructing the shape and spatially varying surface appearances of a physical-world object as well as its surrounding illumination based on 2D images (e.g., photographs) of the object has been a long-standing problem in computer vision and graphics. In this paper, we introduce an accurate and highly efficient object reconstruction pipeline combining neural based object reconstruction and physics-based inverse rendering (PBIR). Our pipeline firstly leverages a neural SDF based shape reconstruction to produce high-quality but potentially imperfect object shape. Then, we introduce a neural material and lighting distillation stage to achieve high-quality predictions for material and illumination. In the last stage, initialized by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction of object shape, material, and illumination. Experimental results demonstrate our pipeline significantly outperforms existing methods quality-wise and performance-wise. Comment: ICCV 2023. Project page at https://neural-pbir.github.io/ Update Stanford-ORB results
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2304.13445 Zobrazit plný text záznamu View this record from Arxiv