Neural Radiance Field-Inspired Depth Map Refinement for Accurate Multi-View Stereo

Autor: Shintaro Ito, Kanta Miura, Koichi Ito, Takafumi Aoki
Jazyk: angličtina
Rok vydání: 2024
Předmět:
Zdroj: Journal of Imaging, Vol 10, Iss 3, p 68 (2024)
Druh dokumentu: article
ISSN: 2313-433X
DOI: 10.3390/jimaging10030068
Popis: In this paper, we propose a method to refine the depth maps obtained by Multi-View Stereo (MVS) through iterative optimization of the Neural Radiance Field (NeRF). MVS accurately estimates the depths on object surfaces, and NeRF accurately estimates the depths at object boundaries. The key ideas of the proposed method are to combine MVS and NeRF to utilize the advantages of both in depth map estimation and to use NeRF for depth map refinement. We also introduce a Huber loss into the NeRF optimization to improve the accuracy of the depth map refinement, where the Huber loss reduces the estimation error in the radiance fields by placing constraints on errors larger than a threshold. Through a set of experiments using the Redwood-3dscan dataset and the DTU dataset, which are public datasets consisting of multi-view images, we demonstrate the effectiveness of the proposed method compared to conventional methods: COLMAP, NeRF, and DS-NeRF.
Databáze: Directory of Open Access Journals