Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFs

Autor:	Warburg, Frederik, Weber, Ethan, Tancik, Matthew, Holynski, Aleksander, Kanazawa, Angjoo
Rok vydání:	2023
Předmět:	Computer Science - Computer Vision and Pattern Recognition Computer Science - Artificial Intelligence Computer Science - Graphics
Druh dokumentu:	Working Paper
Popis:	Casually captured Neural Radiance Fields (NeRFs) suffer from artifacts such as floaters or flawed geometry when rendered outside the camera trajectory. Existing evaluation protocols often do not capture these effects, since they usually only assess image quality at every 8th frame of the training capture. To push forward progress in novel-view synthesis, we propose a new dataset and evaluation procedure, where two camera trajectories are recorded of the scene: one used for training, and the other for evaluation. In this more challenging in-the-wild setting, we find that existing hand-crafted regularizers do not remove floaters nor improve scene geometry. Thus, we propose a 3D diffusion-based method that leverages local 3D priors and a novel density-based score distillation sampling loss to discourage artifacts during NeRF optimization. We show that this data-driven prior removes floaters and improves scene geometry for casual captures. Comment: ICCV 2023, project page: https://ethanweber.me/nerfbusters
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2304.10532 Zobrazit plný text záznamu View this record from Arxiv