NeRFuser: Large-Scale Scene Representation by NeRF Fusion

Autor:	Fang, Jiading, Lin, Shengjie, Vasiljevic, Igor, Guizilini, Vitor, Ambrus, Rares, Gaidon, Adrien, Shakhnarovich, Gregory, Walter, Matthew R.
Rok vydání:	2023
Předmět:	Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	A practical benefit of implicit visual representations like Neural Radiance Fields (NeRFs) is their memory efficiency: large scenes can be efficiently stored and shared as small neural nets instead of collections of images. However, operating on these implicit visual data structures requires extending classical image-based vision techniques (e.g., registration, blending) from image sets to neural fields. Towards this goal, we propose NeRFuser, a novel architecture for NeRF registration and blending that assumes only access to pre-generated NeRFs, and not the potentially large sets of images used to generate them. We propose registration from re-rendering, a technique to infer the transformation between NeRFs based on images synthesized from individual NeRFs. For blending, we propose sample-based inverse distance weighting to blend visual information at the ray-sample level. We evaluate NeRFuser on public benchmarks and a self-collected object-centric indoor dataset, showing the robustness of our method, including to views that are challenging to render from the individual source NeRFs. Comment: Code available at https://github.com/ripl/nerfuser
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2305.13307 Zobrazit plný text záznamu View this record from Arxiv