Real-Time Neural Rasterization for Large Scenes

Autor:	Liu, Jeffrey Yunfan, Chen, Yun, Yang, Ze, Wang, Jingkang, Manivasagam, Sivabalan, Urtasun, Raquel
Rok vydání:	2023
Předmět:	Computer Science - Computer Vision and Pattern Recognition Computer Science - Artificial Intelligence Computer Science - Graphics
Druh dokumentu:	Working Paper
Popis:	We propose a new method for realistic real-time novel-view synthesis (NVS) of large scenes. Existing neural rendering methods generate realistic results, but primarily work for small scale scenes (<50 square meters) and have difficulty at large scale (>10000 square meters). Traditional graphics-based rasterization rendering is fast for large scenes but lacks realism and requires expensive manually created assets. Our approach combines the best of both worlds by taking a moderate-quality scaffold mesh as input and learning a neural texture field and shader to model view-dependant effects to enhance realism, while still using the standard graphics pipeline for real-time rendering. Our method outperforms existing neural rendering methods, providing at least 30x faster rendering with comparable or better realism for large self-driving and drone scenes. Our work is the first to enable real-time rendering of large real-world scenes. Comment: Published in ICCV 2023. webpage: https://waabi.ai/NeuRas/
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2311.05607 Zobrazit plný text záznamu View this record from Arxiv