Reconstructing Objects in-the-wild for Realistic Sensor Simulation

Autor:	Yang, Ze, Manivasagam, Sivabalan, Chen, Yun, Wang, Jingkang, Hu, Rui, Urtasun, Raquel
Rok vydání:	2023
Předmět:	Computer Science - Computer Vision and Pattern Recognition Computer Science - Robotics
Druh dokumentu:	Working Paper
Popis:	Reconstructing objects from real world data and rendering them at novel views is critical to bringing realism, diversity and scale to simulation for robotics training and testing. In this work, we present NeuSim, a novel approach that estimates accurate geometry and realistic appearance from sparse in-the-wild data captured at distance and at limited viewpoints. Towards this goal, we represent the object surface as a neural signed distance function and leverage both LiDAR and camera sensor data to reconstruct smooth and accurate geometry and normals. We model the object appearance with a robust physics-inspired reflectance representation effective for in-the-wild data. Our experiments show that NeuSim has strong view synthesis performance on challenging scenarios with sparse training views. Furthermore, we showcase composing NeuSim assets into a virtual world and generating realistic multi-sensor data for evaluating self-driving perception models. Comment: ICRA 2023. Project page: https://waabi.ai/neusim/
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2311.05602 Zobrazit plný text záznamu View this record from Arxiv