SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans

Autor: Angela Dai, Tatiana Khanova, Armen Avetisyan, Christopher Choy, Matthias Nießner, Denver Dash
Jazyk: angličtina
Předmět:
Zdroj: Lecture Notes in Computer Science
Lecture Notes in Computer Science-Computer Vision – ECCV 2020
Computer Vision – ECCV 2020 ISBN: 9783030585419
ECCV (22)
ISSN: 0302-9743
1611-3349
DOI: 10.1007/978-3-030-58542-6_36
Popis: We present a novel approach to reconstructing lightweight, CAD-based representations of scanned 3D environments from commodity RGB-D sensors. Our key idea is to jointly optimize for both CAD model alignments as well as layout estimations of the scanned scene, explicitly modeling inter-relationships between objects-to-objects and objects-to-layout. Since object arrangement and scene layout are intrinsically coupled, we show that treating the problem jointly significantly helps to produce globally-consistent representations of a scene. Object CAD models are aligned to the scene by establishing dense correspondences between geometry, and we introduce a hierarchical layout prediction approach to estimate layout planes from corners and edges of the scene. To this end, we propose a message-passing graph neural network to model the inter-relationships between objects and layout, guiding generation of a globally object alignment in a scene. By considering the global scene layout, we achieve significantly improved CAD alignments compared to state-of-the-art methods, improving from 41.83% to 58.41% alignment accuracy on SUNCG and from 50.05% to 61.24% on ScanNet, respectively. The resulting CAD-based representations makes our method well-suited for applications in content creation such as augmented- or virtual reality.
Databáze: OpenAIRE