SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans
Autor: | Angela Dai, Tatiana Khanova, Armen Avetisyan, Christopher Choy, Matthias Nießner, Denver Dash |
---|---|
Jazyk: | angličtina |
Předmět: |
Graph neural networks
business.industry Computer science 020207 software engineering CAD 02 engineering and technology Content creation Virtual reality Object (computer science) 0202 electrical engineering electronic engineering information engineering Key (cryptography) RGB color model 020201 artificial intelligence & image processing Computer vision Artificial intelligence business |
Zdroj: | Lecture Notes in Computer Science Lecture Notes in Computer Science-Computer Vision – ECCV 2020 Computer Vision – ECCV 2020 ISBN: 9783030585419 ECCV (22) |
ISSN: | 0302-9743 1611-3349 |
DOI: | 10.1007/978-3-030-58542-6_36 |
Popis: | We present a novel approach to reconstructing lightweight, CAD-based representations of scanned 3D environments from commodity RGB-D sensors. Our key idea is to jointly optimize for both CAD model alignments as well as layout estimations of the scanned scene, explicitly modeling inter-relationships between objects-to-objects and objects-to-layout. Since object arrangement and scene layout are intrinsically coupled, we show that treating the problem jointly significantly helps to produce globally-consistent representations of a scene. Object CAD models are aligned to the scene by establishing dense correspondences between geometry, and we introduce a hierarchical layout prediction approach to estimate layout planes from corners and edges of the scene. To this end, we propose a message-passing graph neural network to model the inter-relationships between objects and layout, guiding generation of a globally object alignment in a scene. By considering the global scene layout, we achieve significantly improved CAD alignments compared to state-of-the-art methods, improving from 41.83% to 58.41% alignment accuracy on SUNCG and from 50.05% to 61.24% on ScanNet, respectively. The resulting CAD-based representations makes our method well-suited for applications in content creation such as augmented- or virtual reality. |
Databáze: | OpenAIRE |
Externí odkaz: |