Snap2cad: 3D indoor environment reconstruction for AR/VR applications using a smartphone device
| Autor: | Alessandro Manni, Damiano Oriti, Federico Manuri, Andrea Sanna, Francesco De Pace |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
3D Object retrieval
Computer science business.industry ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION General Engineering 3D Object pose estimation Augmented reality Virtual reality Scale factor Computer Graphics and Computer-Aided Design Machine learning Scene reconstruction Human-Computer Interaction Computer graphics RGB color model Computer vision Noise (video) Artificial intelligence Android (operating system) business Rotation (mathematics) |
| Popis: | Indoor environment reconstruction is a challenging task in Computer Vision and Computer Graphics, especially when Extended Reality (XR) technologies are considered. Current solutions that employ dedicated depth sensors require scanning of the environment and tend to suffer from low resolution and noise, whereas solutions that rely on a single photo of a scene cannot predict the actual position and scale of objects due to scale ambiguity. The proposed system addresses these limitations by allowing the user to capture single views of objects using an Android smartphone equipped with a single RGB camera and supported by Google ARCore. The system includes 1) an Android app tracking the smartphone’s position relative to the world, capturing a single RGB image for each object and estimating depth information of the scene, 2) a program running on a server that classifies the framed objects, retrieves the corresponding 3D models from a database and estimates their position, vertical rotation, and scale factor without deforming the shape. The system has been assessed measuring the translational, rotational and scaling errors of the considered objects with respect to the physical ones acting as a ground truth. The main outcomes show that the proposed solution obtains a maximum error of 18% for the scaling factor, less than nine centimeters for the position and less than 18 ∘ for the rotation. These results suggest that the proposed system can be employed for XR applications, thus bridging the gap between the real and virtual worlds. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|