SceneCode: Monocular Dense Semantic Reconstruction using Learned Encoded Scene Representations
| Autor: | Andrew J. Davison, Shuaifeng Zhi, Michael Bloesch, Stefan Leutenegger |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
0209 industrial biotechnology Computer Science - Machine Learning Monocular business.industry Computer science Computer Vision and Pattern Recognition (cs.CV) Probabilistic logic Computer Science - Computer Vision and Pattern Recognition Motion (geometry) Pattern recognition 02 engineering and technology 010501 environmental sciences Semantics 01 natural sciences Image (mathematics) Machine Learning (cs.LG) Set (abstract data type) 020901 industrial engineering & automation Semantic mapping Artificial intelligence business 0105 earth and related environmental sciences |
| Zdroj: | CVPR |
| DOI: | 10.48550/arxiv.1903.06482 |
| Popis: | Systems which incrementally create 3D semantic maps from image sequences must store and update representations of both geometry and semantic entities. However, while there has been much work on the correct formulation for geometrical estimation, state-of-the-art systems usually rely on simple semantic representations which store and update independent label estimates for each surface element (depth pixels, surfels, or voxels). Spatial correlation is discarded, and fused label maps are incoherent and noisy. We introduce a new compact and optimisable semantic representation by training a variational auto-encoder that is conditioned on a colour image. Using this learned latent space, we can tackle semantic label fusion by jointly optimising the low-dimenional codes associated with each of a set of overlapping images, producing consistent fused label maps which preserve spatial correlation. We also show how this approach can be used within a monocular keyframe based semantic mapping system where a similar code approach is used for geometry. The probabilistic formulation allows a flexible formulation where we can jointly estimate motion, geometry and semantics in a unified optimisation. Comment: To be published in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2019) |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|