Peeking Behind Objects: Layered Depth Prediction from a Single Image
| Autor: | Keisuke Tateno, Iro Laina, Helisa Dhamo, Nassir Navab, Federico Tombari |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2018 |
| Předmět: |
FOS: Computer and information sciences
Computer science Computer Vision and Pattern Recognition (cs.CV) Layered depth image RGB-D inpainting Generative adversarial networks Occlusion ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION Computer Science - Computer Vision and Pattern Recognition 02 engineering and technology Virtual reality 01 natural sciences Convolutional neural network Artificial Intelligence Depth map 0103 physical sciences 0202 electrical engineering electronic engineering information engineering Computer vision 010306 general physics ComputingMethodologies_COMPUTERGRAPHICS business.industry View synthesis ddc Hallucinating Signal Processing 020201 artificial intelligence & image processing Computer Vision and Pattern Recognition Artificial intelligence Focus (optics) business Software Reference frame |
| Popis: | While conventional depth estimation can infer the geometry of a scene from a single RGB image, it fails to estimate scene regions that are occluded by foreground objects. This limits the use of depth prediction in augmented and virtual reality applications, that aim at scene exploration by synthesizing the scene from a different vantage point, or at diminished reality. To address this issue, we shift the focus from conventional depth map prediction to the regression of a specific data representation called Layered Depth Image (LDI), which contains information about the occluded regions in the reference frame and can fill in occlusion gaps in case of small view changes. We propose a novel approach based on Convolutional Neural Networks (CNNs) to jointly predict depth maps and foreground separation masks used to condition Generative Adversarial Networks (GANs) for hallucinating plausible color and depths in the initially occluded areas. We demonstrate the effectiveness of our approach for novel scene view synthesis from a single image. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|