Neural Ray Surfaces for Self-Supervised Learning of Depth and Ego-motion
| Autor: | Wolfram Burgard, Vitor Guizilini, Greg Shakhnarovich, Sudeep Pillai, Rares Ambrus, Igor Vasiljevic, Adrien Gaidon |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning 0209 industrial biotechnology Computer science Computer Vision and Pattern Recognition (cs.CV) Computer Science - Computer Vision and Pattern Recognition ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION 02 engineering and technology Solid modeling Machine Learning (cs.LG) Computer Science - Robotics Catadioptric system 020901 industrial engineering & automation Motion estimation 0202 electrical engineering electronic engineering information engineering Computer vision Visual odometry Projection (set theory) business.industry 020201 artificial intelligence & image processing Pinhole (optics) Ray tracing (graphics) Artificial intelligence Geometric modeling business Robotics (cs.RO) |
| Zdroj: | 3DV |
| DOI: | 10.48550/arxiv.2008.06630 |
| Popis: | Self-supervised learning has emerged as a powerful tool for depth and ego-motion estimation, leading to state-of-the-art results on benchmark datasets. However, one significant limitation shared by current methods is the assumption of a known parametric camera model -- usually the standard pinhole geometry -- leading to failure when applied to imaging systems that deviate significantly from this assumption (e.g., catadioptric cameras or underwater imaging). In this work, we show that self-supervision can be used to learn accurate depth and ego-motion estimation without prior knowledge of the camera model. Inspired by the geometric model of Grossberg and Nayar, we introduce Neural Ray Surfaces (NRS), convolutional networks that represent pixel-wise projection rays, approximating a wide range of cameras. NRS are fully differentiable and can be learned end-to-end from unlabeled raw videos. We demonstrate the use of NRS for self-supervised learning of visual odometry and depth estimation from raw videos obtained using a wide variety of camera systems, including pinhole, fisheye, and catadioptric. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|