A Unified Learning-Based Framework for Light Field Reconstruction From Coded Projections
| Autor: | Kaushik Mitra, Sharath Girish, Anil Kumar Vadathya |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
business.industry
Computer science Aperture ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION 020207 software engineering 02 engineering and technology Iterative reconstruction Multiplexing Computer Science Applications View synthesis Computational Mathematics Signal Processing 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Computer vision Artificial intelligence Coded aperture Image warping business Image resolution Light field |
| Zdroj: | IEEE Transactions on Computational Imaging. 6:304-316 |
| ISSN: | 2334-0118 2573-0436 |
| DOI: | 10.1109/tci.2019.2948780 |
| Popis: | Light fields present a rich way to represent the 3D world by capturing the spatio-angular dimensions of the visual signal. However, the popular way of capturing light fields (LF) via a plenoptic camera presents a spatio-angular resolution trade-off. To address this issue, computational imaging techniques such as compressive light field and programmable coded aperture have been proposed, which reconstruct full sensor resolution LF from coded projections of the LF. Here, we present a unified learning framework that can reconstruct LF from a variety of multiplexing schemes with minimal number of coded images as input. We consider three light field capture schemes: heterodyne capture scheme with code placed near the sensor, coded aperture scheme with code at the camera aperture and finally the dual exposure scheme of capturing a focus-defocus pair where there is no explicit coding. Our algorithm consists of three stages: Firstly, we recover the all-in-focus image from the coded image. Secondly, we estimate the disparity maps for all the LF views from the coded image and the all-in-focus image. And finally, we render the LF by warping the all-in-focus image using the estimated disparity maps. We show that our proposed learning algorithm performs either on par with or better than the state-of-the-art methods for all the three multiplexing schemes. LF from focus-defocus pair is especially attractive as it requires no hardware modification and produces LF reconstructions that are comparable to the current state of the art learning-based view synthesis approaches from multiple images. Thus, our work paves the way for capturing full-resolution LF using conventional cameras such as DSLRs and smartphones. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|