End-to-End Optimization of Scene Layout
| Autor: | Zhoutong Zhang, Joshua B. Tenenbaum, Andrew Luo, Jiajun Wu |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Computer science business.industry Color image Computer Vision and Pattern Recognition (cs.CV) ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION Computer Science - Computer Vision and Pattern Recognition 020207 software engineering 02 engineering and technology Solid modeling 010501 environmental sciences 01 natural sciences Rendering (computer graphics) Generative model End-to-end principle 0202 electrical engineering electronic engineering information engineering Computer vision Scene graph Differentiable function Artificial intelligence business 0105 earth and related environmental sciences ComputingMethodologies_COMPUTERGRAPHICS |
| Zdroj: | CVPR |
| Popis: | We propose an end-to-end variational generative model for scene layout synthesis conditioned on scene graphs. Unlike unconditional scene layout generation, we use scene graphs as an abstract but general representation to guide the synthesis of diverse scene layouts that satisfy relationships included in the scene graph. This gives rise to more flexible control over the synthesis process, allowing various forms of inputs such as scene layouts extracted from sentences or inferred from a single color image. Using our conditional layout synthesizer, we can generate various layouts that share the same structure of the input example. In addition to this conditional generation design, we also integrate a differentiable rendering module that enables layout refinement using only 2D projections of the scene. Given a depth and a semantics map, the differentiable rendering module enables optimizing over the synthesized layout to fit the given input in an analysis-by-synthesis fashion. Experiments suggest that our model achieves higher accuracy and diversity in conditional scene synthesis and allows exemplar-based scene generation from various input forms. CVPR 2020 (Oral). Project page: http://3dsln.csail.mit.edu/ |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|