Instruct 3D-to-3D: Text Instruction Guided 3D-to-3D conversion

Autor:	Kamata, Hiromichi, Sakuma, Yuiko, Hayakawa, Akio, Ishii, Masato, Narihira, Takuya
Rok vydání:	2023
Předmět:	Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	We propose a high-quality 3D-to-3D conversion method, Instruct 3D-to-3D. Our method is designed for a novel task, which is to convert a given 3D scene to another scene according to text instructions. Instruct 3D-to-3D applies pretrained Image-to-Image diffusion models for 3D-to-3D conversion. This enables the likelihood maximization of each viewpoint image and high-quality 3D generation. In addition, our proposed method explicitly inputs the source 3D scene as a condition, which enhances 3D consistency and controllability of how much of the source 3D scene structure is reflected. We also propose dynamic scaling, which allows the intensity of the geometry transformation to be adjusted. We performed quantitative and qualitative evaluations and showed that our proposed method achieves higher quality 3D-to-3D conversions than baseline methods. Comment: Project page: https://sony.github.io/Instruct3Dto3D-doc/
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2303.15780 Zobrazit plný text záznamu View this record from Arxiv