HybridBooth: Hybrid Prompt Inversion for Efficient Subject-Driven Generation

Autor:	Guan, Shanyan, Ge, Yanhao, Tai, Ying, Yang, Jian, Li, Wei, You, Mingyu
Rok vydání:	2024
Předmět:	Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	Recent advancements in text-to-image diffusion models have shown remarkable creative capabilities with textual prompts, but generating personalized instances based on specific subjects, known as subject-driven generation, remains challenging. To tackle this issue, we present a new hybrid framework called HybridBooth, which merges the benefits of optimization-based and direct-regression methods. HybridBooth operates in two stages: the Word Embedding Probe, which generates a robust initial word embedding using a fine-tuned encoder, and the Word Embedding Refinement, which further adapts the encoder to specific subject images by optimizing key parameters. This approach allows for effective and fast inversion of visual concepts into textual embedding, even from a single image, while maintaining the model's generalization capabilities. Comment: ECCV 2024, the project page: https://sites.google.com/view/hybridbooth
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2410.08192 Zobrazit plný text záznamu View this record from Arxiv