MagicAvatar: Multimodal Avatar Generation and Animation

Autor:	Zhang, Jianfeng, Yan, Hanshu, Xu, Zhongcong, Feng, Jiashi, Liew, Jun Hao
Rok vydání:	2023
Předmět:	Computer Science - Graphics Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	This report presents MagicAvatar, a framework for multimodal video generation and animation of human avatars. Unlike most existing methods that generate avatar-centric videos directly from multimodal inputs (e.g., text prompts), MagicAvatar explicitly disentangles avatar video generation into two stages: (1) multimodal-to-motion and (2) motion-to-video generation. The first stage translates the multimodal inputs into motion/ control signals (e.g., human pose, depth, DensePose); while the second stage generates avatar-centric video guided by these motion signals. Additionally, MagicAvatar supports avatar animation by simply providing a few images of the target person. This capability enables the animation of the provided human identity according to the specific motion derived from the first stage. We demonstrate the flexibility of MagicAvatar through various applications, including text-guided and video-guided avatar generation, as well as multimodal avatar animation. Comment: Project page: https://magic-avatar.github.io/
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2308.14748 Zobrazit plný text záznamu View this record from Arxiv