VirtualConductor: Music-driven Conducting Video Generation System

Autor:	Chen, Delong, Liu, Fan, Li, Zewen, Xu, Feng
Rok vydání:	2021
Předmět:	Computer Science - Computer Vision and Pattern Recognition Computer Science - Artificial Intelligence
Druh dokumentu:	Working Paper
Popis:	In this demo, we present VirtualConductor, a system that can generate conducting video from any given music and a single user's image. First, a large-scale conductor motion dataset is collected and constructed. Then, we propose Audio Motion Correspondence Network (AMCNet) and adversarial-perceptual learning to learn the cross-modal relationship and generate diverse, plausible, music-synchronized motion. Finally, we combine 3D animation rendering and a pose transfer model to synthesize conducting video from a single given user's image. Therefore, any user can become a virtual conductor through the system. Comment: Accepted by IEEE International Conference on Multimedia and Expo (ICME) 2021, demo track. Best demo
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2108.04350 Zobrazit plný text záznamu View this record from Arxiv