Facial optical flow estimation via neural non-rigid registration

Autor: Zhuang Peng, Boyi Jiang, Haofei Xu, Wanquan Feng, Juyong Zhang
Jazyk: angličtina
Rok vydání: 2022
Předmět:
Zdroj: Computational Visual Media, Vol 9, Iss 1, Pp 109-122 (2022)
Druh dokumentu: article
ISSN: 2096-0433
2096-0662
DOI: 10.1007/s41095-021-0267-z
Popis: Abstract Optical flow estimation in human facial video, which provides 2D correspondences between adjacent frames, is a fundamental pre-processing step for many applications, like facial expression capture and recognition. However, it is quite challenging as human facial images contain large areas of similar textures, rich expressions, and large rotations. These characteristics also result in the scarcity of large, annotated real-world datasets. We propose a robust and accurate method to learn facial optical flow in a self-supervised manner. Specifically, we utilize various shape priors, including face depth, landmarks, and parsing, to guide the self-supervised learning task via a differentiable nonrigid registration framework. Extensive experiments demonstrate that our method achieves remarkable improvements for facial optical flow estimation in the presence of significant expressions and large rotations.
Databáze: Directory of Open Access Journals
Externí odkaz: