JointMETRO: a 3D reconstruction model for human figures in works of art based on transformer.

Autor:	Pang, Shanchen, Peng, Rongrong, Dong, Yukun, Yuan, Qi, Wang, Shengtao, Sun, Junqi
Předmět:	HUMAN figure in art TRANSFORMER models JOINTS (Anatomy) HUMAN body COMPUTATIONAL complexity
Zdroj:	Neural Computing & Applications; Jul2024, Vol. 36 Issue 20, p11711-11725, 15p
Abstrakt:	Sculptures and paintings are an important part of our cultural heritage, and 3D reconstruction of the human in them can help us better preserve and inherit this heritage. By creating 3D models of sculptures and paintings, these artworks become more accessible to a wider audience. This allows more people to appreciate and learn about different cultures and their heritage. 3D human reconstruction based on Transformer structures has recently achieved state-of-the-art results, but this approach requires a large number of parameters and expensive computations. The neglected computational complexity and the size of the model make it difficult to use such models for practical applications. In this paper, we propose a lightweight method based on human body joints and Transformer, called JointMETRO (Joint MEsh TRansfOrmer). It reconstructs the human body mesh from 2D human joints and has better capability to reconstruct characters in artistic works. We propose a joint extraction module, which obtains the locations of the 13 human joint points and the confidence coefficient of each joint, and a mesh regression module, which is used to combine the extracted pose features with a mesh template to obtain the final reconstructed human mesh structure. Finally, the accuracy of the reconstructed human model can be seen visually by applying different colors to the mesh vertices based on the confidence coefficient of each joint. We have demonstrated the efficiency of the model through extensive evaluation on both the Human3.6M and 3DPW datasets. On the complex in-the-wild 3DPW dataset, JointMETRO achieved better accuracy than the pose-based SOTA method. [ABSTRACT FROM AUTHOR]
Databáze:	Complementary Index
Externí odkaz:	Zobrazit plný text záznamu Plný text ve formátu HTML
Nepřihlášeným uživatelům se plný text nezobrazuje	K zobrazení výsledku je třeba se přihlásit.