GITPose: going shallow and deeper using vision transformers for human pose estimation

Autor: Evans Aidoo, Xun Wang, Zhenguang Liu, Abraham Opanfo Abbam, Edwin Kwadwo Tenagyei, Victor Nonso Ejianya, Seth Larweh Kodjiku, Esther Stacy E. B. Aggrey
Jazyk: angličtina
Rok vydání: 2024
Předmět:
Zdroj: Complex & Intelligent Systems, Vol 10, Iss 3, Pp 4507-4520 (2024)
Druh dokumentu: article
ISSN: 2199-4536
2198-6053
69000859
DOI: 10.1007/s40747-024-01361-y
Popis: Abstract In comparison to convolutional neural networks (CNN), the newly created vision transformer (ViT) has demonstrated impressive outcomes in human pose estimation (HPE). However, (1) there is a quadratic rise in complexity with respect to image size, which causes the traditional ViT to be unsuitable for scaling, and (2) the attention process at the transformer encoder as well as decoder also adds substantial computational costs to the detector’s overall processing time. Motivated by this, we propose a novel Going shallow and deeper with vIsion Transformers for human Pose estimation (GITPose) without CNN backbones for feature extraction. In particular, we introduce a hierarchical transformer in which we utilize multilayer perceptrons to encode the richest local feature tokens in the initial phases (i.e., shallow), whereas self-attention modules are employed to encode long-term relationships in the deeper layers (i.e., deeper), and a decoder for keypoint detection. In addition, we offer a learnable deformable token association module (DTA) to non-uniformly and dynamically combine informative keypoint tokens. Comprehensive evaluation and testing on the COCO and MPII benchmark datasets reveal that GITPose achieves a competitive average precision (AP) on pose estimation compared to its state-of-the-art approaches.
Databáze: Directory of Open Access Journals
Externí odkaz: