Hand and Pose-Based Feature Selection for Zero-Shot Sign Language Recognition

Autor: Giray Sercan Ozcan, Yunus Can Bilge, Emre Sumer
Jazyk: angličtina
Rok vydání: 2024
Předmět:
Zdroj: IEEE Access, Vol 12, Pp 107757-107768 (2024)
Druh dokumentu: article
ISSN: 2169-3536
DOI: 10.1109/ACCESS.2024.3436671
Popis: Sign language functions as an indispensable interaction method for a certain portion of people in society, offering a unique way of communication. A significant challenge in advancing towards this objective is the difficulty in obtaining suitable training data for each sign in supervised learning. This challenge comes from the complex process of labeling signs and the limited number of skilled people available to do this job. This work introduces a new approach to the problem of Zero-Shot Sign Language Recognition (ZSSLR). We basically utilize and model hand and landmark data streams extracted from the body of the signer. Based on these extracted and modeled features, we employ a data grading approach to facilitate visual embedding with the self-attention mechanism. We utilize textual sign description features along with visual embedding in the Zero-Shot Learning (ZSL) settings. We assess the efficacy of our methodology in two of the suggested ZSL benchmarks.
Databáze: Directory of Open Access Journals