Independent Sign Language Recognition with 3d Body, Hands, and Face Reconstruction
| Autor: | Agelos Kratimenos, Petros Maragos, Georgios Pavlakos |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Facial expression Computer science business.industry Computer Vision and Pattern Recognition (cs.CV) 3D reconstruction Computer Science - Computer Vision and Pattern Recognition ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION Optical flow Sign language Facial recognition system Recurrent neural network Gesture recognition Computer vision Artificial intelligence business Gesture |
| Zdroj: | ICASSP |
| DOI: | 10.1109/icassp39728.2021.9414278 |
| Popis: | Independent Sign Language Recognition is a complex visual recognition problem that combines several challenging tasks of Computer Vision due to the necessity to exploit and fuse information from hand gestures, body features and facial expressions. While many state-of-the-art works have managed to deeply elaborate on these features independently, to the best of our knowledge, no work has adequately combined all three information channels to efficiently recognize Sign Language. In this work, we employ SMPL-X, a contemporary parametric model that enables joint extraction of 3D body shape, face and hands information from a single image. We use this holistic 3D reconstruction for SLR, demonstrating that it leads to higher accuracy than recognition from raw RGB images and their optical flow fed into the state-of-the-art I3D-type network for 3D action recognition and from 2D Openpose skeletons fed into a Recurrent Neural Network. Finally, a set of experiments on the body, face and hand features showed that neglecting any of these, significantly reduces the classification accuracy, proving the importance of jointly modeling body shape, facial expression and hand pose for Sign Language Recognition. Comment: Submitted to ICASSP 2021 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|