Expressive ASL Recognition using Millimeter-wave Wireless Signals
| Autor: | Riley Wilkerson, Parth H. Pathak, Raja Kushalnagar, Ding Zhang, Yuanqi Du, Al Amin Hosain, Panneer Selvam Santhalingam, Huzefa Rangwala |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
050101 languages & linguistics
American Sign Language Computer science business.industry Deep learning Speech recognition 05 social sciences Word error rate Context (language use) 02 engineering and technology Torso language.human_language medicine.anatomical_structure 0202 electrical engineering electronic engineering information engineering medicine language 020201 artificial intelligence & image processing 0501 psychology and cognitive sciences Artificial intelligence Cluster analysis business Sentence Gesture |
| Zdroj: | SECON |
| DOI: | 10.1109/secon48991.2020.9158441 |
| Popis: | Over half a million people in the United States use American Sign Language (ASL) as their primary mode of communication. Automatic ASL recognition would enable Deaf and Hard of Hearing (DHH) users to interact with others who are not familiar with ASL as well as voice-controlled digital assistants (e.g., Alexa, Siri, etc.). While ASL recognition has been extensively studied, there is a little attention given to recognition of ASL non-manual body markers. The non-manual markers are typically expressed through head, torso and shoulder movements, and add essential meaning and context to the signed sentences. In this work, we present ExASL, a sentence-level ASL recognition system using millimeter-wave radars. ExASL can recognize manual markers (hand gestures) and non-manual markers (head and torso movements). It utilizes multi-distance clustering to recognize body parts and cluster mmWave point clouds. We then present a multi-view deep learning algorithm that can learn from clustered body part representation for an expressive sentence-level recognition. Our evaluation shows that ExASL can recognize ASL sentences with a word error rate of 0.79%, sentence error rate of 1.25%, and non-manual markers with an accuracy of 83.5%. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|