A Wearable Smart Glove and Its Application of Pose and Gesture Detection to Sign Language Classification
| Autor: | Joseph DelPreto, Josie Hughes, Matteo D'Aria, Marco de Fazio, Daniela Rus |
|---|---|
| Rok vydání: | 2022 |
| Předmět: |
Interactive computer systems
Control and Optimization Embedded systems Biomedical Engineering Capacitive sensors Degrees of freedom (mechanics) Artificial Intelligence Long short-term memory American sign language Embedded-system Facial Expressions Microcontrollers Robot sensing system Sensors and actuators Classification (of information) Controllers Learning systems Wearable robotics Soft sensors Mechanical Engineering Soft actuators Real time systems Computer Science Applications Human-Computer Interaction Gesture posture and facial expression Control and Systems Engineering Wearable sensors Computer Vision and Pattern Recognition Real - Time system Accelerometers |
| Zdroj: | IEEE Robotics and Automation Letters. 7:10589-10596 |
| ISSN: | 2377-3774 |
| DOI: | 10.1109/lra.2022.3191232 |
| Popis: | Advances in soft sensors coupled with machine learning are enabling increasingly capable wearable systems. Since hand motion in particular can convey useful information for developing intuitive interfaces, glove-based systems can have a significant impact on many application areas. A key remaining challenge for wearables is to capture, process, and analyze data from the high-degree-of-freedom hand in real time. We propose using a commercially available conductive knit to create an unobtrusive network of resistive sensors that spans all hand joints, coupling this with an accelerometer, and deploying machine learning on a low-profile microcontroller to process and classify data. This yields a self-contained wearable device with rich sensing capabilities for hand pose and orientation, low fabrication time, and embedded activity prediction. To demonstrate its capabilities, we use it to detect static poses and dynamic gestures from American Sign Language (ASL). By pre-training a long short-term memory (LSTM) neural network and using tools to deploy it in an embedded context, the glove and an ST microcontroller can classify 12 ASL letters and 12 ASL words in real time. Using a leave-one-experiment-out cross validation methodology, networks successfully classify 96.3% of segmented examples and generate correct rolling predictions during 92.8% of real-time streaming trials. Author |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|