mmPose-NLP: A Natural Language Processing Approach to Precise Skeletal Pose Estimation Using mmWave Radars
| Autor: | Siyang Cao, Arindam Sengupta |
|---|---|
| Rok vydání: | 2022 |
| Předmět: |
Signal Processing (eess.SP)
FOS: Computer and information sciences Artificial Intelligence Computer Networks and Communications Computer Vision and Pattern Recognition (cs.CV) FOS: Electrical engineering electronic engineering information engineering ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION Computer Science - Computer Vision and Pattern Recognition Electrical Engineering and Systems Science - Signal Processing Software Computer Science Applications |
| Zdroj: | IEEE Transactions on Neural Networks and Learning Systems. :1-12 |
| ISSN: | 2162-2388 2162-237X |
| DOI: | 10.1109/tnnls.2022.3151101 |
| Popis: | In this paper we presented mmPose-NLP, a novel Natural Language Processing (NLP) inspired Sequence-to-Sequence (Seq2Seq) skeletal key-point estimator using millimeter-wave (mmWave) radar data. To the best of the author's knowledge, this is the first method to precisely estimate upto 25 skeletal key-points using mmWave radar data alone. Skeletal pose estimation is critical in several applications ranging from autonomous vehicles, traffic monitoring, patient monitoring, gait analysis, to defense security forensics, and aid both preventative and actionable decision making. The use of mmWave radars for this task, over traditionally employed optical sensors, provide several advantages, primarily its operational robustness to scene lighting and adverse weather conditions, where optical sensor performance degrade significantly. The mmWave radar point-cloud (PCL) data is first voxelized (analogous to tokenization in NLP) and $N$ frames of the voxelized radar data (analogous to a text paragraph in NLP) is subjected to the proposed mmPose-NLP architecture, where the voxel indices of the 25 skeletal key-points (analogous to keyword extraction in NLP) are predicted. The voxel indices are converted back to real world 3-D coordinates using the voxel dictionary used during the tokenization process. Mean Absolute Error (MAE) metrics were used to measure the accuracy of the proposed system against the ground truth, with the proposed mmPose-NLP offering Submitted to IEEE Transactions |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|