An intelligent shopping support robot: understanding shopping behavior from 2D skeleton data using GRU network
| Autor: | Md Matiqul Islam, Antony Lam, Hisato Fukuda, Yoshinori Kobayashi, Yoshinori Kuno |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: |
GRU
Shopping behavior OpenPose Head orientation Body orientation DNN Technology Mechanical engineering and machinery TJ1-1570 Control engineering systems. Automatic machinery (General) TJ212-225 Machine design and drawing TJ227-240 Technology (General) T1-995 Industrial engineering. Management engineering T55.4-60.8 Automation T59.5 Information technology T58.5-58.64 |
| Zdroj: | ROBOMECH Journal, Vol 6, Iss 1, Pp 1-10 (2019) |
| Druh dokumentu: | article |
| ISSN: | 2197-4225 |
| DOI: | 10.1186/s40648-019-0150-1 |
| Popis: | Abstract In supermarkets or grocery, a shopping cart is a necessary tool for shopping. In this paper, we have developed an intelligent shopping support robot that can carry a shopping cart while following its owners and provide the shopping support by observing the customer’s head orientation, body orientation and recognizing different shopping behaviors. Recognizing shopping behavior or the intensity of such action is important for understanding the best way to support the customer without disturbing him or her. This system also liberates elderly and disabled people from the burden of pushing shopping carts, because our proposed shopping cart is essentially a type of autonomous mobile robots that recognizes its owner and following him or her. The proposed system discretizes the head and body orientation of customer into 8 directions to estimate whether the customer is looking or turning towards a merchandise shelf. From the robot’s video stream, a DNN-based human pose estimator called OpenPose is used to extract the skeleton of 18 joints for each detected body. Using this extracted body joints information, we built a dataset and developed a novel Gated Recurrent Neural Network (GRU) topology to classify different actions that are typically performed in front of shelves: reach to shelf, retract from shelf, hand in shelf, inspect product, inspect shelf. Our GRU network model takes series of 32 frames skeleton data then gives the prediction. Using cross-validation tests, our model achieves an overall accuracy of 82%, which is a significant result. Finally, from the customer’s head orientation, body orientation and shopping behavior recognition we develop a complete system for our shopping support robot. |
| Databáze: | Directory of Open Access Journals |
| Externí odkaz: |
|
Full text from SpringerLink