Multimodal Deep Feature Fusion (MMDFF) for RGB-D Tracking
| Autor: | Chao Deng, Mingxin Jiang, Haiyan Zhang, Ming-min Zhang, Jing-song Shan |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
0209 industrial biotechnology
Multidisciplinary Article Subject General Computer Science business.industry Computer science Frame (networking) Optical flow 02 engineering and technology Tracking (particle physics) Convolutional neural network lcsh:QA75.5-76.95 Image (mathematics) 020901 industrial engineering & automation 0202 electrical engineering electronic engineering information engineering RGB color model Eye tracking 020201 artificial intelligence & image processing Computer vision lcsh:Electronic computers. Computer science Artificial intelligence business |
| Zdroj: | Complexity, Vol 2018 (2018) |
| ISSN: | 1099-0526 1076-2787 |
| DOI: | 10.1155/2018/5676095 |
| Popis: | Visual tracking is still a challenging task due to occlusion, appearance changes, complex motion, etc. We propose a novel RGB-D tracker based on multimodal deep feature fusion (MMDFF) in this paper. MMDFF model consists of four deep Convolutional Neural Networks (CNNs): Motion-specific CNN, RGB- specific CNN, Depth-specific CNN, and RGB-Depth correlated CNN. The depth image is encoded into three channels which are sent into depth-specific CNN to extract deep depth features. The optical flow image is calculated for every frame and then is fed to motion-specific CNN to learn deep motion features. Deep RGB, depth, and motion information can be effectively fused at multiple layers via MMDFF model. Finally, multimodal fusion deep features are sent into the C-COT tracker to obtain the tracking result. For evaluation, experiments are conducted on two recent large-scale RGB-D datasets and results demonstrate that our proposed RGB-D tracking method achieves better performance than other state-of-art RGB-D trackers. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Plný text