A deep neural network ensemble of multimodal signals for classifying excavator operations
| Autor: | Jin Young Kim, Sung-Bae Cho |
|---|---|
| Rok vydání: | 2022 |
| Předmět: |
0209 industrial biotechnology
Artificial neural network Computer science Cognitive Neuroscience Feature vector 02 engineering and technology computer.software_genre Computer Science Applications Weighting Excavator 020901 industrial engineering & automation Artificial Intelligence Feature (computer vision) 0202 electrical engineering electronic engineering information engineering Prognostics 020201 artificial intelligence & image processing Data mining computer |
| Zdroj: | Neurocomputing. 470:290-299 |
| ISSN: | 0925-2312 |
| Popis: | The prognostics and health management (PHM) aims to provide a comprehensive solution for equipment health care. Classifying the operation mode of excavator, one of the tasks in the PHM, is important to evaluate the remaining useful lifetime. Several studies have been conducted to classify the operations with either video or sensor data, but they have several limitations to use only one type of data. A model trained with sensor data cannot classify the similar operations such as “digging” and “ditch digging”, whereas a model with video data is vulnerable to surrounding condition like weather. In this paper, to overcome these shortcomings, we propose a deep neural network ensemble called FusionNet that classifies the operations of excavator. Two models are trained with sensor data and video frames respectively, where the feature extractors are transferred to the FusionNet. The proposed network ensemble performs a flexible and well-optimized classification by automatically calculating weights according to the extracted feature vectors and combining them. To verify the proposed model, several experiments are conducted with the real-world data. The proposed model achieves the accuracy of 99.17% which outperforms the conventional methods. We also confirm that the proposed model can address the shortcomings of using only one type of data and maximize the benefits through the automatic weighting of extracted features. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect