Employing multimodal co-learning to evaluate the robustness of sensor fusion for industry 5.0 tasks.

Autor: Rahate, Anil, Mandaokar, Shruti, Chandel, Pulkit, Walambe, Rahee, Ramanna, Sheela, Kotecha, Ketan
Předmět:
Zdroj: Soft Computing - A Fusion of Foundations, Methodologies & Applications; Apr2023, Vol. 27 Issue 7, p4139-4155, 17p
Abstrakt: Industry 5.0 focuses on collaboration between humans and machines, demanding robustness and efficiency and the accuracy of intelligent and innovative components used. The use of sensors and the fusion of data obtained from various sensors/modes increased with the rise of the internet of things. The multimodal sensor fusion provides better accuracies than a single source/mode system. Typically, in multimodal fusion, data from all sources is assumed to be available all the time, aligned, and noiseless. Contrary to this highly unrealistic assumption, data from one or more sources is generally unavailable or noisy in most real-world applications. Hence, there is a need for robust sensor fusion, which is a more realistic scenario and critical for Industry 5.0 implementations. Multimodal co-learning is one such approach to study the robustness of sensor fusion for missing and noisy modalities. In this work, to demonstrate the effectiveness of multimodal co-learning for robustness study, gas detection systems are considered a case study. Such gas detection systems are prevalent and crucial to avoid accidents due to gas leaks in many industries and form a part of the industry 5.0 setup. This work considers the primary dataset of gas sensors data and thermal images for robustness experimentation. The results demonstrate that multi-task fusion is robust to missing and noisy modalities than intermediate fusion. Having an additional low-resolution thermal modality supports co-learning and makes it robust to 20% missing sensor data, 90% missing thermal image data, and Gaussian and Normal noise. The end-to-end system architecture proposed can be easily extended to other multimodal applications in Industry 5.0. This study is a step towards creating standard practices for a multimodal co-learning charter for various industrial applications. [ABSTRACT FROM AUTHOR]
Databáze: Complementary Index
Externí odkaz: