Multimodal integration for data-driven classification of mental fatigue during construction equipment operations: Incorporating electroencephalography, electrodermal activity, and video signals

Autor: Imran Mehmood, Heng Li, Waleed Umer, Aamir Arsalan, Shahnawaz Anwer, Mohammed Aquil Mirza, Jie Ma, Maxwell Fordjour Antwi-Afari
Jazyk: angličtina
Rok vydání: 2023
Předmět:
Zdroj: Developments in the Built Environment, Vol 15, Iss , Pp 100198- (2023)
Druh dokumentu: article
ISSN: 2666-1659
DOI: 10.1016/j.dibe.2023.100198
Popis: Construction equipment operations that require high levels of attention can cause mental fatigue, which can lead to inefficiencies and accidents. Previous studies classified mental fatigue using single-modal data with acceptable accuracy. However, mental fatigue is a multimodal problem, and no single modality is superior. Moreover, none of the previous studies in construction industry have investigated multimodal data fusion for classifying mental fatigue and whether such an approach would improve mental fatigue detection. This study proposes a novel approach using three machine learning models and multimodal data fusion to classify mental fatigue states. Electroencephalography, electrodermal activity, and video signals were acquired during an excavation operation, and the decision tree model using multimodal sensor data fusion outperformed other models with 96.2% accuracy and 96.175%–98.231% F1 scores. Multimodal sensor data fusion can aid in the development of a real-time system to classify mental fatigue and improve safety management at construction sites.
Databáze: Directory of Open Access Journals