{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T04:21:28Z","timestamp":1778646088816,"version":"3.51.4"},"reference-count":29,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T00:00:00Z","timestamp":1750118400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Despite significant technological advancements in aviation safety systems, human-operator condition monitoring remains a critical challenge, with more than 75% of aircraft incidents stemming from attention-related perceptual failures. This study addresses a fundamental question in sensor-based condition monitoring: how can temporal- and frequency-domain EEG sensor data be optimally integrated to detect precursors of system failure in human\u2013machine interfaces? We propose a three-stage diagnostic framework that mirrors industrial condition monitoring approaches. First, raw EEG sensor signals undergo preprocessing into standardized one-second epochs. Second, a novel hybrid feature-extraction methodology combines time- and frequency-domain features to create comprehensive sensor signatures of neural states. Finally, our dual-architecture CNN\u2013LSTM model processes spatial patterns via CNNs while capturing temporal degradation signals via LSTMs, enabling robust classification in noisy operational environments. Our contributions include (1) a multimodal data fusion approach for EEG sensors that provides a more comprehensive representation of operator conditions, and (2) an artificial intelligence architecture that balances spatial and temporal analysis for the predictive maintenance of attention states. When validated on aviation-related EEG datasets, our condition monitoring system achieved significantly higher diagnostic accuracy across various noise conditions compared to existing approaches. The practical applications extend beyond theoretical improvement, offering a pathway to implement more reliable human\u2013machine interface monitoring in critical systems, potentially preventing catastrophic failures by detecting condition anomalies before they propagate through the system.<\/jats:p>","DOI":"10.3390\/info16060503","type":"journal-article","created":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T07:04:52Z","timestamp":1750143892000},"page":"503","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Enhanced Pilot Attention Monitoring: A Time-Frequency EEG Analysis Using CNN\u2013LSTM Networks for Aviation Safety"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0453-7051","authenticated-orcid":false,"given":"Quynh Anh","family":"Nguyen","sequence":"first","affiliation":[{"name":"Faculty of Information Technology, Electric Power University, Ha Noi 100000, Vietnam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0536-8686","authenticated-orcid":false,"given":"Nam Anh","family":"Dao","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Electric Power University, Ha Noi 100000, Vietnam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7673-7955","authenticated-orcid":false,"given":"Long","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, Speed School of Engineering, University of Louisville, Louisville, KY 40241, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,17]]},"reference":[{"key":"ref_1","first-page":"507","article-title":"Sources of situation awareness errors in aviation","volume":"67","author":"Jones","year":"1996","journal-title":"Aviat. Space Environ. Med."},{"key":"ref_2","unstructured":"International Air Transport Association (2022). 2021 Safety Report Edition, International Air Transport Association."},{"key":"ref_3","unstructured":"International Air Transport Association (2019). Loss of Control In-Flight Accident Analysis Report 2019 Edition, International Air Transport Association."},{"key":"ref_4","unstructured":"Commercial Aviation Safety Team (2025, March 03). SE211: Airplane State Awareness\u2014Training for Attention Management. Available online: http:\/\/www.skybrary.aero\/index.php\/SE211:_Airplane_State_Awareness_-_Training_for_Attention_Management_(R-D)."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1016\/j.jairtraman.2009.01.001","article-title":"An Investigation of Fatigue Issues on Different Flight Operations","volume":"15","author":"Yen","year":"2009","journal-title":"J. Air Transp. Manag."},{"key":"ref_6","first-page":"360","article-title":"A Comparison of Heart Rate, Eye Activity, EEG and Subjective Measures of Pilot Mental Workload during Flight","volume":"69","author":"Hankins","year":"1998","journal-title":"Aviat. Space Environ. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.brainresrev.2008.07.001","article-title":"Mental Fatigue: Costs and Benefits","volume":"59","author":"Boksem","year":"2008","journal-title":"Brain Res. Rev."},{"key":"ref_8","first-page":"16","article-title":"The PREP Pipeline: Standardized Preprocessing for Large-Scale EEG Analysis","volume":"9","author":"Mullen","year":"2015","journal-title":"Front. Neuroinform."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"101077","DOI":"10.1016\/j.dcn.2022.101077","article-title":"Automated Pipeline for Infants Continuous EEG (APICE): A Flexible Pipeline for Developmental Cognitive Studies","volume":"54","author":"Gennari","year":"2022","journal-title":"Dev. Cogn. Neurosci."},{"key":"ref_10","unstructured":"Kaggle (2025, March 03). Reducing Commercial Aviation Fatalities. Available online: https:\/\/www.kaggle.com\/competitions\/reducing-commercial-aviation-fatalities\/overview."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Roza, V., and Postolache, O. (2019). Multimodal Approach for Emotion Recognition Based on Simulated Flight Experiments. Sensors, 19.","DOI":"10.3390\/s19245516"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1016\/j.bbe.2019.12.002","article-title":"Classification of Pilots\u2019 Mental States Using a Multimodal Deep Learning Network","volume":"40","author":"Han","year":"2020","journal-title":"Biocybern. Biomed. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Alreshidi, I., Moulitsas, I., and Jenkins, K. (2022). Miscellaneous EEG Preprocessing and Machine Learning for Pilots\u2019 Mental States Classification: Implications, Association for Computing Machinery.","DOI":"10.1145\/3571560.3571565"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1016\/j.neuroimage.2017.06.030","article-title":"Autoreject: Automated Artifact Rejection for MEG and EEG Data","volume":"159","author":"Jas","year":"2017","journal-title":"Neuroimage"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Bonassi, A., Ghilardi, T., Gabrieli, G., Truzzi, A., Doi, H., Borelli, J., Lepri, B., Shinohara, K., and Esposito, G. (2021). The Recognition of Cross-Cultural Emotional Faces Is Affected by Intensity and Ethnicity in a Japanese Sample. Behav. Sci., 11.","DOI":"10.3390\/bs11050059"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Pousson, J., Voicikas, A., Bernhofs, V., Pipinis, E., Burmistrova, L., Lin, Y., and Gri\u0161kova-Bulanova, I. (2021). Spectral Characteristics of EEG during Active Emotional Musical Performance. Sensors, 21.","DOI":"10.3390\/s21227466"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Harrivel, A., Stephens, C., Milletich, R., Heinich, C., Last, M., Napoli, N., Abraham, N., Prinzel, L., Motter, M., and Pope, A. (2017, January 9\u201313). Prediction of Cognitive States during Flight Simulation Using Multimodal Psychophysiological Sensing. Proceedings of the AIAA Information Systems\u2014AIAA Infotech at Aerospace, Grapevine, TX, USA.","DOI":"10.2514\/6.2017-1135"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hasan, M.J., Shon, D., Im, K., Choi, H.K., Yoo, D.S., and Kim, J.M. (2020). Sleep State Classification Using Power Spectral Density and Residual Neural Network with Multichannel EEG Signals. Appl. Sci., 10.","DOI":"10.3390\/app10217639"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3907","DOI":"10.1109\/TIM.2018.2885608","article-title":"Pilots\u2019 Fatigue Status Recognition Using Deep Contractive Autoencoder Network","volume":"68","author":"Wu","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2703513","DOI":"10.1155\/2018\/2703513","article-title":"A Machine Learning Approach to the Detection of Pilot\u2019s Reaction to Unexpected Events Based on EEG Signals","volume":"2018","author":"Binias","year":"2018","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ji, L., Yi, L., Li, H., Han, W., and Zhang, N. (2024). Detection of Pilots\u2019 Psychological Workload during Turning Phases Using EEG Characteristics. Sensors, 24.","DOI":"10.3390\/s24165176"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2561","DOI":"10.1017\/aer.2024.36","article-title":"\u03b2-wave-based exploration of sensitive EEG features and classification of situation awareness","volume":"128","author":"Feng","year":"2024","journal-title":"Aeronaut. J."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"920","DOI":"10.1109\/TBME.2011.2172210","article-title":"Multiclass Brain-Computer Interface Classification by Riemannian Geometry","volume":"59","author":"Barachant","year":"2012","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Majidov, I., and Whangbo, T. (2019). Efficient Classification of Motor Imagery Electroencephalography Signals Using Deep Learning Methods. Sensors, 19.","DOI":"10.3390\/s19071736"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Johnson, M., Blanco, J., Gentili, R., Jaquess, K., Oh, H., and Hatfield, B. (2015, January 22\u201324). Probe-Independent EEG Assessment of Mental Workload in Pilots. Proceedings of the International IEEE\/EMBS Conference on Neural Engineering, NER, Montpellier, France.","DOI":"10.1109\/NER.2015.7146689"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Harrivel, A., Liles, C., Stephens, C., Ellis, K., Prinzel, L., and Pope, A. (2016, January 4\u20138). Psychophysiological Sensing and State Classification for Attention Management in Commercial Aviation. Proceedings of the AIAA Infotech @ Aerospace Conference, Crew Systems and Aviation Operations Branch, NASA Langley Research Center, Hampton, VA, USA.","DOI":"10.2514\/6.2016-1490"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"e35872","DOI":"10.1016\/j.heliyon.2024.e35872","article-title":"Spying with a pilot\u2019s eye: Using eye tracking to investigate pilots\u2019 attention allocation and workload during helicopter autorotative gliding","volume":"10","author":"Cheng","year":"2024","journal-title":"Heliyon"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1007\/s41872-025-00314-9","article-title":"The drowsy driver detection for accident mitigation using facial recognition system","volume":"14","author":"Samy","year":"2025","journal-title":"Life Cycle Reliab. Saf. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Alreshidi, I., Moulitsas, I., and Jenkins, K.W. (2023). Multimodal Approach for Pilot Mental State Detection Based on EEG. Sensors, 23.","DOI":"10.3390\/s23177350"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/503\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:53:29Z","timestamp":1760032409000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/503"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,17]]},"references-count":29,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["info16060503"],"URL":"https:\/\/doi.org\/10.3390\/info16060503","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,17]]}}}