{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T19:56:21Z","timestamp":1781639781791,"version":"3.54.5"},"reference-count":21,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2025,8,26]],"date-time":"2025-08-26T00:00:00Z","timestamp":1756166400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["52072293"],"award-info":[{"award-number":["52072293"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>In the field of cognitive workload assessment for aerospace training, existing methods exhibit significant limitations in unimodal feature extraction and in leveraging complementary synergy among multimodal signals, while current fusion paradigms struggle to effectively capture nonlinear dynamic coupling characteristics across modalities. This study proposes DST-Net (Cross-Modal Downsampling Transformer Network), which synergistically integrates pilots\u2019 multimodal physiological signals (electromyography, electrooculography, electrodermal activity) with flight dynamics data through an Anti-Aliasing and Average Pooling LSTM (AAL-LSTM) data fusion strategy combined with cross-modal attention mechanisms. Evaluation on the \u201cCogPilot\u201d dataset for flight task difficulty prediction demonstrates that AAL-LSTM achieves substantial performance improvements over existing approaches (AUC = 0.97, F1 Score = 94.55). Given the dataset\u2019s frequent sensor data missingness, the study further enhances simulated flight experiments. By incorporating eye-tracking features via cross-modal attention mechanisms, the upgraded DST-Net framework achieves even higher performance (AUC = 0.998, F1 Score = 97.95) and reduces the root mean square error (RMSE) of cumulative flight error prediction to 1750. These advancements provide critical support for safety-critical aviation training systems.<\/jats:p>","DOI":"10.3390\/mti9090089","type":"journal-article","created":{"date-parts":[[2025,8,26]],"date-time":"2025-08-26T14:43:18Z","timestamp":1756219398000},"page":"89","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Cognitive Workload Assessment in Aerospace Scenarios: A Cross-Modal Transformer Framework for Multimodal Physiological Signal Fusion"],"prefix":"10.3390","volume":"9","author":[{"given":"Pengbo","family":"Wang","sequence":"first","affiliation":[{"name":"School of Mechatronic Engineering, Xi\u2019an Technological University, Xi\u2019an 710000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongxi","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Mechatronic Engineering, Xi\u2019an Technological University, Xi\u2019an 710000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Heming","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Optical Engineering, Xi\u2019an Technological University, Xi\u2019an 710000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"44","DOI":"10.23890\/IJAST.vm01is02.0201","article-title":"An exploratory study about the challenges with pilot training and recruitment in Europe","volume":"1","author":"Adanov","year":"2020","journal-title":"Int. J. Aviat. Sci. Technol."},{"key":"ref_2","unstructured":"Air Force Personnel Center (2021). Statistical Analysis & Retrieval System: Interactive Demographics Analysis System [Data Set], U.S. Air Force."},{"key":"ref_3","unstructured":"McCarthy, N. (2024, August 23). Infographic: The Cost of Training U.S Air Force Pilots. Available online: https:\/\/www.statista.com\/chart\/17661\/estimated-total-cost-of-training-a-basic-qualified-pilot-by-platform\/."},{"key":"ref_4","first-page":"198","article-title":"An air force pilot training recommendation system using advanced analytical methods","volume":"52","author":"Forrest","year":"2022","journal-title":"Inf. J. Appl. Anal."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2242","DOI":"10.1080\/00140139.2023.2194592","article-title":"Integrating physiological monitoring systems in military aviation: A brief narrative review of its importance, opportunities, and risks","volume":"66","author":"Shaw","year":"2023","journal-title":"Ergonomics"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Shaw, D.M., Cabre, G., and Gant, N. (2021). Hypoxic Hypoxia and Brain Function in Military Aviation: Basic Physiology and Applied Perspectives. Front. Physiol., 12.","DOI":"10.3389\/fphys.2021.665821"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.psychsport.2014.08.005","article-title":"Investigating the Efficacy of Neurofeedback Training for Expediting Expertise and Excellence in Sport","volume":"16","author":"Ring","year":"2015","journal-title":"Psychol. Sport Exerc."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Duchowski, A.T., Krejtz, K., Krejtz, I., Biele, C., Niedzielska, A., Kiefer, P., Martin, R., and Giannopoulos, I. (2018, January 21\u201327). The index of pupillary activity: Measuring cognitive load vis-\u00e0-vis task difficulty with pupil oscillation. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3173574.3173856"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"102220","DOI":"10.1016\/j.inffus.2023.102220","article-title":"Incongruity-aware multimodal physiology signals fusion for emotion recognition","volume":"105","author":"Li","year":"2024","journal-title":"Inf. Fusion"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/j.inffus.2021.11.006","article-title":"Multi-sensor information fusion based on machine learning for real applications in human activity recognition: State-of-the-art and research challenges","volume":"80","author":"Qiu","year":"2022","journal-title":"Inf. Fusion"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liggins, M., Hall, D., and Llinas, J. (2017). Handbook of Multisensor Data Fusion: Theory and Practice, CRC Press.","DOI":"10.1201\/9781420053098"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4386","DOI":"10.1109\/TCYB.2020.2987575","article-title":"Emotion recognition from multimodal physiological signals using a regularized deep fusion of kernel machine","volume":"51","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"5012","DOI":"10.1109\/JSEN.2020.3033256","article-title":"Classification of task weight during dynamic motion using EEG\u2013EMG fusion","volume":"21","author":"Tryon","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"21642","DOI":"10.1109\/ACCESS.2021.3055933","article-title":"Multi-modal physiological data fusion for affect estimation using deep learning","volume":"9","author":"Hssayeni","year":"2021","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.neuroimage.2013.11.007","article-title":"Multimodal fusion framework: A multiresolution approach for emotion classification and recognition from physiological signals","volume":"102","author":"Verma","year":"2014","journal-title":"NeuroImage"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1391","DOI":"10.1109\/TAFFC.2021.3093923","article-title":"Multimodal affective states recognition based on multiscale cnns and biologically inspired decision fusion model","volume":"14","author":"Zhao","year":"2021","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"187","DOI":"10.3389\/fnhum.2018.00187","article-title":"In silico vs. over the clouds: On-the-fly mental state estimation of aircraft pilots, using a functional near infrared spectroscopy based passive-BCI","volume":"12","author":"Gateau","year":"2018","journal-title":"Front. Hum. Neurosci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1660","DOI":"10.1016\/j.aap.2011.03.024","article-title":"Do young novice drivers overestimate their driving skills more than experienced drivers? Different methods lead to different conclusions","volume":"43","author":"Twisk","year":"2011","journal-title":"Accid. Anal. Prev."},{"key":"ref_19","unstructured":"Jie, Y.E., Kah, M.T., Tan, S., and Xue, Y. (2024, August 23). cogpilotdatachallenge [EB\/OL]. GitHub. Available online: https:\/\/github.com\/skulu\/cogpilotdatachallenge."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"120711","DOI":"10.1016\/j.eswa.2023.120711","article-title":"Toward Automated Instructor Pilots in Legacy Air Force Systems: Physiology-Based Flight Difficulty Classification Via Machine Learning","volume":"231","author":"Caballero","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lee, N., Moore, P.W., and Brattain, L.J. (2023, January 25\u201329). Scalable Deep Learning for Pilot Performance Analysis Using Multimodal Physiological Time Series. Proceedings of the 2023 IEEE High Performance Extreme Computing Conference (HPEC), Boston, MA, USA.","DOI":"10.1109\/HPEC58863.2023.10363513"}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/9\/9\/89\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:33:06Z","timestamp":1760034786000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/9\/9\/89"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,26]]},"references-count":21,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["mti9090089"],"URL":"https:\/\/doi.org\/10.3390\/mti9090089","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,26]]}}}