{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T18:39:24Z","timestamp":1785436764328,"version":"3.56.0"},"reference-count":40,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T00:00:00Z","timestamp":1782172800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Sichuan Science and Technology Program","award":["2024ZDZX0046"],"award-info":[{"award-number":["2024ZDZX0046"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U2333209"],"award-info":[{"award-number":["U2333209"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"award":["U2333209"],"award-info":[{"award-number":["U2333209"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]},{"name":"Sichuan Provincial Civil Aviation Flight Technology and Flight Safety Engineering Research Center","award":["GY2025-07C"],"award-info":[{"award-number":["GY2025-07C"]}]},{"award":["GY2025-07C"],"award-info":[{"award-number":["GY2025-07C"]}],"id":[{"id":"https:\/\/ror.org\/032t23b63","id-type":"ROR","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","award":["26CAFUC03045"],"award-info":[{"award-number":["26CAFUC03045"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Bioengineering"],"abstract":"<jats:p>Fatigue detection in remote tower air traffic controllers is important for maintaining operational safety under sustained visual monitoring and high cognitive workload. This study proposes MFD-Net, a dual-stream multimodal fusion framework using eye-tracking and electrocardiogram (ECG) signals. The model separately encodes eye-tracking and ECG-derived temporal inputs, incorporates an ECG-derived RMSSD expert feature, and performs lightweight late fusion for fatigue-state classification. Under the mixed-subject random-window protocol, MFD-Net achieved an Accuracy of 85.20%, a Recall of 83.33%, and an AUC of 0.9337. Because overlapping windows from the same participant and scenario could appear in both training and test sets, this result should be interpreted as a potentially optimistic within-distribution estimate. Under the stricter zero-shot leave-one-subject-out (LOSO) protocol, performance decreased substantially, with an Accuracy of 70.95\u00b121.59%, a Recall of 22.98\u00b136.30%, and an AUC of 0.6025\u00b10.2984. This low zero-shot Recall indicates limited subject-independent fatigue-detection capability. Lightweight target-subject calibration and sequential probability aggregation improved adaptation and temporal stability, although the calibration results should be interpreted cautiously because random target-subject windows were used for fine-tuning. These findings suggest that eye-tracking and ECG fusion are promising under controlled conditions, while practical deployment requires deployment-oriented calibration protocols, recall-oriented optimization, and further real-world validation.<\/jats:p>","DOI":"10.3390\/bioengineering13070717","type":"journal-article","created":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T08:14:12Z","timestamp":1782375252000},"page":"717","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Dual-Stream Fusion of Eye-Tracking and ECG Signals for Fatigue Detection in Remote Tower Air Traffic Controllers"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-5725-0170","authenticated-orcid":false,"given":"Dajiang","family":"Song","sequence":"first","affiliation":[{"name":"Key Laboratory of Flight Techniques and Flight Safety, Civil Aviation Flight University of China, Guanghan 618307, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weijun","family":"Pan","sequence":"additional","affiliation":[{"name":"Key Laboratory of Flight Techniques and Flight Safety, Civil Aviation Flight University of China, Guanghan 618307, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4022-7424","authenticated-orcid":false,"given":"Hugo","family":"Gamboa","sequence":"additional","affiliation":[{"name":"LIBPhys (Laboratory for Instrumentation, Biomedical Engineering and Radiation Physics), Department of Physics, NOVA School of Science and Technology, NOVA University Lisbon, Largo da Torre, 2829-516 Caparica, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2765-670X","authenticated-orcid":false,"given":"Zirui","family":"Yin","sequence":"additional","affiliation":[{"name":"School of Transportation and Logistics, Southwest Jiaotong University, Chengdu 611756, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shengjie","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Flight Techniques and Flight Safety, Civil Aviation Flight University of China, Guanghan 618307, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,23]]},"reference":[{"key":"ref_1","unstructured":"International Civil Aviation Organization (2016). Manual for the Oversight of Fatigue Management Approaches, International Civil Aviation Organization."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1177\/1555343415572631","article-title":"Situation awareness misconceptions and misunderstandings","volume":"9","author":"Endsley","year":"2015","journal-title":"J. Cogn. Eng. Decis. Mak."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.neubiorev.2012.10.003","article-title":"Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness","volume":"44","author":"Borghini","year":"2014","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lohani, M., Payne, B.R., and Strayer, D.L. (2019). A review of psychophysiological measures to assess cognitive states in real-world driving. Front. Hum. Neurosci., 13.","DOI":"10.3389\/fnhum.2019.00057"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"026017","DOI":"10.1088\/1741-2552\/aa5a98","article-title":"A multimodal approach to estimating vigilance using EEG and forehead EOG","volume":"14","author":"Zheng","year":"2017","journal-title":"J. Neural Eng."},{"key":"ref_6","first-page":"423","article-title":"Multimodal machine learning: A survey and taxonomy","volume":"41","author":"Ahuja","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1109\/MSP.2017.2738401","article-title":"Deep multimodal learning: A survey on recent advances and trends","volume":"34","author":"Ramachandram","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1518\/001872095779049499","article-title":"Measurement of situation awareness in dynamic systems","volume":"37","author":"Endsley","year":"1995","journal-title":"Hum. Factors"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1177\/0018720820969071","article-title":"Physiological measurements of situation awareness: A systematic review","volume":"65","author":"Zhang","year":"2023","journal-title":"Hum. Factors"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.tra.2018.06.005","article-title":"Multiple remote tower for Single European Sky: The evolution from initial operational concept to regulatory approved implementation","volume":"116","author":"Kearney","year":"2018","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.ergon.2018.05.005","article-title":"How much is too much on monitoring tasks? Visual scan patterns of single air traffic controller performing multiple remote tower operations","volume":"67","author":"Li","year":"2018","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"110179","DOI":"10.1016\/j.knosys.2022.110179","article-title":"Recognising situation awareness associated with different workloads using EEG and eye-tracking features in air traffic control tasks","volume":"260","author":"Li","year":"2023","journal-title":"Knowl.-Based Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"101698","DOI":"10.1016\/j.aei.2022.101698","article-title":"Towards safe and collaborative aerodrome operations: Assessing shared situational awareness for adverse weather detection with EEG-enabled Bayesian neural networks","volume":"53","author":"Yiu","year":"2022","journal-title":"Adv. Eng. Inform."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"108047","DOI":"10.1016\/j.knosys.2021.108047","article-title":"Artificial intelligence-enabled non-intrusive vigilance assessment approach to reducing traffic controller\u2019s human errors","volume":"239","author":"Li","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Liang, R., Pan, W., Zuo, Q., Zhang, C., Chen, S., Chen, S., and Deng, L. (2025). Modeling visual fatigue in remote tower air traffic controllers: A multimodal physiological data-based approach. Aerospace, 12.","DOI":"10.3390\/aerospace12060474"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103506","DOI":"10.1016\/j.aei.2025.103506","article-title":"RTFnet: A multi-modal physiological data fusion approach for fatigue detection in remote tower controllers","volume":"67","author":"Yin","year":"2025","journal-title":"Adv. Eng. Inform."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ad\u00e3o Martins, N.R., Annaheim, S., Spengler, C.M., and Rossi, R.M. (2021). Fatigue monitoring through wearables: A state-of-the-art review. Front. Physiol., 12.","DOI":"10.3389\/fphys.2021.790292"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1080\/24721840.2018.1514978","article-title":"Eye-tracking measures in aviation: A selective literature review","volume":"28","author":"Wickens","year":"2018","journal-title":"Int. J. Aerosp. Psychol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1080\/10508414.2017.1313096","article-title":"Gaze behavior and visual attention: A review of eye tracking studies in aviation","volume":"26","author":"Ziv","year":"2016","journal-title":"Int. J. Aviat. Psychol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.apergo.2019.01.012","article-title":"The effects of flight complexity on gaze entropy: An experimental study with fighter pilots","volume":"77","author":"Rieiro","year":"2019","journal-title":"Appl. Ergon."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1007\/s00421-003-0807-5","article-title":"Experimental evaluation of eye-blink parameters as a drowsiness measure","volume":"89","author":"Caffier","year":"2003","journal-title":"Eur. J. Appl. Physiol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"zpad006","DOI":"10.1093\/sleepadvances\/zpad006","article-title":"PERCLOS-based technologies for detecting drowsiness: Current evidence and future directions","volume":"4","author":"Abe","year":"2023","journal-title":"Sleep Adv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Forte, G., Favieri, F., and Casagrande, M. (2019). Heart rate variability and cognitive function: A systematic review. Front. Neurosci., 13.","DOI":"10.3389\/fnins.2019.00710"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Shaffer, F., Meehan, Z.M., and Zerr, C.L. (2020). A critical review of ultra-short-term heart rate variability norms research. Front. Neurosci., 14.","DOI":"10.3389\/fnins.2020.594880"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Dasari, D., Shou, G., and Ding, L. (2017). ICA-derived EEG correlates to mental fatigue, effort, and workload in a realistically simulated air traffic control task. Front. Neurosci., 11.","DOI":"10.3389\/fnins.2017.00297"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ahn, S., Nguyen, T., Jang, H., Kim, J.G., and Jun, S.C. (2016). Exploring neurophysiological correlates of drivers\u2019 mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and fNIRS data. Front. Hum. Neurosci., 10.","DOI":"10.3389\/fnhum.2016.00219"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Rim, B., Sung, N.J., Min, S., and Hong, M. (2020). Deep learning in physiological signal data: A survey. Sensors, 20.","DOI":"10.3390\/s20040969"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.eswa.2018.07.054","article-title":"Automatic driver sleepiness detection using EEG, EOG and contextual information","volume":"115","author":"Barua","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"114693","DOI":"10.1016\/j.eswa.2021.114693","article-title":"Driver stress detection via multimodal fusion using attention-based CNN-LSTM","volume":"173","author":"Mou","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kong, L., Xie, K., Niu, K., He, J., and Zhang, W. (2024). Remote photoplethysmography and motion tracking convolutional neural network with bidirectional long short-term memory: Non-invasive fatigue detection method based on multi-modal fusion. Sensors, 24.","DOI":"10.3390\/s24020455"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"132818","DOI":"10.1016\/j.eswa.2026.132818","article-title":"Empirical comparison of sequence-aware augmentation strategies for real operational AHU AFDD under data scarcity and class imbalance","volume":"326","author":"Wang","year":"2026","journal-title":"Expert Syst. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1016\/j.eswa.2019.02.005","article-title":"Cross-subject driver status detection from physiological signals based on hybrid feature selection and transfer learning","volume":"137","author":"Chen","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zeng, H., Li, X., Borghini, G., Zhao, Y., Arico, P., Di Flumeri, G., Sciaraffa, N., Zakaria, W., Kong, W., and Babiloni, F. (2021). An EEG-based transfer learning method for cross-subject fatigue mental state prediction. Sensors, 21.","DOI":"10.3390\/s21072369"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ko, W., Jeon, E., Jeong, S., Phyo, J., and Suk, H.-I. (2021). A survey on deep learning-based short\/zero-calibration approaches for EEG-based brain\u2013computer interfaces. Front. Hum. Neurosci., 15.","DOI":"10.3389\/fnhum.2021.643386"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/TCDS.2020.3007453","article-title":"Transfer learning for EEG-based brain\u2013computer interfaces: A review of progress made since 2016","volume":"14","author":"Wu","year":"2020","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/s13748-016-0094-0","article-title":"Learning from imbalanced data: Open challenges and future directions","volume":"5","author":"Krawczyk","year":"2016","journal-title":"Prog. Artif. Intell."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"789","DOI":"10.1007\/s10115-022-01780-8","article-title":"AdaCC: Cumulative cost-sensitive boosting for imbalanced classification","volume":"65","author":"Iosifidis","year":"2023","journal-title":"Knowl. Inf. Syst."},{"key":"ref_38","unstructured":"Samn, S.W., and Perelli, L.P. (1982). Estimating Aircrew Fatigue: A Technique with Application to Airlift Operations."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1037\/1082-989X.10.2.178","article-title":"Use of the extreme groups approach: A critical reexamination and new recommendations","volume":"10","author":"Preacher","year":"2005","journal-title":"Psychol. Methods"},{"key":"ref_40","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."}],"container-title":["Bioengineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5354\/13\/7\/717\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T04:33:45Z","timestamp":1783053225000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5354\/13\/7\/717"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,23]]},"references-count":40,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["bioengineering13070717"],"URL":"https:\/\/doi.org\/10.3390\/bioengineering13070717","relation":{},"ISSN":["2306-5354"],"issn-type":[{"value":"2306-5354","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,23]]}}}