{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T01:48:27Z","timestamp":1792806507683,"version":"4.5.0"},"reference-count":41,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2024,10,14]],"date-time":"2024-10-14T00:00:00Z","timestamp":1728864000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["51775325"],"award-info":[{"award-number":["51775325"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U21A20121"],"award-info":[{"award-number":["U21A20121"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2023Z218"],"award-info":[{"award-number":["2023Z218"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Joint Funds of the National Natural Science Foundation of China","award":["51775325"],"award-info":[{"award-number":["51775325"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Joint Funds of the National Natural Science Foundation of China","award":["U21A20121"],"award-info":[{"award-number":["U21A20121"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Joint Funds of the National Natural Science Foundation of China","award":["2023Z218"],"award-info":[{"award-number":["2023Z218"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Key Research and Development Program of Ningbo","award":["51775325"],"award-info":[{"award-number":["51775325"]}]},{"name":"Key Research and Development Program of Ningbo","award":["U21A20121"],"award-info":[{"award-number":["U21A20121"]}]},{"name":"Key Research and Development Program of Ningbo","award":["2023Z218"],"award-info":[{"award-number":["2023Z218"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Motion sickness is a common issue in electric vehicles, significantly impacting passenger comfort. This study aims to develop a functional brain network analysis model by integrating electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals to evaluate motion sickness symptoms. During real-world testing with the Feifan F7 series of new energy-electric vehicles from SAIC Motor Corp, data were collected from 32 participants. The EEG signals were divided into four frequency bands: delta-range, theta-range, alpha-range, and beta-range, and brain oxygenation variation was calculated from the fNIRS signals. Functional connectivity between brain regions was measured to construct functional brain network models for motion sickness analysis. A motion sickness detection model was developed using a graph convolutional network (GCN) to integrate EEG and fNIRS data. Our results show significant differences in brain functional connectivity between participants in motion and non-motion sickness states. The model that combined fNIRS data with high-frequency EEG signals achieved the best performance, improving the F1 score by 11.4% compared to using EEG data alone and by 8.2% compared to using fNIRS data alone. These results highlight the effectiveness of integrating EEG and fNIRS signals using GCN for motion sickness detection. They demonstrate the model\u2019s superiority over single-modality approaches, showcasing its potential for real-world applications in electric vehicles.<\/jats:p>","DOI":"10.3390\/s24206613","type":"journal-article","created":{"date-parts":[[2024,10,16]],"date-time":"2024-10-16T07:58:32Z","timestamp":1729065512000},"page":"6613","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["A Brain Network Analysis Model for Motion Sickness in Electric Vehicles Based on EEG and fNIRS Signal Fusion"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2551-4839","authenticated-orcid":false,"given":"Bin","family":"Ren","sequence":"first","affiliation":[{"name":"Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China"},{"name":"Zhejiang Key Laboratory of Robotics and Intelligent Manufacturing Equipment Technology, Ningbo Institute of Materials Technology & Engineering, Chinese Academy of Sciences, Ningbo 315201, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pengyu","family":"Ren","sequence":"additional","affiliation":[{"name":"Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenfa","family":"Luo","sequence":"additional","affiliation":[{"name":"SAIC Motor R&D Innovation Headquarters, SAIC Motor Corporation Limited, Shanghai 201804, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingze","family":"Xin","sequence":"additional","affiliation":[{"name":"SAIC Motor R&D Innovation Headquarters, SAIC Motor Corporation Limited, Shanghai 201804, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1339","DOI":"10.1177\/0018720819876139","article-title":"Knowing What\u2019s Coming: Unpredictable Motion Causes More Motion Sickness","volume":"62","author":"Kuiper","year":"2020","journal-title":"Hum. Factors J. Hum. Factors Ergon. Soc."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"\u2018Atifah Saruchi, S., Izni, N.A., Ariff, M.H.M., and Wahid, N. (2022). A Brief Review on Motion Sickness for Autonomous Vehicle. Enabling Industry 4.0 through Advances in Mechatronics, Springer. Lecture Notes in Electrical Engineering.","DOI":"10.1007\/978-981-19-2095-0_24"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hwang, J.-U., Bang, J.-S., and Lee, S.-W. (2022, January 9\u201312). Classification of Motion Sickness Levels Using Multimodal Biosignals in Real Driving Conditions. Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Prague, Czech Republic.","DOI":"10.1109\/SMC53654.2022.9945559"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1918","DOI":"10.1109\/TBME.2011.2139210","article-title":"Electrophysiological Imaging of Brain Activity and Connectivity\u2014Challenges and Opportunities","volume":"58","author":"He","year":"2011","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Qin, B., Wu, B., Zhou, L., Chen, Y., Qian, Z., and Zhu, Q. (2021, January 13\u201314). Study on Motion Sickness Based on EEG Power Spectrum Characteristics. Proceedings of the 2021 IEEE International Conference on Medical Imaging Physics and Engineering (ICMIPE), Hefei, China.","DOI":"10.1109\/ICMIPE53131.2021.9698957"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yeo, S.S., Kwon, J.W., and Park, S.Y. (2022). EEG-Based Analysis of Various Sensory Stimulation Effects to Reduce Visually Induced Motion Sickness in Virtual Reality. Sci. Rep., 12.","DOI":"10.1038\/s41598-022-21307-z"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Li, X., Zhu, C., Xu, C., Zhu, J., Li, Y., and Wu, S. (2020). VR Motion Sickness Recognition by Using EEG Rhythm Energy Ratio Based on Wavelet Packet Transform. Comput. Methods Programs Biomed., 188.","DOI":"10.1016\/j.cmpb.2019.105266"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5442","DOI":"10.1109\/TITS.2023.3240407","article-title":"Motion Sickness Prediction Based on Dry EEG in Real Driving Environment","volume":"24","author":"Bang","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.neuroimage.2013.11.033","article-title":"Twenty Years of Functional Near-Infrared Spectroscopy: Introduction for the Special Issue","volume":"85","author":"Boas","year":"2014","journal-title":"Neuroimage"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kamran, M.A., Mannan, M.M.N., and Jeong, M.Y. (2016). Cortical Signal Analysis and Advances in Functional Near-Infrared Spectroscopy Signal: A Review. Front. Hum. Neurosci., 10.","DOI":"10.3389\/fnhum.2016.00261"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Balardin, J.B., Zimeo Morais, G.A., Furucho, R.A., Trambaiolli, L., Vanzella, P., Biazoli, C., and Sato, J.R. (2017). Imaging Brain Function with Functional Near-Infrared Spectroscopy in Unconstrained Environments. Front. Hum. Neurosci., 11.","DOI":"10.3389\/fnhum.2017.00258"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"207415","DOI":"10.1109\/ACCESS.2020.3038039","article-title":"Analysis of Motion Sickness Associated Brain Activity Using FNIRS: A Driving Simulator Study","volume":"8","author":"Zhang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_13","unstructured":"Kinoshita, F., Okuno, H., Touyama, H., Takada, M., Miyao, M., and Takada, H. (2020). Effect of Background Element Difference on Regional Cerebral Blood Flow While Viewing Stereoscopic Video Clips. Universal Access in Human-Computer Interaction. Design Approaches and Supporting Technologies: 14th International Conference, UAHCI 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, 19\u201324 July 2020, Springer."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Hoppes, C.W., Sparto, P.J., Whitney, S.L., Furman, J.M., and Huppert, T.J. (2018). Functional Near-Infrared Spectroscopy during Optic Flow with and without Fixation. PLoS ONE, 13.","DOI":"10.1371\/journal.pone.0193710"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ren, B., Guan, W., and Zhou, Q. (2023). Study of Motion Sickness Model Based on FNIRS Multiband Features during Car Rides. Diagnostics, 13.","DOI":"10.3390\/diagnostics13081462"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ren, B., and Zhou, Q. (2023). Assessing Passengers\u2019 Motion Sickness Levels Based on Cerebral Blood Oxygen Signals and Simulation of Actual Ride Sensation. Diagnostics, 13.","DOI":"10.3390\/diagnostics13081403"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"S220","DOI":"10.1016\/j.neuroimage.2004.07.013","article-title":"Modeling the Hemodynamic Response to Brain Activation","volume":"23","author":"Buxton","year":"2004","journal-title":"Neuroimage"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Deligani, R.J., Borgheai, S.B., McLinden, J., and Shahriari, Y. (2021). Multimodal Fusion of EEG-FNIRS: A Mutual Information-Based Hybrid Classification Framework. Biomed. Opt. Express, 12.","DOI":"10.1364\/BOE.413666"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1198","DOI":"10.1109\/TNSRE.2020.2980772","article-title":"Enhancing Communication for People in Late-Stage ALS Using an FNIRS-Based BCI System","volume":"28","author":"Borgheai","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1109\/MSP.2021.3128348","article-title":"Deep Learning in Neuroimaging: Promises and Challenges","volume":"39","author":"Yan","year":"2022","journal-title":"IEEE Signal Process Mag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"284","DOI":"10.1016\/j.neuroscience.2018.06.049","article-title":"Detecting Concealed Information with Fused Electroencephalography and Functional Near-Infrared Spectroscopy","volume":"386","author":"Lin","year":"2018","journal-title":"Neuroscience"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Khan, M.U., and Hasan, M.A.H. (2020). Hybrid EEG-FNIRS BCI Fusion Using Multi-Resolution Singular Value Decomposition (MSVD). Front. Hum. Neurosci., 14.","DOI":"10.3389\/fnhum.2020.599802"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.neuroimage.2016.05.058","article-title":"Hemodynamic Correlates of Spontaneous Neural Activity Measured by Human Whole-Head Resting State EEG + FNIRS","volume":"138","author":"Keles","year":"2016","journal-title":"Neuroimage"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Khan, M.J., and Hong, K.-S. (2017). Hybrid EEG\u2013FNIRS-Based Eight-Command Decoding for BCI: Application to Quadcopter Control. Front. Neurorobot., 11.","DOI":"10.3389\/fnbot.2017.00006"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Hong, K.-S., and Khan, M.J. (2017). Hybrid Brain\u2013Computer Interface Techniques for Improved Classification Accuracy and Increased Number of Commands: A Review. Front. Neurorobot., 11.","DOI":"10.3389\/fnbot.2017.00035"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Berger, A., Pixa, N.H., Steinberg, F., and Doppelmayr, M. (2018). Brain Oscillatory and Hemodynamic Activity in a Bimanual Coordination Task Following Transcranial Alternating Current Stimulation (TACS): A Combined EEG-FNIRS Study. Front. Behav. Neurosci., 12.","DOI":"10.3389\/fnbeh.2018.00067"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ge, S., Wang, P., Liu, H., Lin, P., Gao, J., Wang, R., Iramina, K., Zhang, Q., and Zheng, W. (2019). Neural Activity and Decoding of Action Observation Using Combined EEG and FNIRS Measurement. Front. Hum. Neurosci., 13.","DOI":"10.3389\/fnhum.2019.00357"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1007\/BF00199545","article-title":"The Connectivity of the Brain: Multi-Level Quantitative Analysis","volume":"73","author":"Murre","year":"1995","journal-title":"Biol. Cybern."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Li, X., La, R., Wang, Y., Hu, B., and Zhang, X. (2020). A Deep Learning Approach for Mild Depression Recognition Based on Functional Connectivity Using Electroencephalography. Front. Neurosci., 14.","DOI":"10.3389\/fnins.2020.00192"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Cao, J., Garro, E.M., and Zhao, Y. (2022). EEG\/FNIRS Based Workload Classification Using Functional Brain Connectivity and Machine Learning. Sensors, 22.","DOI":"10.3390\/s22197623"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Jiang, Y., Wang, P., Liu, H., and Ge, S. (2019, January 12\u201315). Decoding Action Observation Using Complex Brain Networks from Simultaneously Recorded EEG-FNIRS Signals. Proceedings of the Neural Information Processing: 26th International Conference, ICONIP 2019, Sydney, NSW, Australia.","DOI":"10.1007\/978-3-030-36808-1_61"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.autneu.2016.10.003","article-title":"Motion Sickness Increases Functional Connectivity between Visual Motion and Nausea-Associated Brain Regions","volume":"202","author":"Toschi","year":"2017","journal-title":"Auton. Neurosci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1517","DOI":"10.1113\/JP277474","article-title":"Functional Brain Networks and Neuroanatomy Underpinning Nausea Severity Can Predict Nausea Susceptibility Using Machine Learning","volume":"597","author":"Ruffle","year":"2019","journal-title":"J. Physiol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1007\/s10620-019-05766-5","article-title":"Central Nervous System Mechanisms of Nausea in Gastroparesis: An FMRI-Based Case\u2013Control Study","volume":"65","author":"Snodgrass","year":"2020","journal-title":"Dig. Dis. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"N\u00fcrnberger, M., Klingner, C., Witte, O.W., and Brodoehl, S. (2021). Mismatch of Visual-Vestibular Information in Virtual Reality: Is Motion Sickness Part of the Brains Attempt to Reduce the Prediction Error?. Front. Hum. Neurosci., 15.","DOI":"10.3389\/fnhum.2021.757735"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.neuroimage.2010.04.256","article-title":"ICA-Based Muscle Artefact Correction of EEG Data: What Is Muscle and What Is Brain?","volume":"54","author":"Olbrich","year":"2011","journal-title":"Neuroimage"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2098","DOI":"10.1016\/S1388-2457(01)00668-X","article-title":"EEG-Defined Subtypes of Children with Attention-Deficit\/Hyperactivity Disorder","volume":"112","author":"Clarke","year":"2001","journal-title":"Clin. Neurophysiol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1038\/nrn2575","article-title":"Complex Brain Networks: Graph Theoretical Analysis of Structural and Functional Systems","volume":"10","author":"Bullmore","year":"2009","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1548","DOI":"10.1016\/j.neuroimage.2011.01.055","article-title":"An Improved Index of Phase-Synchronization for Electrophysiological Data in the Presence of Volume-Conduction, Noise and Sample-Size Bias","volume":"55","author":"Vinck","year":"2011","journal-title":"Neuroimage"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1089\/brain.2022.0082","article-title":"Threshold Selection for Brain Connectomes","volume":"13","author":"Theis","year":"2023","journal-title":"Brain Connect."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"026010","DOI":"10.1088\/1741-2552\/ab7613","article-title":"Graph Theory Analysis of Directed Functional Brain Networks in Major Depressive Disorder Based on EEG Signal","volume":"17","author":"Hasanzadeh","year":"2020","journal-title":"J. Neural Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/20\/6613\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:12:55Z","timestamp":1760112775000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/20\/6613"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,14]]},"references-count":41,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["s24206613"],"URL":"https:\/\/doi.org\/10.3390\/s24206613","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,14]]}}}