{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,15]],"date-time":"2026-08-15T22:52:55Z","timestamp":1786834375102,"version":"3.56.0"},"reference-count":127,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,11,27]],"date-time":"2024-11-27T00:00:00Z","timestamp":1732665600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Driving performance can be significantly impacted when a person experiences intense emotions behind the wheel. Research shows that emotions such as anger, sadness, agitation, and joy can increase the risk of traffic accidents. This study introduces a methodology to recognize four specific emotions using an intelligent model that processes and analyzes signals from motor activity and driver behavior, which are generated by interactions with basic driving elements, along with facial geometry images captured during emotion induction. The research applies machine learning to identify the most relevant motor activity signals for emotion recognition. Furthermore, a pre-trained Convolutional Neural Network (CNN) model is employed to extract probability vectors from images corresponding to the four emotions under investigation. These data sources are integrated through a unidimensional network for emotion classification. The main proposal of this research was to develop a multimodal intelligent model that combines motor activity signals and facial geometry images to accurately recognize four specific emotions (anger, sadness, agitation, and joy) in drivers, achieving a 96.0% accuracy in a simulated environment. The study confirmed a significant relationship between drivers' motor activity, behavior, facial geometry, and the induced emotions.<\/jats:p>","DOI":"10.3389\/frai.2024.1467051","type":"journal-article","created":{"date-parts":[[2024,11,27]],"date-time":"2024-11-27T06:34:02Z","timestamp":1732689242000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Multimodal driver emotion recognition using motor activity and facial expressions"],"prefix":"10.3389","volume":"7","author":[{"given":"Carlos H.","family":"Espino-Salinas","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huizilopoztli","family":"Luna-Garc\u00eda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jos\u00e9 M.","family":"Celaya-Padilla","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cristian","family":"Barr\u00eda-Huidobro","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nadia Karina","family":"Gamboa Rosales","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Rondon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Klinge Orlando","family":"Villalba-Condori","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,11,27]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"3643","DOI":"10.1007\/s11760-024-03029-z","article-title":"Emotion recognition of the driver based on KLT algorithm and ShuffleNet V2","volume":"18","author":"Ahmad","year":"2024","journal-title":"Signal, Image Video Proc"},{"key":"B2","doi-asserted-by":"publisher","first-page":"4445","DOI":"10.1007\/s13042-024-02158-8","article-title":"A robust semi-supervised deep learning approach for emotion recognition using EEG signals","volume":"15","author":"Al-Asadi","year":"2024","journal-title":"Int. J. Mach. Learn. Cybernet"},{"key":"B3","doi-asserted-by":"crossref","DOI":"10.1145\/3242969.3242972","article-title":"\u201cJoint discrete and continuous emotion prediction using ensemble and end-to-end approaches,\u201d","author":"AlBadawy","year":"2018","journal-title":"ICMI 2018"},{"key":"B4","doi-asserted-by":"publisher","first-page":"4750","DOI":"10.3390\/app13084750","article-title":"Speech emotion recognition through hybrid features and convolutional neural network","volume":"13","author":"Alluhaidan","year":"2023","journal-title":"Appl. Sci"},{"key":"B5","doi-asserted-by":"publisher","first-page":"118076","DOI":"10.1016\/j.enconman.2024.118076","article-title":"Faults detection and diagnosis of pv systems based on machine learning approach using random forest classifier","volume":"301","author":"Amiri","year":"2024","journal-title":"Energy Convers. Managem"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/brainsci11010106","article-title":"Single-trial recognition of video gamer expertise from brain haemodynamic and facial emotion responses","volume":"11","author":"Andreu-Perez","year":"2021","journal-title":"Brain Sci"},{"key":"B7","doi-asserted-by":"publisher","first-page":"8769","DOI":"10.3390\/app14198769","article-title":"Towards emotionally intelligent virtual environments: classifying emotions through a biosignal-based approach","volume":"14","author":"Arslan","year":"2024","journal-title":"Appl. Sci"},{"key":"B8","doi-asserted-by":"publisher","first-page":"2047","DOI":"10.3390\/mi14112047","article-title":"Classification and identification of contaminants in recyclable containers based on a recursive feature elimination-light gradient boosting machine algorithm using an electronic nose","volume":"14","author":"Ba","year":"2023","journal-title":"Micromachines"},{"key":"B9","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1007\/s12530-023-09506-z","article-title":"Facial emotion recognition and music recommendation system using CNN-based deep learning techniques","volume":"15","author":"Bakariya","year":"2024","journal-title":"Evol. Syst"},{"key":"B10","doi-asserted-by":"publisher","first-page":"3609","DOI":"10.1007\/s12652-021-03488-z","article-title":"Transfer learning for image classification using VGG19: CALTECH-101 image data set","volume":"14","author":"Bansal","year":"2023","journal-title":"J. Ambient Intell. Humaniz. Comput"},{"key":"B11","first-page":"279","volume-title":"Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution","author":"Barsoum","year":"2016"},{"key":"B12","doi-asserted-by":"publisher","first-page":"839","DOI":"10.3390\/electronics12040839","article-title":"Speech emotion recognition based on multiple acoustic features and deep convolutional neural network","volume":"12","author":"Bhangale","year":"2023","journal-title":"Electronics"},{"key":"B13","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1007\/s00138-021-01249-8","article-title":"Early, intermediate and late fusion strategies for robust deep learning-based multimodal action recognition","volume":"32","author":"Boulahia","year":"2021","journal-title":"Mach. Vis. Appl"},{"key":"B14","first-page":"77","article-title":"\u201cA comparison of emotion elicitation methods for affective driving studies,\u201d","author":"Braun","year":"2018","journal-title":"Adjunct Proceedings"},{"key":"B15","doi-asserted-by":"publisher","first-page":"121198","DOI":"10.1016\/j.ins.2024.121198","article-title":"Eeg emotion recognition using EEG-SWTNS neural network through eeg spectral image","volume":"680","author":"Cai","year":"2024","journal-title":"Inf. Sci"},{"key":"B16","doi-asserted-by":"publisher","first-page":"2455","DOI":"10.3390\/s23052455","article-title":"Emotion recognition using different sensors, emotion models, methods and datasets: A comprehensive review","volume":"23","author":"Cai","year":"2023","journal-title":"Sensors"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13634-021-00740-8","article-title":"Application of a modified Inception-v3 model in the dynasty-based classification of ancient murals","volume":"2021","author":"Cao","year":"2021","journal-title":"EURASIP J. Adv. Signal Process"},{"key":"B18","doi-asserted-by":"publisher","first-page":"7752","DOI":"10.3390\/s21227752","article-title":"In-vehicle alcohol detection using low-cost sensors and genetic algorithms to aid in the drinking and driving detection","volume":"21","author":"Celaya-Padilla","year":"2021","journal-title":"Sensors"},{"key":"B19","doi-asserted-by":"publisher","first-page":"3","DOI":"10.11591\/ijeecs.v24.i3.pp1700-1710","article-title":"An approach to categorize chest X-ray images using sparse categorical cross entropy","volume":"24","author":"Chaithanya","year":"2021","journal-title":"Indon. J. Elect. Eng. Comp. Sci"},{"key":"B20","doi-asserted-by":"publisher","first-page":"105411","DOI":"10.1016\/j.bspc.2023.105411","article-title":"EEG-based emotion recognition for road accidents in a simulated driving environment","volume":"87","author":"Chen","year":"2024","journal-title":"Biomed. Signal Process. Control"},{"key":"B21","doi-asserted-by":"publisher","first-page":"23311","DOI":"10.1007\/s00521-021-06012-8","article-title":"Deep learning-based facial emotion recognition for human-computer interaction applications","volume":"35","author":"Chowdary","year":"2023","journal-title":"Neural Comp. Appl"},{"key":"B22","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s10462-022-10275-5","article-title":"Recent advances in decision trees: an updated survey","volume":"56","author":"Costa","year":"2023","journal-title":"Artif. Intellig. Rev"},{"key":"B23","doi-asserted-by":"publisher","first-page":"650","DOI":"10.1016\/j.ifacol.2021.04.155","article-title":"Multi-EmoNet: a novel multi-task neural network for driver emotion recognition","volume":"53","author":"Cui","year":"2020","journal-title":"IFAC-PapersOnLine"},{"key":"B24","article-title":"\u201cCoAtNet: marrying convolution and attention for all data sizes,\u201d","author":"Dai","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2024\/7184018","article-title":"Emotion modeling in speech signals: Discrete wavelet transform and machine learning tools for emotion recognition system","volume":"2024","author":"Daqrouq","year":"2024","journal-title":"Appl. Comp. Intellig. Soft Comp"},{"key":"B26","doi-asserted-by":"publisher","first-page":"55","DOI":"10.3390\/safety6040055","article-title":"On driver behavior recognition for increased safety: a roadmap","volume":"6","author":"Davoli","year":"2020","journal-title":"Safety"},{"key":"B27","doi-asserted-by":"publisher","first-page":"2636","DOI":"10.1073\/pnas.1513271113","article-title":"Driver crash risk factors and prevalence evaluation using naturalistic driving data","volume":"113","author":"Dingus","year":"2016","journal-title":"Proc. Nat. Acad. Sci"},{"key":"B28","first-page":"1","article-title":"CARLA: An open urban driving simulator,\u201d","author":"Dosovitskiy","year":"2017","journal-title":"Proceedings of the 1st Annual Conference on Robot Learning, volume 78 of Proceedings of Machine Learning Research"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10055-024-00950-z","article-title":"Driving emotions: using virtual reality to explore the effect of low and high arousal on driver's attention","volume":"28","author":"Dozio","year":"2024","journal-title":"Virtual Real"},{"key":"B30","doi-asserted-by":"publisher","first-page":"4570","DOI":"10.1109\/TITS.2020.3007357","article-title":"A convolution bidirectional long short-term memory neural network for driver emotion recognition","volume":"22","author":"Du","year":"2021","journal-title":"IEEE Trans. Intellig. Transp. Syst"},{"key":"B31","article-title":"\u201cIncorporating second-order functional knowledge for better option pricing,\u201d","author":"Dugas","year":"2000","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B32","doi-asserted-by":"publisher","first-page":"429","DOI":"10.2217\/pme-2018-0044","author":"Dunn","year":"2018","journal-title":"Wearables and the Medical Revolution, Vol. 15"},{"key":"B33","doi-asserted-by":"publisher","first-page":"105907","DOI":"10.1016\/j.bspc.2023.105907","article-title":"Ecg-based emotion recognition using random convolutional kernel method","volume":"91","author":"Fang","year":"2024","journal-title":"Biomed. Signal Process. Control"},{"key":"B34","doi-asserted-by":"crossref","DOI":"10.23919\/FUSION45008.2020.9190246","article-title":"\u201cEarly vs late fusion in multimodal convolutional neural networks,\u201d","volume-title":"Proceedings of 2020 23rd International Conference on Information Fusion, FUSION 2020","author":"Gadzicki","year":"2020"},{"key":"B35","doi-asserted-by":"publisher","first-page":"377","DOI":"10.1016\/j.aej.2022.12.009","article-title":"Real-time driver sentiment analysis using hybrid deep learning algorithm","volume":"12","author":"Gite","year":"2023","journal-title":"Int. J. Intellig. Syst. Appl. Eng"},{"key":"B36","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1080\/02699930701626582","article-title":"The Karolinska directed emotional faces: a validation study","volume":"22","author":"Goeleven","year":"2008","journal-title":"Cogn. Emot"},{"key":"B37","doi-asserted-by":"publisher","first-page":"45543","DOI":"10.1109\/ACCESS.2024.3380847","article-title":"Facial emotion recognition (FER) through custom lightweight cnn model: Performance evaluation in public datasets","volume":"12","author":"Gursesli","year":"2024","journal-title":"IEEE Access"},{"key":"B38","doi-asserted-by":"publisher","first-page":"102136","DOI":"10.1016\/j.inffus.2023.102136","article-title":"A systematic review of data fusion techniques for optimized structural health monitoring","volume":"103","author":"Hassani","year":"2024","journal-title":"Inform. Fusion"},{"key":"B39","doi-asserted-by":"publisher","first-page":"19083","DOI":"10.1109\/ACCESS.2022.3151048","article-title":"Mlcm: Multi-label confusion matrix","volume":"10","author":"Heydarian","year":"2022","journal-title":"IEEE Access"},{"key":"B40","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1007\/s10015-023-00858-y","article-title":"Negative emotion recognition using multimodal physiological signals for advanced driver assistance systems","volume":"28","author":"Hieida","year":"2023","journal-title":"Artif. Life Robot"},{"key":"B41","first-page":"1","author":"Hu","year":"2024","journal-title":"MDEmoNet: A Multimodal Driver Emotion Recognition Network for Smart Cockpit"},{"key":"B42","article-title":"\u201cElectrocardiograph based emotion recognition via wgan-gp data enhancement and improved cnn,\u201d","author":"Hu","year":"2022","journal-title":"Intelligent Robotics and Applications"},{"key":"B43","doi-asserted-by":"publisher","first-page":"167","DOI":"10.3390\/technologies11060167","article-title":"Hyperparameter optimization and combined data sampling techniques in machine learning for customer churn prediction: a comparative analysis","volume":"11","author":"Imani","year":"2023","journal-title":"Technologies"},{"key":"B44","doi-asserted-by":"publisher","first-page":"104659","DOI":"10.1016\/j.imavis.2023.104659","article-title":"An automated hyperparameter tuned deep learning model enabled facial emotion recognition for autonomous vehicle drivers","volume":"133","author":"Jain","year":"2023","journal-title":"Image Vis. Comput"},{"key":"B45","doi-asserted-by":"publisher","first-page":"9","DOI":"10.3390\/app10093211","article-title":"Hybrid-recursive feature elimination for efficient feature selection","volume":"10","author":"Jeon","year":"2020","journal-title":"Appl. Sci"},{"key":"B46","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1007\/s42979-023-02543-0","article-title":"Emotion recognition from electroencephalogram (EEG) signals using a multiple column convolutional neural network model","volume":"5","author":"Jha","year":"2024","journal-title":"SN Comp. Sci"},{"key":"B47","doi-asserted-by":"publisher","first-page":"122946","DOI":"10.1016\/j.eswa.2023.122946","article-title":"MSER: multimodal speech emotion recognition using cross-attention with deep fusion","volume":"245","author":"Khan","year":"2024","journal-title":"Expert Syst. Appl"},{"key":"B48","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1007\/s11042-021-11298-w","article-title":"An efficient deep learning technique for facial emotion recognition","volume":"81","author":"Khattak","year":"2022","journal-title":"Multimed. Tools Appl"},{"key":"B49","doi-asserted-by":"publisher","first-page":"2","DOI":"10.3390\/s24020348","article-title":"Machine learning for multimodal mental health detection: a systematic review of passive sensing approaches","volume":"24","author":"Khoo","year":"2024","journal-title":"Sensors"},{"key":"B50","doi-asserted-by":"publisher","first-page":"2","DOI":"10.3390\/s18020401","article-title":"A brief review of facial emotion recognition based on visual information","volume":"18","author":"Ko","year":"2018","journal-title":"Sensors"},{"key":"B51","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/T-AFFC.2011.15","article-title":"DEAP: A database for emotion analysis; using physiological signals","volume":"3","author":"Koelstra","year":"2012","journal-title":"IEEE Trans. Affect. Comp"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2022.108863","article-title":"Depress-DCNF: A deep convolutional neuro-fuzzy model for detection of depression episodes using IoMT","author":"Kumar","year":"2022","journal-title":"Appl. Soft Comput"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1007\/s41870-024-02027-1","article-title":"Transfer learning for emotion detection in conversational text: a hybrid deep learning approach with pre-trained embeddings","author":"Kusal","year":"2024","journal-title":"Int. J. Inform. Technol"},{"key":"B54","author":"Lee","year":"2020","journal-title":"Continuous Emotion Estimation of Facial Expressions on JAFFE and CK+ Datasets for Human Robot Interaction"},{"key":"B55","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/s18040957","article-title":"Convolutional neural network-based classification of driver's emotion during aggressive and smooth driving using multi-modal camera sensors","volume":"18","author":"Lee","year":"2018","journal-title":"Sensors"},{"key":"B56","volume-title":"EEG Based Emotion Recognition: A Tutorial and Review","author":"Li","year":"2022"},{"key":"B57","first-page":"2980","article-title":"\u201cFocal loss for dense object detection,\u201d","author":"Lin","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B58","doi-asserted-by":"publisher","first-page":"4","DOI":"10.3390\/app13042573","article-title":"Review of studies on emotion recognition and judgment based on physiological signals","volume":"13","author":"Lin","year":"2023","journal-title":"Appl. Sci"},{"key":"B59","doi-asserted-by":"publisher","first-page":"e238434","DOI":"10.1371\/journal.pone.0238434","article-title":"Autobiographical emotional induction in older people through popular songs: Effect of reminiscence bump and enculturation","volume":"15","author":"L\u00f3pez-Cano","year":"2020","journal-title":"PLoS ONE"},{"key":"B60","doi-asserted-by":"publisher","first-page":"107122","DOI":"10.1016\/j.buildenv.2020.107122","article-title":"Research on work efficiency and light comfort based on eeg evaluation method","volume":"183","author":"Lu","year":"2020","journal-title":"Build. Environ"},{"key":"B61","article-title":"\u201cThe extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression,\u201d","author":"Lucey","year":"2010","journal-title":"2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition"},{"key":"B62","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1016\/j.trf.2020.06.020","article-title":"From riskier to safer driving decisions: the role of feedback and the experiential automatic processing system","volume":"73","author":"Maldonado","year":"2020","journal-title":"Transp. Res. Part F: Traffic Psychol. Behav"},{"key":"B63","doi-asserted-by":"publisher","first-page":"106691","DOI":"10.1016\/j.asoc.2020.106691","article-title":"Automated medical diagnosis of COVID-19 through EfficientNet convolutional neural network","volume":"96","author":"Marques","year":"2020","journal-title":"Appl. Soft Comp. J"},{"key":"B64","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s42452-020-2234-1","article-title":"Facial emotion recognition using convolutional neural networks (FERC)","volume":"2","author":"Mehendale","year":"2020","journal-title":"SN Appl. Sci"},{"key":"B65","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1109\/ICDT61202.2024.10489678","article-title":"\u201cFacial emotion recognition and detection using convolutional neural networks with low computation cost,\u201d","volume-title":"2024 2nd International Conference on Disruptive Technologies (ICDT)","author":"Mehrotra","year":"2024"},{"key":"B66","doi-asserted-by":"publisher","first-page":"4","DOI":"10.48550\/arXiv.2002.10061","article-title":"Dimensional models for continuous-to-discrete affect mapping in speech emotion recognition","volume":"83","author":"Mihalache","year":"2021","journal-title":"UPB Sci. Bull. Series C: Elect. Eng. Comp. Sci"},{"key":"B67","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1007\/s11760-023-02716-7","article-title":"Speech emotion recognition using MFCC-based entropy feature","volume":"18","author":"Mishra","year":"2024","journal-title":"Signal, Image Video Proc"},{"key":"B68","doi-asserted-by":"publisher","first-page":"104676","DOI":"10.1016\/j.imavis.2023.104676","article-title":"Multimodal emotion recognition using cross modal audio-video fusion with attention and deep metric learning","volume":"133","author":"Mocanu","year":"2023","journal-title":"Image Vis. Comput"},{"key":"B69","first-page":"1339","article-title":"\u201cFacial emotion recognition using convolution neural network,\u201d","author":"Modi","year":"2021","journal-title":"2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS)"},{"key":"B70","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/TAFFC.2017.2740923","article-title":"Affectnet: a database for facial expression, valence, and arousal computing in the wild","volume":"10","author":"Mollahosseini","year":"2017","journal-title":"IEEE Trans. Affect. Comp"},{"key":"B71","doi-asserted-by":"publisher","first-page":"2970","DOI":"10.1109\/TAFFC.2023.3250460","article-title":"Driver emotion recognition with a hybrid attentional multimodal fusion framework","volume":"14","author":"Mou","year":"2023","journal-title":"IEEE Trans. Affect. Comp"},{"key":"B72","doi-asserted-by":"publisher","first-page":"79861","DOI":"10.1109\/ACCESS.2020.2990405","article-title":"Clustering-based speech emotion recognition by incorporating learned features and deep bilstm","volume":"8","author":"Mustaqeem","year":"2020","journal-title":"IEEE Access"},{"key":"B73","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/rs12030587","article-title":"Deep learning-based drivers emotion classification system in time series data for remote applications","volume":"12","author":"Naqvi","year":"2020","journal-title":"Remote Sens"},{"key":"B74","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/s21062166","article-title":"Drer: Deep learning\u2013based driver's real emotion recognizer","volume":"21","author":"Oh","year":"2021","journal-title":"Sensors"},{"key":"B75","doi-asserted-by":"publisher","first-page":"13","DOI":"10.3390\/diagnostics13132145","article-title":"Diagnostic accuracy of machine learning ai architectures in detection and classification of lung cancer: A systematic review","volume":"13","author":"Pacurari","year":"2023","journal-title":"Diagnostics"},{"key":"B76","article-title":"\u201cFast & furious: detecting stress with a car steering wheel,\u201d","author":"Paredes","year":"2018","journal-title":"Conference on Human Factors in Computing Systems"},{"key":"B77","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ICIICT1.2019.8741491","article-title":"\u201cHuman emotion recognition using convolutional neural network in real time,\u201d","volume-title":"2019 1st International Conference on Innovations in Information and Communication Technology (ICIICT)","author":"Pathar","year":"2019"},{"key":"B78","first-page":"0420","article-title":"\u201cDriver emotion recognition for enhancement of human machine interface in vehicles,\u201d","volume-title":"Proceedings of the 2019 IEEE International Conference on Communication and Signal Processing, ICCSP 2019","author":"Patil","year":"2019"},{"key":"B79","first-page":"15","article-title":"\u201cDo vision transformers see like convolutional neural networks?,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Raghu","year":"2021"},{"key":"B80","doi-asserted-by":"publisher","first-page":"5458","DOI":"10.4108\/eetpht.10.5458","article-title":"Cnn based face emotion recognition system for healthcare application","volume":"10","author":"Ravikumar","year":"2024","journal-title":"EAI Endors. Trans. Pervas. Health Technol"},{"key":"B81","doi-asserted-by":"publisher","first-page":"17","DOI":"10.3390\/s23177559","article-title":"Detection of pedestrians in reverse camera using multimodal convolutional neural networks","volume":"23","author":"Reveles-G\u00f3mez","year":"2023","journal-title":"Sensors"},{"key":"B82","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1057\/s41599-022-01041-7","article-title":"Identification of factors associated with various types of impaired driving","volume":"9","author":"Rezapour","year":"2022","journal-title":"Humanit. Soc. Sci. Commun"},{"key":"B83","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1007\/s00034-023-02320-7","article-title":"Performance comparison of facial emotion recognition: a transfer learning-based driver assistance framework for in-vehicle applications","volume":"42","author":"Sahoo","year":"2023","journal-title":"Circuits, Systems, and Signal Proc"},{"key":"B84","first-page":"48","article-title":"\u201cEffects of anger and display urgency on takeover performance in semi-automated vehicles,\u201d","volume-title":"12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI '20","author":"Sanghavi","year":"2020"},{"key":"B85","doi-asserted-by":"publisher","first-page":"3560","DOI":"10.1016\/j.matpr.2021.07.297","article-title":"Facial emotion recognition using convolutional neural networks","volume":"80","author":"Sarvakar","year":"2023","journal-title":"Mater. Today: Proc"},{"key":"B86","doi-asserted-by":"publisher","first-page":"2","DOI":"10.17705\/1jais.00608","article-title":"Research perspectives: the rise of human machines: how cognitive computing systems challenge assumptions of user-system interaction","volume":"21","author":"Schuetz","year":"2020","journal-title":"J. Assoc. Inform. Syst"},{"key":"B87","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1080\/15389588.2017.1315109","article-title":"Difficulties in emotion regulation and risky driving among Lithuanian drivers","volume":"18","author":"\u0160eibokait","year":"2017","journal-title":"Traffic Inj. Prev"},{"key":"B88","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1371\/journal.pone.0256503","article-title":"PyPlutchik: Visualising and comparing emotion-annotated corpora","volume":"16","author":"Semeraro","year":"2021","journal-title":"PLoS ONE"},{"key":"B89","doi-asserted-by":"publisher","first-page":"386","DOI":"10.1007\/978-3-030-21074-8_32","article-title":"Integration of driver behavior into emotion recognition systems: a preliminary study on steering wheel and vehicle acceleration","volume":"11367","author":"Shafaei","year":"2019","journal-title":"Lect. Notes Comp. Sci"},{"key":"B90","doi-asserted-by":"publisher","first-page":"9","DOI":"10.3390\/app13095572","article-title":"Hybrid facial emotion recognition using cnn-based features","volume":"13","author":"Shahzad","year":"2023","journal-title":"Appl. Sci"},{"key":"B91","doi-asserted-by":"publisher","first-page":"12891","DOI":"10.1007\/s00521-023-08428-w","article-title":"Automatic stress detection in car drivers based on non-invasive physiological signals using machine learning techniques","volume":"35","author":"Siam","year":"2023","journal-title":"Neural Comp. Appl"},{"key":"B92","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.trf.2018.08.012","article-title":"Effects of emotions on driving behavior","volume":"59","author":"Steinhauser","year":"2018","journal-title":"Transp. Res. Part F: Traffic Psychol. Behav"},{"key":"B93","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1007\/s12144-023-04744-5","article-title":"Don't sweat the small stuff; anger rumination and lack of forgiveness are related to aggressive driving behaviours","volume":"43","author":"Stephens","year":"2024","journal-title":"Curr. Psychol"},{"key":"B94","doi-asserted-by":"publisher","first-page":"5","DOI":"10.3390\/ijerph19053085","article-title":"A hybrid model for driver emotion detection using feature fusion approach","volume":"19","author":"Sukhavasi","year":"2022","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"B95","doi-asserted-by":"publisher","first-page":"29039","DOI":"10.1109\/JSEN.2024.3434479","article-title":"Ecg-based automated emotion recognition using temporal convolution neural networks","volume":"24","author":"Sweeney-Fanelli","year":"2024","journal-title":"IEEE Sens. J"},{"key":"B96","doi-asserted-by":"publisher","first-page":"6695","DOI":"10.1007\/s00500-023-09477-y","article-title":"Real-time facial emotion recognition model based on kernel autoencoder and convolutional neural network for autism children","volume":"28","author":"Talaat","year":"2024","journal-title":"Soft Computing"},{"key":"B97","article-title":"Rethinking 1DCNN for time series classification: A stronger Baseline","author":"Tang","year":"2020","journal-title":"arXiv"},{"key":"B98","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1016\/j.comcom.2022.07.031","article-title":"Drivers emotion and behavior classification system based on internet of things and deep learning for advanced driver assistance system (adas)","volume":"194","author":"Tauqeer","year":"2022","journal-title":"Comput. Commun"},{"key":"B99","first-page":"37","article-title":"\u201cChapter 2 - artificial intelligence-based emotion recognition using ecg signals,\u201d","author":"Tokmak","year":"2024","journal-title":"Applications of Artificial Intelligence in Healthcare and Biomedicine, Artificial Intelligence Applications in Healthcare and Medicine"},{"key":"B100","doi-asserted-by":"publisher","first-page":"13577","DOI":"10.1111\/exsy.13577","article-title":"Effects of feature reduction on emotion recognition using eeg signals and machine learning","volume":"41","author":"Trujillo","year":"2024","journal-title":"Expert Syst"},{"key":"B101","doi-asserted-by":"publisher","first-page":"124434","DOI":"10.1109\/ACCESS.2021.3110773","article-title":"A systematic review of sensing and differentiating dichotomous emotional states using audio-visual stimuli","volume":"9","author":"Veeranki","year":"2021","journal-title":"IEEE Access"},{"key":"B102","first-page":"1421","article-title":"\u201cA framework for driver emotion recognition using deep learning and grassmann manifolds,\u201d","volume-title":"IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC","author":"Verma","year":""},{"key":"B103","first-page":"1","article-title":"\u201cDeep learning based real-time driver emotion monitoring,\u201d","volume-title":"2018 IEEE International Conference on Vehicular Electronics and Safety, ICVES 2018","author":"Verma","year":""},{"key":"B104","doi-asserted-by":"publisher","first-page":"141890","DOI":"10.1109\/ACCESS.2024.3463746","article-title":"Advancing emotional health assessments: a hybrid deep learning approach using physiological signals for robust emotion recognition","volume":"12","author":"Waheed Awan","year":"2024","journal-title":"IEEE Access"},{"key":"B105","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1049\/iet-its.2019.0499","article-title":"Driver emotion recognition of multiple-ECG feature fusion based on BP network and D-S evidence","volume":"14","author":"Wang","year":"2020","journal-title":"IET Intellig. Transp. Syst"},{"key":"B106","doi-asserted-by":"publisher","first-page":"791","DOI":"10.23940\/ijpe.22.11.p4.791797","article-title":"Real-time prediction of car driver's emotions using facial expression with a convolutional neural network-based intelligent system","volume":"18","author":"Wawage","year":"2022","journal-title":"Int. J. Perf. Eng"},{"key":"B107","volume-title":"WHO: Road Traffic Injuries","year":"2018"},{"key":"B108","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1109\/TKDE.2019.2912815","article-title":"Reliable accuracy estimates from k-fold cross validation","volume":"32","author":"Wong","year":"2020","journal-title":"IEEE Trans. Knowl. Data Eng"},{"key":"B109","doi-asserted-by":"publisher","first-page":"739","DOI":"10.18494\/SAM.2021.3040","article-title":"Evaluation of effect of music on human nervous system by heart rate variability analysis using ecg sensor","volume":"33","author":"Wu","year":"2021","journal-title":"Sensors Mater"},{"key":"B110","doi-asserted-by":"publisher","first-page":"2","DOI":"10.3390\/app12020807","article-title":"On-road driver emotion recognition using facial expression","volume":"12","author":"Xiao","year":"2022","journal-title":"Appl. Sci"},{"key":"B111","first-page":"36","article-title":"\u201cEarly convolutions help transformers see better,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Xiao","year":"2021"},{"key":"B112","doi-asserted-by":"publisher","first-page":"1082","DOI":"10.1109\/TAFFC.2021.3100868","article-title":"Behavioral and physiological signals-based deep multimodal approach for mobile emotion recognition","volume":"14","author":"Yang","year":"2023","journal-title":"IEEE Trans. Affect. Comp"},{"key":"B113","doi-asserted-by":"publisher","first-page":"128177","DOI":"10.1016\/j.neucom.2024.128177","article-title":"Speech emotion recognition based on multi-feature speed rate and lstm","volume":"601","author":"Yang","year":"2024","journal-title":"Neurocomputing"},{"key":"B114","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/j.specom.2020.03.005","article-title":"Speech emotion recognition using fusion of three multi-task learning-based classifiers: HSF-DNN, MS-CNN and LLD-RNN","volume":"120","author":"Yao","year":"2020","journal-title":"Speech Commun"},{"key":"B115","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/JIOT.2024.3363176","article-title":"A multimodal driver emotion recognition algorithm based on the audio and video signals in internet of vehicles platform","volume":"33","author":"Ying","year":"2024","journal-title":"IEEE Intern. Things J"},{"key":"B116","first-page":"1","author":"Zahara","year":"2020","journal-title":"The Facial Emotion Recognition (FER-2013) Dataset for Prediction System of Micro-Expressions Face Using the Convolutional Neural Network (CNN) Algorithm Based Raspberry PI"},{"key":"B117","doi-asserted-by":"publisher","first-page":"4","DOI":"10.3390\/sym14040687","article-title":"Driver emotions recognition based on improved faster R-CNN and neural architectural search network","volume":"14","author":"Zaman","year":"2022","journal-title":"Symmet"},{"key":"B118","doi-asserted-by":"publisher","first-page":"6927","DOI":"10.1007\/s40747-023-01100-9","article-title":"A novel driver emotion recognition system based on deep ensemble classification","volume":"9","author":"Zaman","year":"2023","journal-title":"Complex Intellig. Syst"},{"key":"B119","doi-asserted-by":"publisher","first-page":"101918","DOI":"10.1016\/j.bspc.2020.101918","article-title":"Emotion recognition using eeg phase space dynamics and poincare intersections","volume":"59","author":"Zangeneh Soroush","year":"2020","journal-title":"Biomed. Signal Process. Control"},{"key":"B120","author":"Zepf","year":"2020","journal-title":"Driver Emotion Recognition for Intelligent Vehicles: A Survey"},{"key":"B121","doi-asserted-by":"publisher","DOI":"10.1007\/s12652-020-02049-0","article-title":"EEG-based emotion recognition using an improved radial basis function neural network","author":"Zhang","year":"2020","journal-title":"J. Ambient Intellig. Humaniz. Comp"},{"key":"B122","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1016\/j.trf.2020.02.004","article-title":"The effect of the emotional state on driving performance in a simulated car-following task","volume":"69","author":"Zhang","year":"2020","journal-title":"Transp. Res. Part F: Traffic Psychol. Behav"},{"key":"B123","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1007\/s11042-020-09259-w","article-title":"Driver drowsiness recognition via transferred deep 3D convolutional network and state probability vector","volume":"79","author":"Zhao","year":"2020","journal-title":"Multimed. Tools Appl"},{"key":"B124","doi-asserted-by":"publisher","first-page":"19","DOI":"10.3390\/s23198293","article-title":"Drivers comprehensive emotion recognition based on ham","volume":"23","author":"Zhou","year":"2023","journal-title":"Sensors"},{"key":"B125","doi-asserted-by":"publisher","first-page":"108422","DOI":"10.1016\/j.patcog.2021.108422","article-title":"Neighborhood linear discriminant analysis","volume":"123","author":"Zhu","year":"2022","journal-title":"Pattern Recognit"},{"key":"B126","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.trf.2016.12.005","article-title":"Are happy drivers safer drivers? Evidence from hazard response times and eye tracking data","volume":"46","author":"Zimasa","year":"2017","journal-title":"Transp. Res. Part F: Traffic Psychol. Behav"},{"key":"B127","first-page":"8697","article-title":"\u201cLearning transferable architectures for scalable image recognition,\u201d","author":"Zoph","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1467051\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,18]],"date-time":"2024-12-18T13:12:58Z","timestamp":1734527578000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1467051\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,27]]},"references-count":127,"alternative-id":["10.3389\/frai.2024.1467051"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1467051","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,27]]},"article-number":"1467051"}}