{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T15:22:35Z","timestamp":1784301755207,"version":"3.55.0"},"reference-count":31,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2022,10,21]],"date-time":"2022-10-21T00:00:00Z","timestamp":1666310400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Princess Nourah bint Abdulrahman University","award":["PNURS2022R178"],"award-info":[{"award-number":["PNURS2022R178"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Human ideas and sentiments are mirrored in facial expressions. They give the spectator a plethora of social cues, such as the viewer\u2019s focus of attention, intention, motivation, and mood, which can help develop better interactive solutions in online platforms. This could be helpful for children while teaching them, which could help in cultivating a better interactive connect between teachers and students, since there is an increasing trend toward the online education platform due to the COVID-19 pandemic. To solve this, the authors proposed kids\u2019 emotion recognition based on visual cues in this research with a justified reasoning model of explainable AI. The authors used two datasets to work on this problem; the first is the LIRIS Children Spontaneous Facial Expression Video Database, and the second is an author-created novel dataset of emotions displayed by children aged 7 to 10. The authors identified that the LIRIS dataset has achieved only 75% accuracy, and no study has worked further on this dataset in which the authors have achieved the highest accuracy of 89.31% and, in the authors\u2019 dataset, an accuracy of 90.98%. The authors also realized that the face construction of children and adults is different, and the way children show emotions is very different and does not always follow the same way of facial expression for a specific emotion as compared with adults. Hence, the authors used 3D 468 landmark points and created two separate versions of the dataset from the original selected datasets, which are LIRIS-Mesh and Authors-Mesh. In total, all four types of datasets were used, namely LIRIS, the authors\u2019 dataset, LIRIS-Mesh, and Authors-Mesh, and a comparative analysis was performed by using seven different CNN models. The authors not only compared all dataset types used on different CNN models but also explained for every type of CNN used on every specific dataset type how test images are perceived by the deep-learning models by using explainable artificial intelligence (XAI), which helps in localizing features contributing to particular emotions. The authors used three methods of XAI, namely Grad-CAM, Grad-CAM++, and SoftGrad, which help users further establish the appropriate reason for emotion detection by knowing the contribution of its features in it.<\/jats:p>","DOI":"10.3390\/s22208066","type":"journal-article","created":{"date-parts":[[2022,10,24]],"date-time":"2022-10-24T10:09:23Z","timestamp":1666606163000},"page":"8066","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":40,"title":["Kids\u2019 Emotion Recognition Using Various Deep-Learning Models with Explainable AI"],"prefix":"10.3390","volume":"22","author":[{"given":"Manish","family":"Rathod","sequence":"first","affiliation":[{"name":"Symbiosis Centre for Applied Artificial Intelligence (SCAAI), Symbiosis International University (Deemed University), Pune 412115, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chirag","family":"Dalvi","sequence":"additional","affiliation":[{"name":"Symbiosis Centre for Applied Artificial Intelligence (SCAAI), Symbiosis International University (Deemed University), Pune 412115, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kulveen","family":"Kaur","sequence":"additional","affiliation":[{"name":"Symbiosis Centre for Applied Artificial Intelligence (SCAAI), Symbiosis International University (Deemed University), Pune 412115, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4903-1540","authenticated-orcid":false,"given":"Shruti","family":"Patil","sequence":"additional","affiliation":[{"name":"Computer Science and Information Technology Department, Symbiosis Institute of Technology, Symbiosis International University (Deemed University), Pune 412115, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shilpa","family":"Gite","sequence":"additional","affiliation":[{"name":"Computer Science and Information Technology Department, Symbiosis Institute of Technology, Symbiosis International University (Deemed University), Pune 412115, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7597-0197","authenticated-orcid":false,"given":"Pooja","family":"Kamat","sequence":"additional","affiliation":[{"name":"Symbiosis Centre for Applied Artificial Intelligence (SCAAI), Symbiosis International University (Deemed University), Pune 412115, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2653-3780","authenticated-orcid":false,"given":"Ketan","family":"Kotecha","sequence":"additional","affiliation":[{"name":"Computer Science and Information Technology Department, Symbiosis Institute of Technology, Symbiosis International University (Deemed University), Pune 412115, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0169-6738","authenticated-orcid":false,"given":"Ajith","family":"Abraham","sequence":"additional","affiliation":[{"name":"Machine Intelligence Research Labs (MIR Labs), Auburn, WA 98071, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6910-6861","authenticated-orcid":false,"given":"Lubna Abdelkareim","family":"Gabralla","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Technology, College of Applied, Princess Nourah bint Abdulrahman University, Riyadh 11671, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"165806","DOI":"10.1109\/ACCESS.2021.3131733","article-title":"A Survey of AI-Based Facial Emotion Recognition: Features, ML DL Techniques, Age-Wise Datasets and Future Directions","volume":"9","author":"Dalvi","year":"2021","journal-title":"IEEE Access"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mahendar, M., Malik, A., and Batra, I. (2021). A Comparative Study of Deep Learning Techniques for Emotion Estimation Based on E-Learning Through Cognitive State Analysis, Springer.","DOI":"10.1007\/978-981-16-3660-8_21"},{"key":"ref_3","unstructured":"Khosravi, H., Shum, S.B., Chen, G., Conati, C., Tsai, Y.S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S. (2022, August 19). Student Engagement Detection Using Emotion Analysis, Eye Tracking and Head Movement with Machine Learning. Available online: https:\/\/ui.adsabs.harvard.edu\/abs\/2019arXiv190912913S."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100074","DOI":"10.1016\/j.caeai.2022.100074","article-title":"Explainable Artificial Intelligence in education","volume":"3","author":"Khosravi","year":"2022","journal-title":"Comput. Educ. Artif. Intell."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1016\/j.procs.2015.12.030","article-title":"Facial Expression Recognition for Intelligent Tutoring Systems in Remote Laboratories Platform","volume":"73","author":"Khalfallah","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1016\/j.procs.2016.05.264","article-title":"Student Emotion Recognition System (SERS) for e-learning Improvement Based on Learner Concentration Metric","volume":"85","author":"Gg","year":"2016","journal-title":"Procedia Comput. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1367","DOI":"10.1007\/s10639-015-9388-2","article-title":"Towards real-time speech emotion recognition for affective e-learning","volume":"21","author":"Bahreini","year":"2016","journal-title":"Educ. Inf. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1080\/10447318.2016.1159799","article-title":"Data Fusion for Real-time Multimodal Emotion Recognition through Webcams and Microphones in E-Learning","volume":"32","author":"Bahreini","year":"2016","journal-title":"Int. J. Hum. -Comput. Interact."},{"key":"ref_9","first-page":"446","article-title":"Using Facial Expression to Detect Emotion in E-learning System: A Deep Learning Method","volume":"Volume 10676","author":"Sun","year":"2017","journal-title":"Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.procs.2017.12.003","article-title":"An Emotion Recognition Model Based on Facial Recognition in Virtual Learning Environment","volume":"125","author":"Yang","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_11","first-page":"479","article-title":"Automatic speech based emotion recognition using paralinguistics features","volume":"67","author":"Hook","year":"2019","journal-title":"Bull. Pol. Acad. Sci. Tech. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3960","DOI":"10.3758\/s13428-016-0756-7","article-title":"The Creation and Validation of the Developmental Emotional Faces Stimulus Set","volume":"49","author":"Meuwissen","year":"2017","journal-title":"Behav. Res. Methods"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1002\/mpr.343","article-title":"The NIMH Child Emotional Faces Picture Set (NIMH-ChEFS): A new set of children\u2019s facial emotion stimuli","volume":"20","author":"Egger","year":"2011","journal-title":"Int. J. Methods Psychiatr. Res."},{"key":"ref_14","first-page":"1532","article-title":"The Child Affective Facial Expression (CAFE) set: Validity and reliability from untrained adults","volume":"5","author":"LoBue","year":"2014","journal-title":"Front. Psychol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Nojavanasghari, B., Baltru\u0161aitis, T., Hughes, C.E., and Morency, L.-P. (2016, January 12\u201316). Emo react: A multimodal approach and dataset for recognizing emotional responses in children. Proceedings of the ICMI 2016\u201418th ACM International Conference on Multimodal Interaction, Tokyo, Japan.","DOI":"10.1145\/2993148.2993168"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.imavis.2019.02.004","article-title":"A novel database of children\u2019s spontaneous facial expressions (LIRIS-CSE)","volume":"83\u201384","author":"Khan","year":"2019","journal-title":"Image Vis. Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1235","DOI":"10.1037\/a0028297","article-title":"Age-Related Differences in Emotional Reactivity, Regulation, and Rejection Sensitivity in Adolescence","volume":"12","author":"Silvers","year":"2012","journal-title":"Emotion"},{"key":"ref_18","unstructured":"Redmon, J., and Farhadi, A. (2022, October 01). YOLOv3: An Incremental Improvement. Available online: https:\/\/doi.org\/10.48550\/arxiv.1804.02767."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"8032673","DOI":"10.1155\/2022\/8032673","article-title":"Deploying Machine Learning Techniques for Human Emotion Detection","volume":"2022","author":"Siam","year":"2022","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ouanan, H., Ouanan, M., and Aksasse, B. (2016, January 24\u201326). Facial landmark localization: Past, present and future. Proceedings of the 2016 4th IEEE International Colloquium on Information Science and Technology (CiSt), Tangier, Morocco.","DOI":"10.1109\/CIST.2016.7805097"},{"key":"ref_21","unstructured":"Tan, M., and Le, Q.V. (2020). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. arXiv."},{"key":"ref_22","unstructured":"Simonyan, K., and Zisserman, A. (2022, August 19). Very Deep Convolutional Networks for Large-Scale Image Recognition. September 2014. Available online: https:\/\/ui.adsabs.harvard.edu\/abs\/2014arXiv1409.1556S."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Weinberger, Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"115","DOI":"10.30595\/juita.v9i1.9624","article-title":"A Deep Learning Using DenseNet201 to Detect Masked or Non-masked Face","volume":"9","author":"Adhinata","year":"2021","journal-title":"JUITA J. Inform."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the Inception Architecture for Computer Vision. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2016, January 4\u20139). Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. Proceedings of the 31st AAAI Conference on Artificial Intelligence, AAAI, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","article-title":"Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization","volume":"128","author":"Selvaraju","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chattopadhay, A., Sarkar, A., Howlader, P., and Balasubramanian, V.N. (2017, January 12\u201315). Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018, Lake Tahoe, NV, USA.","DOI":"10.1109\/WACV.2018.00097"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., and Hu, X. (2019, January 19\u201320). Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, New Orleans, LA, USA.","DOI":"10.1109\/CVPRW50498.2020.00020"},{"key":"ref_31","unstructured":"Omeiza, D., Speakman, S., Cintas, C., and Weldermariam, K. (2019). Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/20\/8066\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:59:02Z","timestamp":1760144342000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/20\/8066"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,21]]},"references-count":31,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["s22208066"],"URL":"https:\/\/doi.org\/10.3390\/s22208066","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,21]]}}}