{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T15:21:16Z","timestamp":1776784876447,"version":"3.51.2"},"reference-count":29,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2023,4,24]],"date-time":"2023-04-24T00:00:00Z","timestamp":1682294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Research Center for Computing and Multimedia Studies, Hosei University"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Students\u2019 affective states describe their engagement, concentration, attitude, motivation, happiness, sadness, frustration, off-task behavior, and confusion level in learning. In online learning, students\u2019 affective states are determinative of the learning quality. However, measuring various affective states and what influences them is exceedingly challenging for the lecturer without having real interaction with the students. Existing studies primarily use self-reported data to understand students\u2019 affective states, while this paper presents a novel learning analytics system called MOEMO (Motion and Emotion) that could measure online learners\u2019 affective states of engagement and concentration using emotion data. Therefore, the novelty of this research is to visualize online learners\u2019 affective states on lecturers\u2019 screens in real-time using an automated emotion detection process. In real-time and offline, the system extracts emotion data by analyzing facial features from the lecture videos captured by the typical built-in web camera of a laptop computer. The system determines online learners\u2019 five types of engagement (\u201cstrong engagement\u201d, \u201chigh engagement\u201d, \u201cmedium engagement\u201d, \u201clow engagement\u201d, and \u201cdisengagement\u201d) and two types of concentration levels (\u201cfocused\u201d and \u201cdistracted\u201d). Furthermore, the dashboard is designed to provide insight into students\u2019 emotional states, the clusters of engaged and disengaged students\u2019, assistance with intervention, create an after-class summary report, and configure the automation parameters to adapt to the study environment.<\/jats:p>","DOI":"10.3390\/s23094243","type":"journal-article","created":{"date-parts":[[2023,4,25]],"date-time":"2023-04-25T01:37:01Z","timestamp":1682386621000},"page":"4243","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["A Real-Time Learning Analytics Dashboard for Automatic Detection of Online Learners\u2019 Affective States"],"prefix":"10.3390","volume":"23","author":[{"given":"Mohammad Nehal","family":"Hasnine","sequence":"first","affiliation":[{"name":"Research Center for Computing and Multimedia Studies, Hosei University, Tokyo 102-8160, Japan"}]},{"given":"Ho Tan","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Research Center for Computing and Multimedia Studies, Hosei University, Tokyo 102-8160, Japan"}]},{"given":"Thuy Thi Thu","family":"Tran","sequence":"additional","affiliation":[{"name":"Research Center for Computing and Multimedia Studies, Hosei University, Tokyo 102-8160, Japan"}]},{"given":"Huyen T. T.","family":"Bui","sequence":"additional","affiliation":[{"name":"Research Center for Computing and Multimedia Studies, Hosei University, Tokyo 102-8160, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0742-1612","authenticated-orcid":false,"given":"G\u00f6khan","family":"Ak\u00e7ap\u0131nar","sequence":"additional","affiliation":[{"name":"Department of Computer Education and Instructional Technology, Hacettepe University, 06230 Ankara, T\u00fcrkiye"}]},{"given":"Hiroshi","family":"Ueda","sequence":"additional","affiliation":[{"name":"Research Center for Computing and Multimedia Studies, Hosei University, Tokyo 102-8160, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,24]]},"reference":[{"key":"ref_1","first-page":"356","article-title":"Measuring students affective states through online learning logs\u2014an application of learning analytics","volume":"9","author":"Wang","year":"2019","journal-title":"Int. J. Inf. Educ. Technol."},{"key":"ref_2","unstructured":"Guragain, N. (2022, April 19). E-learning benefits and applications. Available online: https:\/\/urn.fi\/URN:NBN:fi:amk-201602122192."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"104192","DOI":"10.1016\/j.compedu.2021.104192","article-title":"Expectations for supporting student engagement with learning analytics: An academic path perspective","volume":"168","author":"Silvola","year":"2021","journal-title":"Comput. Educ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"i-231","DOI":"10.2307\/1166220","article-title":"Individual Differences and the Development of Perceived Control","volume":"63","author":"Skinner","year":"1998","journal-title":"Monogr. Soc. Res. Child Dev."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kop, R., Fournier, H., and Durand, G. (2017). A Critical Perspective on Learning Analytics and Educational Data Mining, Society for Learning Analytics Research Publishing.","DOI":"10.18608\/hla17.027"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1111\/j.1751-228X.2007.00004.x","article-title":"We Feel, Therefore We Learn: The Relevance of Affective and Social Neuroscience to Education","volume":"1","author":"Damasio","year":"2007","journal-title":"Mind Brain Educ."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3423","DOI":"10.1016\/j.procs.2021.09.115","article-title":"Students\u2019 emotion extraction and visualization for engagement detection in online learning","volume":"192","author":"Hasnine","year":"2021","journal-title":"Procedia Comput. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1111\/j.1464-0597.1992.tb00712.x","article-title":"The Impact of Emotions on Learning and Achievement: Towards a Theory of Cognitive\/Motivational Mediators","volume":"41","author":"Pekrun","year":"1992","journal-title":"Appl. Psychol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1037\/mot0000014","article-title":"Achievement goals, emotions, learning, and performance: A process model","volume":"1","author":"Ranelluci","year":"2015","journal-title":"Motiv. Sci."},{"key":"ref_10","first-page":"99","article-title":"A new emotion\u2013based affective model to detect student\u2019s engagement","volume":"33","author":"Altuwairqi","year":"2018","journal-title":"J. King Saud Univ. Comput. Inf. Sci."},{"key":"ref_11","first-page":"114","article-title":"Social emotional development: A new model of student learning in higher education","volume":"10","author":"Seal","year":"2011","journal-title":"Res. High. Educ. J."},{"key":"ref_12","unstructured":"SOLAR (2022, April 19). What Is Learning Analytics. Available online: https:\/\/www.solaresearch.org\/about\/what-is-learning-analytics\/."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1454","DOI":"10.3389\/fpsyg.2017.01454","article-title":"The Influences of Emotion on Learning and Memory","volume":"8","author":"Tyng","year":"2017","journal-title":"Front. Psychol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"101213","DOI":"10.1016\/j.learninstruc.2019.05.010","article-title":"Searching for the role of emotions in e-learning","volume":"70","author":"Mayer","year":"2019","journal-title":"Learn. Instr."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1080\/1358165042000283101","article-title":"Affect and learning: An exploratory look into the role of affect in learning with AutoTutor","volume":"29","author":"Craig","year":"2004","journal-title":"J. Educ. Media"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1109\/TAFFC.2014.2316163","article-title":"The faces of engagement: Automatic recognition of student engagementfrom facial expressions","volume":"5","author":"Whitehill","year":"2014","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_17","unstructured":"Sharma, P., Joshi, S., Gautam, S., Filipe, V., and Reis, M.J. (2019). Student Engagement Detection Using Emotion Analysis, Eye Tracking and Head Movement with Machine Learning. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"101212","DOI":"10.1016\/j.learninstruc.2019.05.009","article-title":"Emotions are the experiential glue of learning environments in the 21st century","volume":"70","author":"Graesser","year":"2019","journal-title":"Learn. Instr."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.cedpsych.2010.10.002","article-title":"Measuring emotions in students\u2019 learning and performance: The Achievement Emotions Questionnaire (AEQ)","volume":"36","author":"Pekrun","year":"2011","journal-title":"Contemp. Educ. Psychol."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Fontaine, J.R.J., Scherer, K.R., and Soriano, C. (2013). Components of Emotional Meaning: A Sourcebook, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780199592746.001.0001"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1016\/j.procs.2016.05.264","article-title":"Student Emotion Recognition System (SERS) for e-learning Improvement Based on Learner Concentration Metric","volume":"85","author":"Krithika","year":"2016","journal-title":"Procedia Comput. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint face detection and alignment using multitask cascaded convolutional networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_23","unstructured":"Jain, V., and Learned-Miller, E. (2010). FDDB: A Benchmark for Face Detection in Unconstrained Setting, University of Massachusetts."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yang, S., Luo, P., Loy, C.-C., and Tang, X. (2016, January 27\u201330). Wider face: A face detection benchmark. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.596"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Koestinger, M., Wohlhart, P., Roth, P.M., and Bischof, H. (2011, January 6\u201313). Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV workshops), Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130513"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"012027","DOI":"10.1088\/1757-899X\/1042\/1\/012027","article-title":"Real Time Emotion Detection of Humans Using Mini-Xception Algorithm","volume":"1042","author":"Fatima","year":"2021","journal-title":"IOP Conf. Series Mater. Sci. Eng."},{"key":"ref_27","unstructured":"Panagiotis, G., Perikos, I., and Hatzilygeroudis, I. (2018). Advances in Hybridization of Intelligent Methods, Springer."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Revanth, A., and Prathibamol, C.P. (2022, January 20\u201321). Graph Emotion Distribution Learning Using EmotionGCN. Proceedings of the Future Technologies Conference (FTC) 2022, Vancouver, BC, Canada.","DOI":"10.1007\/978-3-031-18461-1_14"},{"key":"ref_29","unstructured":"Damasio, A.R. (1999). The Feeling of What Happens: Body and Emotion in the Making of Consciousness, Harcourt Brace Publishing."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4243\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:22:36Z","timestamp":1760124156000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4243"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,24]]},"references-count":29,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["s23094243"],"URL":"https:\/\/doi.org\/10.3390\/s23094243","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,24]]}}}