{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T18:49:34Z","timestamp":1776106174229,"version":"3.50.1"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"12","license":[{"start":{"date-parts":[[2023,5,1]],"date-time":"2023-05-01T00:00:00Z","timestamp":1682899200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,1]],"date-time":"2023-05-01T00:00:00Z","timestamp":1682899200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["UIDB\/04152\/2020 - Centro de Investiga\u00e7\u00e3o em Gest\u00e3o de Informa\u00e7\u00e3o (MagIC)\/NOVA IMS"],"award-info":[{"award-number":["UIDB\/04152\/2020 - Centro de Investiga\u00e7\u00e3o em Gest\u00e3o de Informa\u00e7\u00e3o (MagIC)\/NOVA IMS"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005855","name":"Universidade Nova de Lisboa","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005855","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Educ Inf Technol"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Electronic learning (e-learning) is considered the new norm of learning. One of the significant drawbacks of e-learning in comparison to the traditional classroom is that teachers cannot monitor the students' attentiveness. Previous literature used physical facial features or emotional states in detecting attentiveness. Other studies proposed combining physical and emotional facial features; however, a mixed model that only used a webcam was not tested. The study objective is to develop a machine learning (ML) model that automatically estimates students' attentiveness during e-learning classes using only a webcam. The model would help in evaluating teaching methods for e-learning. This study collected videos from seven students. The webcam of personal computers is used to obtain a video, from which we build a feature set that characterizes a student's physical and emotional state based on their face. This characterization includes eye aspect ratio (EAR), Yawn aspect ratio (YAR), head pose, and emotional states.<\/jats:p><jats:p>A total of eleven variables are used in the training and validation of the model. ML algorithms are used to estimate individual students' attention levels. The ML models tested are decision trees, random forests, support vector machines (SVM), and extreme gradient boosting (XGBoost). Human observers' estimation of attention level is used as a reference. Our best attention classifier is the XGBoost, which achieved an average accuracy of 80.52%, with an AUROC OVR of 92.12%. The results indicate that a combination of emotional and non-emotional measures can generate a classifier with an accuracy comparable to other attentiveness studies. The study would also help assess the e-learning lectures through students' attentiveness. Hence will assist in developing the e-learning lectures by generating an attentiveness report for the tested lecture.<\/jats:p>","DOI":"10.1007\/s10639-023-11814-5","type":"journal-article","created":{"date-parts":[[2023,5,1]],"date-time":"2023-05-01T03:24:46Z","timestamp":1682911486000},"page":"15717-15737","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Machine Learning applied to student attentiveness detection: Using emotional and non-emotional measures"],"prefix":"10.1007","volume":"28","author":[{"given":"Mohamed","family":"Elbawab","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4862-8177","authenticated-orcid":false,"given":"Roberto","family":"Henriques","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,5,1]]},"reference":[{"key":"11814_CR1","doi-asserted-by":"crossref","unstructured":"Alam, A. (2021). Should Robots Replace Teachers? Mobilization of AI and Learning Analytics in Education. In\u00a02021 International Conference on Advances in Computing, Communication, and Control (ICAC3)\u00a0(pp. 1\u201312). IEEE","DOI":"10.1109\/ICAC353642.2021.9697300"},{"issue":"1","key":"11814_CR2","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1007\/s10639-017-9616-z","volume":"23","author":"B Bakhshinategh","year":"2018","unstructured":"Bakhshinategh, B., Zaiane, O. R., ElAtia, S., & Ipperciel, D. (2018). Educational data mining applications and tasks: A survey of the last 10 years. Education and Information Technologies, 23(1), 537\u2013553.","journal-title":"Education and Information Technologies"},{"key":"11814_CR3","first-page":"542","volume":"1\u20133","author":"D Berrar","year":"2018","unstructured":"Berrar, D. (2018). Cross-validation. Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics, 1\u20133, 542\u2013545.","journal-title":"Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics"},{"key":"11814_CR4","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","volume":"16","author":"NV Chawla","year":"2002","unstructured":"Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321\u2013357.","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"4","key":"11814_CR5","doi-asserted-by":"publisher","first-page":"371","DOI":"10.2190\/EC.47.4.b","volume":"47","author":"HR Chen","year":"2012","unstructured":"Chen, H. R. (2012). Assessment of to e-learning by monitoring facial expressions for computer network courses. Journal of Educational Computing Research, 47(4), 371\u2013385.","journal-title":"Journal of Educational Computing Research"},{"key":"11814_CR6","doi-asserted-by":"publisher","first-page":"75264","DOI":"10.1109\/ACCESS.2020.2988510","volume":"8","author":"L Chen","year":"2020","unstructured":"Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. Ieee Access, 8, 75264\u201375278.","journal-title":"Ieee Access"},{"key":"11814_CR7","doi-asserted-by":"crossref","unstructured":"Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining 785\u2013794","DOI":"10.1145\/2939672.2939785"},{"key":"11814_CR8","doi-asserted-by":"crossref","unstructured":"Cutler, A., Cutler, D. R., & Stevens, J. R. (2012). Random forests. In Ensemble machine learning (pp. 157\u2013175). Springer, Boston, MA","DOI":"10.1007\/978-1-4419-9326-7_5"},{"issue":"3","key":"11814_CR9","first-page":"032042","volume":"199","author":"Q Deng","year":"2018","unstructured":"Deng, Q., & Wu, Z. (2018). Students\u2019 attention assessment in elearning based on machine learning. In IOP Conference Series: Earth and Environmental Science, 199(3), 032042. IOP Publishing.","journal-title":"In IOP Conference Series: Earth and Environmental Science"},{"key":"11814_CR10","unstructured":"Domingos, P., & Provost, F. (2000). Well-trained PETs: Improving probability estimation trees. CDER WorkingPaper, Stern School of Business. New York, NY: New York University"},{"issue":"8","key":"11814_CR11","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","volume":"27","author":"T Fawcett","year":"2006","unstructured":"Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861\u2013874.","journal-title":"Pattern Recognition Letters"},{"issue":"1","key":"11814_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/10904018.2000.10499032","volume":"14","author":"WSZ Ford","year":"2000","unstructured":"Ford, W. S. Z., Wolvin, A. D., & Chung, S. (2000). Students\u2019 self-perceived listening competencies in the basic speech communication course. International Journal of Listening, 14(1), 1\u201313.","journal-title":"International Journal of Listening"},{"key":"11814_CR13","doi-asserted-by":"crossref","unstructured":"Gherhe\u0219, V., Stoian, C. E., F\u0103rca\u0219iu, M. A., & Stanici, M. (2021). E-learning vs. face-to-face learning: Analyzing students'preferences and behaviors.\u00a0Sustainability,\u00a013(8), 4381","DOI":"10.3390\/su13084381"},{"key":"11814_CR14","doi-asserted-by":"crossref","unstructured":"Gligori\u0107, N., Uzelac, A., & Krco, S. (2012, March). Smart classroom: real-time feedback on lecture quality. In 2012 IEEE International Conference on Pervasive Computing and Communications Workshops. 391\u2013394. IEEE","DOI":"10.1109\/PerComW.2012.6197517"},{"key":"11814_CR15","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1007\/s10648-019-09514-z","volume":"33","author":"P Goldberg","year":"2021","unstructured":"Goldberg, P., S\u00fcmer, \u00d6., St\u00fcrmer, K., Wagner, W., G\u00f6llner, R., Gerjets, P., Kasneci, E., & Trautwein, U. (2021). Attentive or not? Toward a machine learning approach to assessing students\u2019 visible engagement in classroom instruction. Educational Psychology Review, 33, 27\u201349.","journal-title":"Educational Psychology Review"},{"key":"11814_CR16","doi-asserted-by":"crossref","unstructured":"Goodfellow, I. J., Erhan, D., Carrier, P. L., Courville, A., Mirza, M., Hamner, B., & Bengio, Y. (2013). Challenges in representation learning: A report on three machine learning contests. In International conference on neural information processing 117\u2013124. Springer, Berlin, Heidelberg","DOI":"10.1007\/978-3-642-42051-1_16"},{"issue":"11","key":"11814_CR17","doi-asserted-by":"publisher","first-page":"3894","DOI":"10.3390\/app10113894","volume":"10","author":"R Hasan","year":"2020","unstructured":"Hasan, R., Palaniappan, S., Mahmood, S., Abbas, A., Sarker, K. U., & Sattar, M. U. (2020). Predicting student performance in higher educational institutions using video learning analytics and data mining techniques. Applied Sciences, 10(11), 3894.","journal-title":"Applied Sciences"},{"key":"11814_CR18","volume-title":"OpenCV computer vision with python","author":"J Howse","year":"2013","unstructured":"Howse, J. (2013). OpenCV computer vision with python. Packt Publishing."},{"key":"11814_CR19","volume":"1","author":"GJ Hwang","year":"2020","unstructured":"Hwang, G. J., Xie, H., Wah, B. W., & Ga\u0161evi\u0107, D. (2020). Vision, challenges, roles and research issues of Artificial Intelligence in Education. Computers and Education: Artificial Intelligence, 1, 100001.","journal-title":"Computers and Education: Artificial Intelligence"},{"issue":"5","key":"11814_CR20","doi-asserted-by":"publisher","first-page":"2797","DOI":"10.1007\/s10639-019-09892-5","volume":"24","author":"A Jalal","year":"2019","unstructured":"Jalal, A., & Mahmood, M. (2019). Students\u2019 behavior mining in e-learning environment using cognitive processes with information technologies. Education and Information Technologies, 24(5), 2797\u20132821.","journal-title":"Education and Information Technologies"},{"key":"11814_CR21","doi-asserted-by":"crossref","unstructured":"Karjo, C. H., Andreani, W., Herawati, A., Ying, Y., Yasyfin, A. P., & Marie, K. (2022). Technological Challenges and Strategies in Implementing e-Learning in Higher Education. In\u00a02022 10th International Conference on Information and Education Technology (ICIET) 184\u2013188. IEEE","DOI":"10.1109\/ICIET55102.2022.9778948"},{"key":"11814_CR22","unstructured":"Khaireddin, Y., & Chen, Z. (2021). Facial emotion recognition: State of the art performance on FER2013. arXiv preprint arXiv:2105.03588"},{"key":"11814_CR23","doi-asserted-by":"crossref","unstructured":"Korshunov, P., & Marcel, S. (2018). Speaker inconsistency detection in tampered video. In\u00a02018 26th European signal processing conference (EUSIPCO) 2375-2379. IEEE","DOI":"10.23919\/EUSIPCO.2018.8553270"},{"key":"11814_CR24","doi-asserted-by":"crossref","unstructured":"Ku\u010dak, D., Juri\u010di\u0107, V., & \u0110ambi\u0107, G. (2018). MACHINE LEARNING IN EDUCATION-A SURVEY OF CURRENT RESEARCH TRENDS. Annals of DAAAM & Proceedings, 29","DOI":"10.2507\/29th.daaam.proceedings.059"},{"key":"11814_CR25","unstructured":"Li, S., & Deng, W. (2020). Deep facial expression recognition: A survey. IEEE transactions on affective computing"},{"key":"11814_CR26","doi-asserted-by":"crossref","unstructured":"Lim, W. N. (2017). Improving student engagement in higher education through mobile-based interactive teaching model using socrative. In 2017 IEEE Global Engineering Education Conference (EDUCON) 404\u2013412. IEEE","DOI":"10.1109\/EDUCON.2017.7942879"},{"issue":"1","key":"11814_CR27","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1007\/s12528-021-09274-2","volume":"34","author":"AM Maatuk","year":"2022","unstructured":"Maatuk, A. M., Elberkawi, E. K., Aljawarneh, S., Rashaideh, H., & Alharbi, H. (2022). The COVID-19 pandemic and E-learning: Challenges and opportunities from the perspective of students and instructors. Journal of Computing in Higher Education, 34(1), 21\u201338.","journal-title":"Journal of Computing in Higher Education"},{"key":"11814_CR28","doi-asserted-by":"crossref","unstructured":"Marra, F., Yang, G. Y., Tr\u00e6holt, C., Larsen, E., Rasmussen, C. N., & You, S. (2012). Demand profile study of battery electric vehicle under different charging options. In 2012 IEEE power and energy society general meeting 1\u20137. IEEE","DOI":"10.1109\/PESGM.2012.6345063"},{"key":"11814_CR29","volume":"2","author":"V Matzavela","year":"2021","unstructured":"Matzavela, V., & Alepis, E. (2021). Decision tree learning through a predictive model for student academic performance in intelligent m-learning environments. Computers and Education: Artificial Intelligence, 2, 100035.","journal-title":"Computers and Education: Artificial Intelligence"},{"issue":"3","key":"11814_CR30","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1109\/TNN.2006.873281","volume":"17","author":"ME Mavroforakis","year":"2006","unstructured":"Mavroforakis, M. E., & Theodoridis, S. (2006). A geometric approach to support vector machine (SVM) classification. IEEE Transactions on Neural Networks, 17(3), 671\u2013682.","journal-title":"IEEE Transactions on Neural Networks"},{"issue":"3","key":"11814_CR31","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1080\/08923647.2020.1847626","volume":"35","author":"HI Mellieon Jr","year":"2021","unstructured":"Mellieon, H. I., Jr., & Robinson, P. A. (2021). The new norm: Faculty perceptions of condensed online learning. American Journal of Distance Education, 35(3), 170\u2013183.","journal-title":"American Journal of Distance Education"},{"key":"11814_CR32","doi-asserted-by":"crossref","unstructured":"Negron, T. P., & Graves, C. A. (2017). Classroom Attentiveness Classification Tool (ClassACT): The system introduction. In 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE 26-29","DOI":"10.1109\/PERCOMW.2017.7917513"},{"key":"11814_CR33","doi-asserted-by":"crossref","unstructured":"Pinzon-Gonzalez, J. G., & Barba-Guaman, L. (2022). Use of Head Position Estimation for Attention Level Detection in Remote Classrooms. In Proceedings of the Future Technologies Conference (FTC) 2021\u00a0 1, 275\u2013293. Springer International Publishing.","DOI":"10.1007\/978-3-030-89906-6_20"},{"key":"11814_CR34","doi-asserted-by":"crossref","unstructured":"Revadekar, A., Oak, S., Gadekar, A., & Bide, P. (2020). Gauging attention of students in an e-learning environment. In 2020 IEEE 4th Conference on Information & Communication Technology (CICT) 1\u20136. IEEE","DOI":"10.1109\/CICT51604.2020.9312048"},{"issue":"6","key":"11814_CR35","doi-asserted-by":"publisher","first-page":"601","DOI":"10.1109\/TSMCC.2010.2053532","volume":"40","author":"C Romero","year":"2010","unstructured":"Romero, C., & Ventura, S. (2010). Educational data mining: a review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (applications and Reviews), 40(6), 601\u2013618.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics, Part C (applications and Reviews)"},{"issue":"3","key":"11814_CR36","volume":"10","author":"C Romero","year":"2020","unstructured":"Romero, C., & Ventura, S. (2020). Educational data mining and learning analytics: An updated survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(3), e1355.","journal-title":"Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery"},{"issue":"6","key":"11814_CR37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3365757","volume":"52","author":"MK Saini","year":"2019","unstructured":"Saini, M. K., & Goel, N. (2019). How smart are smart classrooms? A review of smart classroom technologies. ACM Computing Surveys (CSUR), 52(6), 1\u201328.","journal-title":"ACM Computing Surveys (CSUR)"},{"key":"11814_CR38","doi-asserted-by":"crossref","unstructured":"Shah, N. A., Meenakshi, K., Agarwal, A., & Sivasubramanian, S. (2021). Assessment of Student Attentiveness to E-Learning by Monitoring Behavioural Elements. In 2021 International Conference on Computer Communication and Informatics (ICCCI) 1\u20137. IEEE","DOI":"10.1109\/ICCCI50826.2021.9402283"},{"key":"11814_CR39","doi-asserted-by":"crossref","unstructured":"Tabassum, T., Allen, A. A., & De, P. (2020). Non-intrusive identification of student attentiveness and finding their correlation with detectable facial emotions. In Proceedings of the 2020 ACM Southeast Conference 127\u2013134","DOI":"10.1145\/3374135.3385263"},{"key":"11814_CR40","doi-asserted-by":"crossref","unstructured":"Yan, K. (2021). Student performance prediction using XGBoost method from a macro perspective. In 2021 2nd International Conference on Computing and Data Science (CDS). IEEE 453-459","DOI":"10.1109\/CDS52072.2021.00084"},{"issue":"5","key":"11814_CR41","doi-asserted-by":"publisher","first-page":"205","DOI":"10.14738\/assrj.75.8211","volume":"7","author":"BN Yusuf","year":"2020","unstructured":"Yusuf, B. N., & Ahmad, J. (2020). Are we prepared enough? A case study of challenges in online learning in a private higher learning institution during the Covid-19 outbreaks. Advances in Social Sciences Research Journal, 7(5), 205\u2013212.","journal-title":"Advances in Social Sciences Research Journal"},{"key":"11814_CR42","doi-asserted-by":"crossref","unstructured":"Zaletelj, J., & Ko\u0161ir, A. (2017). Predicting students\u2019 attention in the classroom from Kinect facial and body features. EURASIP Journal on Image and Video Processing, 2017(1), 1\u201312.","DOI":"10.1186\/s13640-017-0228-8"}],"container-title":["Education and Information Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10639-023-11814-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10639-023-11814-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10639-023-11814-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,4]],"date-time":"2023-12-04T08:10:07Z","timestamp":1701677407000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10639-023-11814-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,1]]},"references-count":42,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["11814"],"URL":"https:\/\/doi.org\/10.1007\/s10639-023-11814-5","relation":{},"ISSN":["1360-2357","1573-7608"],"issn-type":[{"value":"1360-2357","type":"print"},{"value":"1573-7608","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,1]]},"assertion":[{"value":"25 November 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 April 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 May 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"None.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}