{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T22:08:20Z","timestamp":1776377300997,"version":"3.51.2"},"reference-count":45,"publisher":"Emerald","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,6,10]]},"abstract":"<jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>This study focused on early warning in K-12 online education and aims to address the following research gaps: timing, model performance and understandability by completing the following analytic tasks: (1) determine the best prediction timing based on changes in course requirements; (2) compare early warning models with the correct evaluation indicators; (3) interpret a complex predictive model using interpretable AI techniques. Through holistic analyses, the case study shows that the semester can be divided into three stages with different course requirements. Students who failed to meet the course requirements of individual stages would be predicted as at-risk. Through interpretable AI techniques, the key predictors of individual stages were identified, and the factors causing a student to be predicted as at-risk were also revealed. In addition, multiple at-risk types were identified through the analyses.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Design\/methodology\/approach<\/jats:title>\n                    <jats:p>This study employed a variational autoencoder and time series segmentation to detect changes in learning behavioral patterns resulting from alterations in course requirements. The semester was divided into three stages. Subsequently, complex ensemble machine learning classifiers were employed for early warnings at each stage, enabling accurate prediction of students' learning performance. Interpretable AI techniques were employed to gain insight, identify characteristics of different at-risk types and suggest personalized interventions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Findings<\/jats:title>\n                    <jats:p>The case study analyzed 16,011 K-12 online school students. Major Findings include: (1) The semester was divided into three segments based on learning pattern transition points. XGBoost can identify the most at-risk students in each segment. Results indicate S2 is a relatively appropriate stage for early warning prediction and intervention. By the end of the second stage, the models can already identify over 75% of at-risk students. A student\u2019s at-risk probability in previous stage has the most important influence on his performance in the next stage. If a student has high at-risk probability, he will have more possibility to be labeled as potential at-risk in the next stage, if he has low at-risk probability, he will have more possibility to be labeled as potential successful in the next stage, this reflects the influence between neighbor learning stages. We also found that, early warning can be most effectively conducted in the last week of each of these three stages. (2) From S1 to S3 stages, significant changes in the categories of key behaviors were observed, the key behavior at S1 stage was assignment viewing and submitting, at S2 stage was resource viewing and grade check, at S3 stage was assignment viewing and submitting, the discussion behavior became not important, while the testing behavior gradually became important, this reflected the changes in the course requirements and the learning strategy. It is evident that in specific stages of the semester, not all behaviors are better when more frequent; some behaviors, when excessive at the wrong time, can have a negative impact on predictive results. (3) Five types of at-risk students were found at each stage through interpretable AI analyses: high engaged at-risk, low engaged at-risk, testing at-risk, low interaction at-risk and un-persistent at-risk, which represented five types of learning patterns that conducted potential at-risk. With the progress of course, students adjusted learning strategy and might convert from one at-risk type to other at-risk type or successful type. The S2 stage was the most important stage, students had their last opportunity to reverse their at-risk trends at this stage. (4) Students who do not perform well in the early stages still have the opportunity to reverse the at-risk trend with effort in the later stages, emphasizing the importance of intervention at the end of the second stage. Similarly, students who perform well in the early stages can still face failure if they do not put in effort in the final stage. Low-engaged students tended to maintain their low-engaged status, resulting in failure at the end of the semester compared to other at-risk types, this was caused by that they had no learning activity to adjust their learning strategy.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Research limitations\/implications<\/jats:title>\n                    <jats:p>A follow-up study can focus on the development of personalized interventions and observe whether these interventions can influence potentially at-risk students' learning strategies, more effectively guiding students to align with course requirements and ultimately leading to improved performance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Originality\/value<\/jats:title>\n                    <jats:p>This study is one of the few that focus on large-scale K-12 early warning prediction and aim to identify at-risk types via interpretable AI for personalized intervention. The authors found that the proposed method can effectively determine the best prediction timing, identify more at-risk students, and gain deep insight into students' learning processes. The authors confirm that the research in their work is original, and all the data given in the paper are real and authentic. The study has not been submitted to peer review and has not been accepted for publication in another journal.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1108\/dta-11-2023-0795","type":"journal-article","created":{"date-parts":[[2025,5,7]],"date-time":"2025-05-07T03:47:21Z","timestamp":1746589641000},"page":"516-538","source":"Crossref","is-referenced-by-count":1,"title":["Interpretable AI techniques unveil the\u00a0factors and types of at-risk early warning: a case study in K-12 online learning"],"prefix":"10.1108","volume":"59","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7710-7231","authenticated-orcid":true,"given":"Jui-Long","family":"Hung","sequence":"first","affiliation":[{"name":"Boise State University Department of Educational Technology, , , ,","place":["Boise, Idaho, USA"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5739-8342","authenticated-orcid":true,"given":"Kerry","family":"Rice","sequence":"additional","affiliation":[{"name":"Boise State University , , ,","place":["Boise, Idaho, USA"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4144-0275","authenticated-orcid":true,"given":"Mingyan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Zhejiang Normal University Zhejiang Key Laboratory of Intelligent Education Technology and Application, , ,","place":["Jinhua, China"]}]}],"member":"140","published-online":{"date-parts":[[2025,5,8]]},"reference":[{"key":"2026012706114848200_ref001","doi-asserted-by":"publisher","first-page":"7519","DOI":"10.1109\/access.2021.3049446","article-title":"Predicting at-risk students at different percentages of course length for early intervention using machine learning models","volume":"9","author":"Adnan","year":"2021","journal-title":"IEEE Access"},{"issue":"1","key":"2026012706114848200_ref002","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s41239-019-0172-z","article-title":"Using learning analytics to develop early-warning system for at-risk students","volume":"16","author":"Ak\u00e7ap\u0131nar","year":"2019","journal-title":"International Journal of Educational Technology in Higher Education"},{"key":"2026012706114848200_ref003","doi-asserted-by":"publisher","first-page":"107649","DOI":"10.1109\/access.2022.3211070","article-title":"An explainable model for identifying at-risk student at higher education","volume":"10","author":"Alwarthan","year":"2022","journal-title":"IEEE Access"},{"key":"2026012706114848200_ref004","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1145\/2330601.2330666","article-title":"Course signals at Purdue: using learning analytics to increase student success","author":"Arnold","year":"2012"},{"issue":"2","key":"2026012706114848200_ref005","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1353\/hsj.2013.0000","article-title":"Do we know who will drop out? A review of the predictors of dropping out of high school: precision, sensitivity, and specificity","volume":"96","author":"Bowers","year":"2013","journal-title":"High School Journal"},{"issue":"3","key":"2026012706114848200_ref006","doi-asserted-by":"publisher","first-page":"528","DOI":"10.1111\/bjet.12275","article-title":"Survey of learning experiences and influence of learning style preferences on user intentions regarding MOOC s","volume":"46","author":"Chang","year":"2015","journal-title":"British Journal of Educational Technology"},{"key":"2026012706114848200_ref007","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1016\/j.childyouth.2018.11.030","article-title":"Dropout early warning systems for high school students using machine learning","volume":"96","author":"Chung","year":"2019","journal-title":"Children and Youth Services Review"},{"issue":"3","key":"2026012706114848200_ref008","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1080\/15391523.2021.1899877","article-title":"Support provided for K-12 teachers teaching remotely with technology during emergencies: a systematic review","volume":"54","author":"Crompton","year":"2022","journal-title":"Journal of Research on Technology in Education"},{"key":"2026012706114848200_ref009","doi-asserted-by":"publisher","first-page":"10110","DOI":"10.1109\/access.2020.2964845","article-title":"An integrated framework based on latent variational autoencoder for providing early warning of at-risk students","volume":"8","author":"Du","year":"2020","journal-title":"IEEE Access"},{"issue":"1","key":"2026012706114848200_ref010","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1080\/0144929x.2019.1669712","article-title":"A systematic meta-review and analysis of learning analytics research","volume":"40","author":"Du","year":"2021","journal-title":"Behaviour and Information Technology"},{"issue":"3","key":"2026012706114848200_ref011","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1007\/s11423-010-9161-6","article-title":"Towards understanding international graduate student isolation in traditional and online environments","volume":"59","author":"Erichsen","year":"2011","journal-title":"Educational Technology Research and Development"},{"issue":"14","key":"2026012706114848200_ref013","doi-asserted-by":"publisher","first-page":"8885","DOI":"10.3390\/su14148885","article-title":"An interpretable framework for an efficient analysis of students' academic performance","volume":"14","author":"Gligorea","year":"2022","journal-title":"Sustainability"},{"issue":"1","key":"2026012706114848200_ref014","doi-asserted-by":"crossref","first-page":"4","DOI":"10.64152\/10125\/44592","article-title":"Scaling up and zooming in: big data and personalization in language learning","volume":"21","author":"Godwin-Jones","year":"2017","journal-title":"Language, Learning and Technology"},{"key":"2026012706114848200_ref015","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1016\/j.iheduc.2018.02.001","article-title":"Contrasting prediction methods for early warning systems at undergraduate level","volume":"37","author":"Howard","year":"2018","journal-title":"The Internet and Higher Education"},{"issue":"2","key":"2026012706114848200_ref016","doi-asserted-by":"publisher","first-page":"183","DOI":"10.2190\/ec.40.2.c","article-title":"Examining online learning patterns with data mining techniques in peer-moderated and teacher-moderated courses","volume":"40","author":"Hung","year":"2009","journal-title":"Journal of Educational Computing Research"},{"key":"2026012706114848200_ref017","article-title":"Revealing online learning behaviors and activity patterns and making predictions with data mining techniques in online teaching","author":"Hung","year":"2008","journal-title":"MERLOT Journal of Online Learning and Teaching"},{"issue":"1","key":"2026012706114848200_ref018","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1109\/tetc.2015.2504239","article-title":"Identifying at-risk students for early interventions\u2014a time-series clustering approach","volume":"5","author":"Hung","year":"2015","journal-title":"IEEE Transactions on Emerging Topics in Computing"},{"issue":"12","key":"2026012706114848200_ref019","doi-asserted-by":"publisher","first-page":"4506","DOI":"10.1007\/s10489-020-01776-3","article-title":"Multi-split optimized bagging ensemble model selection for multi-class educational data mining","volume":"50","author":"Injadat","year":"2020","journal-title":"Applied Intelligence"},{"key":"2026012706114848200_ref020","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2021.104271","article-title":"Understanding feedback in online learning\u2013A critical review and metaphor analysis","volume":"173","author":"Jensen","year":"2021","journal-title":"Computers and Education"},{"key":"2026012706114848200_ref021","first-page":"14","volume-title":"The Business Case for E-Learning","author":"Kelly","year":"2005"},{"key":"2026012706114848200_ref022","first-page":"136","article-title":"Predicting success: how learners' prior knowledge, skills and activities predict MOOC performance","author":"Kennedy","year":"2015"},{"issue":"1","key":"2026012706114848200_ref023","doi-asserted-by":"publisher","first-page":"117","DOI":"10.2478\/cait-2022-0008","article-title":"Early student-at-risk detection by current learning performance and learning behavior indicators","volume":"22","author":"Kustitskaya","year":"2022","journal-title":"Cybernetics and Information Technologies"},{"issue":"3","key":"2026012706114848200_ref024","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1016\/j.iheduc.2011.04.001","article-title":"Examining the relationship among student perception of support, course satisfaction, and learning outcomes in online learning","volume":"14","author":"Lee","year":"2011","journal-title":"The Internet and Higher Education"},{"issue":"12","key":"2026012706114848200_ref025","doi-asserted-by":"publisher","first-page":"3197","DOI":"10.1007\/s10115-022-01756-8","article-title":"Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond","volume":"64","author":"Li","year":"2022","journal-title":"Knowledge and Information Systems"},{"key":"2026012706114848200_ref026","first-page":"4765","article-title":"A unified approach to interpreting model predictions","volume":"30","author":"Lundberg","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"7","key":"2026012706114848200_ref012","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1177\/0013124516645164","article-title":"Mentoring students back on-track to graduation: Program results from five communities","volume":"49","author":"Mac Iver","year":"2017","journal-title":"Education and Urban Society"},{"issue":"3","key":"2026012706114848200_ref027","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1080\/19345747.2019.1615156","article-title":"An efficacy study of a ninth-grade early warning indicator intervention","volume":"12","author":"Mac Iver","year":"2019","journal-title":"Journal of Research on Educational Effectiveness"},{"issue":"3","key":"2026012706114848200_ref028","doi-asserted-by":"publisher","first-page":"1124","DOI":"10.14419\/ijet.v7i3.9789","article-title":"Demystifying learning analytics in personalised learning","volume":"7","author":"Maseleno","year":"2018","journal-title":"International Journal of Engineering and Technology (UAE)"},{"issue":"1","key":"2026012706114848200_ref029","doi-asserted-by":"publisher","first-page":"266","DOI":"10.1002\/rev3.3183","article-title":"Re-envisioning the purpose of early warning systems: shifting the mindset from student identification to meaningful prediction and intervention","volume":"8","author":"McMahon","year":"2020","journal-title":"The Review of Education"},{"issue":"1","key":"2026012706114848200_ref030","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1007\/s40747-021-00383-0","article-title":"An interpretable prediction method for university student academic crisis warning","volume":"8","author":"Mingyu","year":"2022","journal-title":"Complex and Intelligent Systems"},{"key":"2026012706114848200_ref031","volume-title":"Interpretable Machine Learning","author":"Molnar","year":"2020"},{"issue":"4","key":"2026012706114848200_ref032","first-page":"207","article-title":"Factors influencing adult learners' decision to drop out or persist in online learning","volume":"12","author":"Park","year":"2009","journal-title":"Journal of Educational Technology and Society"},{"issue":"1","key":"2026012706114848200_ref033","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40561-020-00140-9","article-title":"A systematic literature review of personalized learning terms","volume":"7","author":"Shemshack","year":"2020","journal-title":"Smart Learning Environments"},{"issue":"2","key":"2026012706114848200_ref034","doi-asserted-by":"publisher","first-page":"86","DOI":"10.46627\/silet.v1i2.38","article-title":"The Challenges of online learning during the COVID-19 pandemic: an essay analysis of performing arts education students","volume":"1","author":"Simamora","year":"2020","journal-title":"Studies in Learning and Teaching"},{"issue":"4","key":"2026012706114848200_ref035","article-title":"GPS: shaping student success one conversation at a time","volume":"33","author":"Star","year":"2010","journal-title":"Educause Quarterly"},{"issue":"2","key":"2026012706114848200_ref036","doi-asserted-by":"publisher","first-page":"612","DOI":"10.14569\/ijacsa.2020.0110277","article-title":"Evaluating the impact of GINI index and information gain on classification using decision tree classifier algorithm","volume":"11","author":"Tangirala","year":"2020","journal-title":"International Journal of Advanced Computer Science and Applications"},{"issue":"3","key":"2026012706114848200_ref037","doi-asserted-by":"publisher","first-page":"115","DOI":"10.24059\/olj.v13i3.1659","article-title":"Factors that influence students' decision to drop out of online courses","volume":"13","author":"Willging","year":"2009","journal-title":"Journal of Asynchronous Learning Networks"},{"issue":"4","key":"2026012706114848200_ref038","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbaa321","article-title":"Do we need different machine learning algorithms for QSAR modeling? A comprehensive assessment of 16 machine learning algorithms on 14 QSAR data sets","volume":"22","author":"Wu","year":"2021","journal-title":"Briefings in Bioinformatics"},{"issue":"1","key":"2026012706114848200_ref039","doi-asserted-by":"publisher","first-page":"1019","DOI":"10.1007\/s10639-022-11206-1","article-title":"Early warning mechanism of interactive learning process based on temporal memory enhancement model","volume":"28","author":"Xia","year":"2023","journal-title":"Education and Information Technologies"},{"key":"2026012706114848200_ref040","article-title":"Behavioral patterns of knowledge construction in online cooperative translation activities","volume":"36","author":"Yang","year":"2017","journal-title":"The Internet and Higher Education"},{"issue":"8","key":"2026012706114848200_ref041","volume":"57","author":"Zeng","year":"2021","journal-title":"Survey of Interpretability Research on Deep Learning Models. Computer Engineering and Applications"},{"issue":"5","key":"2026012706114848200_ref042","doi-asserted-by":"publisher","first-page":"1130","DOI":"10.1177\/07356331211065097","article-title":"Analyzing and interpreting students' self-regulated learning patterns combining time-series feature extraction, segmentation, and clustering","volume":"60","author":"Zhang","year":"2022","journal-title":"Journal of Educational Computing Research"},{"issue":"2","key":"2026012706114848200_ref043","doi-asserted-by":"publisher","first-page":"778","DOI":"10.1109\/tip.2017.2771408","article-title":"Robust LSTM-autoencoders for face de-occlusion in the wild","volume":"27","author":"Zhao","year":"2017","journal-title":"IEEE Transactions on Image Processing"},{"issue":"4","key":"2026012706114848200_ref044","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1108\/itse-10-2021-0192","article-title":"The influence of personalized learning intervention system on student learning a study of junior middle school","volume":"19","author":"Zhou","year":"2022","journal-title":"Interactive Technology and Smart Education"},{"key":"2026012706114848200_ref045","first-page":"299","volume-title":"Handbook of Metacognition in Education","author":"Zimmerman","year":"2009"}],"container-title":["Data Technologies and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/DTA-11-2023-0795\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/dta\/article-pdf\/59\/3\/516\/11202708\/dta-11-2023-0795en.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/www.emerald.com\/dta\/article-pdf\/59\/3\/516\/11202708\/dta-11-2023-0795en.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,27]],"date-time":"2026-01-27T11:11:53Z","timestamp":1769512313000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/dta\/article\/59\/3\/516\/1248859\/Interpretable-AI-techniques-unveil-the-factors-and"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,8]]},"references-count":45,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,6,10]]}},"URL":"https:\/\/doi.org\/10.1108\/dta-11-2023-0795","relation":{},"ISSN":["2514-9288","2514-9318"],"issn-type":[{"value":"2514-9288","type":"print"},{"value":"2514-9318","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,8]]}}}