{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T15:02:50Z","timestamp":1782313370328,"version":"3.54.5"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T00:00:00Z","timestamp":1731888000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T00:00:00Z","timestamp":1731888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"publisher","award":["350560"],"award-info":[{"award-number":["350560"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Smart Learn. Environ."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In\u00a0learning analytics and in education at large,\u00a0AI explanations are always computed from aggregate data of all the students to offer the \u201caverage\u201d picture. Whereas the average may work for most students, it does not reflect or capture the individual differences or the variability among students. Therefore, instance-level predictions\u2014where explanations for each particular student are presented according to their own data\u2014may help understand how and why predictions were estimated and how a student or teacher may act or make decisions. This study aims to examine the utility of individualized instance-level AI, its value in informing decision-making, and\u2014more importantly\u2014how they can be used to offer personalized feedback. Furthermore, the study examines mispredictions, their explanations and how they offer explanations or affect decision making. Using data from a full course with 126 students, five ML algorithms were implemented with explanatory mechanisms, compared and the best performing algorithm (Random Forest) was therefore selected. The results show that AI explanations, while useful, cannot achieve their full potential without a nuanced human involvement (i.e., hybrid human AI collaboration). Instance-level explainability may allow us to understand individual algorithmic decisions but may not very helpful for personalization or individualized support. In case of mispredictions, the explanations show that algorithms decide based on the \u201cwrong predictors\u201d which underscores the fact that a full data-driven approach cannot be fully trusted with generating plausible recommendations completely on its own and may require human assistance.<\/jats:p>","DOI":"10.1186\/s40561-024-00343-4","type":"journal-article","created":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T04:32:45Z","timestamp":1731904365000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Why explainable AI may not be enough: predictions and mispredictions in decision making in education"],"prefix":"10.1186","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5881-3109","authenticated-orcid":false,"given":"Mohammed","family":"Saqr","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sonsoles","family":"L\u00f3pez-Pernas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,11,18]]},"reference":[{"key":"343_CR1","doi-asserted-by":"publisher","unstructured":"Alamri, L. H., Almuslim, R. S., Alotibi, M. S., Alkadi, D. K., Khan, U., I., & Aslam, N. (2020). Predicting student academic performance using support vector machine and random forest. In&nbsp;Proceedings of the 2020 3rd international conference on education technology management. ICETM 2020: 2020 3rd International conference on education technology management, London United Kingdom. https:\/\/doi.org\/10.1145\/3446590.3446607","DOI":"10.1145\/3446590.3446607"},{"key":"343_CR2","doi-asserted-by":"publisher","first-page":"129843","DOI":"10.1109\/ACCESS.2022.3227072","volume":"10","author":"M Adnan","year":"2022","unstructured":"Adnan, M., Irfan Uddin, M., Khan, E., Alharithi, F. S., Amin, S., & Alzahrani, A. A. (2022). Earliest possible global and local interpretation of students\u2019 performance in virtual learning environment by leveraging explainable AI. IEEE Access : Practical Innovations, Open Solutions, 10, 129843\u2013129864.","journal-title":"IEEE Access : Practical Innovations, Open Solutions"},{"key":"343_CR3","first-page":"1","volume":"31","author":"A Asselman","year":"2021","unstructured":"Asselman, A., Khaldi, M., & Aammou, S. (2021). Enhancing the prediction of student performance based on the machine learning XGBoost algorithm. Interactive Learning Environments, 31, 1\u201320.","journal-title":"Interactive Learning Environments"},{"issue":"4","key":"343_CR4","doi-asserted-by":"publisher","first-page":"1052","DOI":"10.1007\/s40593-021-00285-9","volume":"32","author":"RS Baker","year":"2022","unstructured":"Baker, R. S., & Hawn, A. (2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32(4), 1052\u20131092.","journal-title":"International Journal of Artificial Intelligence in Education"},{"issue":"6","key":"343_CR5","doi-asserted-by":"publisher","first-page":"53","DOI":"10.14742\/ajet.6436","volume":"36","author":"R Barmaki","year":"2020","unstructured":"Barmaki, R., & Guo, Z. (2020). Deep neural networks for collaborative learning analytics: Evaluating team collaborations using student gaze point prediction: Evaluating team collaborations using students\u2019 gaze point prediction. Australasian Journal of Educational Technology, 36(6), 53\u201371.","journal-title":"Australasian Journal of Educational Technology"},{"key":"343_CR6","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"A Barredo Arrieta","year":"2020","unstructured":"Barredo Arrieta, A., D\u00edaz-Rodr\u00edguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. An International Journal on Information Fusion, 58, 82\u2013115.","journal-title":"An International Journal on Information Fusion"},{"key":"343_CR7","doi-asserted-by":"publisher","DOI":"10.1145\/3637211","author":"N Bernard","year":"2023","unstructured":"Bernard, N., & Balog, K. (2023). A systematic review of fairness, accountability, transparency and ethics in information retrieval. ACM Comput Surv. https:\/\/doi.org\/10.1145\/3637211","journal-title":"ACM Comput Surv"},{"key":"343_CR8","first-page":"1","volume":"19","author":"P Biecek","year":"2018","unstructured":"Biecek, P. (2018). Dalex: Explainers for complex predictive models in R. Journal of Machine Learning Research: JMLR, 19, 1\u20135.","journal-title":"Journal of Machine Learning Research: JMLR"},{"key":"343_CR9","doi-asserted-by":"publisher","DOI":"10.1201\/9780429027192","volume-title":"Explanatory model analysis: Explore, explain, and examine predictive models","author":"P Biecek","year":"2021","unstructured":"Biecek, P., & Burzykowski, T. (2021). Explanatory model analysis: Explore, explain, and examine predictive models. CRC."},{"key":"343_CR10","doi-asserted-by":"publisher","first-page":"102399","DOI":"10.1016\/j.lindif.2023.102399","volume":"109","author":"K Bobrowicz","year":"2024","unstructured":"Bobrowicz, K., L\u00f3pez-Pernas, S., Teuber, Z., Saqr, M., & Greiff, S. (2024). Prospects in the field of learning and individual differences: Examining the past to forecast the future using bibliometrics. Learning and Individual Differences, 109, 102399.","journal-title":"Learning and Individual Differences"},{"key":"343_CR11","doi-asserted-by":"publisher","unstructured":"Chen, T., & Guestrin, C. (2016). XGBoost. In&nbsp;Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. KDD \u201916: The 22nd ACM SIGKDD international conference on knowledge discovery and data mining, San Francisco California USA. https:\/\/doi.org\/10.1145\/2939672.2939785","DOI":"10.1145\/2939672.2939785"},{"key":"343_CR12","unstructured":"Chen, T., He, T., Benesty, M., Khotilovich, V., Tang, Y., Cho, H., Chen, K., Mitchell, R., Cano, I., Zhou, T., Li, M., Xie, J., Lin, M., Geng, Y., Li, Y., & Yuan, J. (2024). xgboost: Extreme Gradient Boosting. https:\/\/CRAN.R-project.org\/package=xgboost."},{"issue":"4","key":"343_CR13","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1080\/00461520.2014.965823","volume":"49","author":"MTH Chi","year":"2014","unstructured":"Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219\u2013243.","journal-title":"Educational Psychologist"},{"key":"343_CR14","doi-asserted-by":"crossref","unstructured":"Do\u0161ilovi\u0107, F. K., Br\u010di\u0107, M., & Hlupi\u0107, N. (2018). Explainable artificial intelligence: A survey. In 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO), 0210\u20130215.","DOI":"10.23919\/MIPRO.2018.8400040"},{"key":"343_CR15","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1007\/978-0-387-84858-7_15","volume-title":"The Elements of Statistical Learning","author":"T Hastie","year":"2009","unstructured":"Hastie, T., Tibshirani, R., & Friedman, J. (2009). Random forests. The elements of statistical learning (pp. 587\u2013604). Springer."},{"key":"343_CR16","volume-title":"The impact of artificial intelligence on learning, teaching, and education","author":"T Ilkka","year":"2018","unstructured":"Ilkka, T. (2018). The impact of artificial intelligence on learning, teaching, and education. European Union."},{"issue":"9","key":"343_CR17","doi-asserted-by":"publisher","first-page":"12855","DOI":"10.1007\/s10639-022-11120-6","volume":"27","author":"Y Jang","year":"2022","unstructured":"Jang, Y., Choi, S., Jung, H., & Kim, H. (2022). Practical early prediction of students\u2019 performance using machine learning and eXplainable AI. Education and Information Technologies, 27(9), 12855\u201312889.","journal-title":"Education and Information Technologies"},{"key":"343_CR18","first-page":"100074","volume":"3","author":"H Khosravi","year":"2022","unstructured":"Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y. S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Ga\u0161evi\u0107, D. (2022). Explainable artificial intelligence in education. Computers and Education: Artificial Intelligence, 3, 100074.","journal-title":"Computers and Education: Artificial Intelligence"},{"issue":"3","key":"343_CR19","doi-asserted-by":"publisher","first-page":"388","DOI":"10.1080\/0960085X.2021.1927212","volume":"31","author":"N Kordzadeh","year":"2022","unstructured":"Kordzadeh, N., & Ghasemaghaei, M. (2022). Algorithmic bias: Review, synthesis, and future research directions. European Journal of Information Systems, 31(3), 388\u2013409.","journal-title":"European Journal of Information Systems"},{"key":"343_CR20","unstructured":"Kuhn, M., & Vaughan, D. (2023). parsnip: A common API to modeling and analysis functions. https:\/\/CRAN.R-project.org\/package=parsnip"},{"issue":"3","key":"343_CR21","first-page":"18","volume":"2","author":"A Liaw","year":"2002","unstructured":"Liaw, A., & Wiener, M. (2002). Classification and regression by randomForest. R News, 2(3), 18\u201322.","journal-title":"R News"},{"key":"343_CR22","doi-asserted-by":"crossref","unstructured":"Lin, J., Dai, W., Lim, L. A., Tsai, Y. S., Mello, R. F., Khosravi, H., Gasevic, D., & Chen, G. (2023). Learner-centred analytics of feedback content in higher education. In LAK23: 13th international learning analytics and knowledge conference, 100\u2013110.","DOI":"10.1145\/3576050.3576064"},{"key":"343_CR23","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1007\/978-3-031-08076-0_4","volume-title":"The Multimodal Learning Analytics Handbook","author":"J Malmberg","year":"2022","unstructured":"Malmberg, J., Saqr, M., J\u00e4rvenoja, H., Haataja, E., Pijeira-D\u00edaz, H. J., & J\u00e4rvel\u00e4, S. (2022). Modeling the complex interplay between monitoring events for regulated learning with psychological networks. In M. Giannakos, D. Spikol, Di D. Mitri, K. Sharma, X. Ochoa, & R. Hammad (Eds.), The Multimodal Learning Analytics Handbook (pp. 79\u2013104). Springer International Publishing."},{"key":"343_CR24","doi-asserted-by":"publisher","first-page":"10","DOI":"10.4337\/9781800375413.00010","volume-title":"Handbook of Artificial Intelligence in Education","author":"G McCalla","year":"2023","unstructured":"McCalla, G. (2023). The history of artificial intelligence in education \u2013 the first quarter century. Handbook of Artificial Intelligence in Education (pp. 10\u201329). Edward Elgar Publishing."},{"key":"343_CR25","unstructured":"Meyer, D., Dimitriadou, E., Hornik, K., Weingessel, A., & Leisch, F. (2023). e1071: misc functions of the department of statistics, probability theory group (formerly: e1071), tu wien. https:\/\/CRAN.R-project.org\/package=e1071"},{"key":"343_CR26","doi-asserted-by":"publisher","first-page":"100214","DOI":"10.1016\/j.tine.2023.100214","volume":"33","author":"M Nachouki","year":"2023","unstructured":"Nachouki, M., Mohamed, E. A., Mehdi, R., & Abou Naaj, M. (2023). Student course grade prediction using the random forest algorithm: Analysis of predictors\u2019 importance. Trends in Neuroscience and Education, 33, 100214.","journal-title":"Trends in Neuroscience and Education"},{"key":"343_CR27","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-023-00331-8","author":"M Nagy","year":"2023","unstructured":"Nagy, M., & Molontay, R. (2023). Interpretable dropout prediction: Towards XAI-Based personalized intervention. International Journal of Artificial Intelligence in Education. https:\/\/doi.org\/10.1007\/s40593-023-00331-8","journal-title":"International Journal of Artificial Intelligence in Education"},{"issue":"1","key":"343_CR28","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1007\/s00146-020-01005-y","volume":"36","author":"MMM Peeters","year":"2021","unstructured":"Peeters, M. M. M., van Diggelen, J., van den Bosch, K., Bronkhorst, A., Neerincx, M. A., Schraagen, J. M., & Raaijmakers, S. (2021). Hybrid collective intelligence in a human\u2013AI society. AI & Society, 36(1), 217\u2013238.","journal-title":"AI & Society"},{"key":"343_CR29","doi-asserted-by":"publisher","first-page":"42200","DOI":"10.1109\/ACCESS.2020.2976199","volume":"8","author":"R Roscher","year":"2020","unstructured":"Roscher, R., Bohn, B., Duarte, M. F., & Garcke, J. (2020). Explainable machine learning for scientific insights and discoveries. Ieee Access : Practical Innovations, Open Solutions, 8, 42200\u201342216.","journal-title":"Ieee Access : Practical Innovations, Open Solutions"},{"key":"343_CR30","doi-asserted-by":"publisher","first-page":"102499","DOI":"10.1016\/j.lindif.2024.102499","volume":"114","author":"M Saqr","year":"2024","unstructured":"Saqr, M., Cheng, R., L\u00f3pez-Pernas, S., & Beck, E. D. (2024). Idiographic artificial intelligence to explain students\u2019 self-regulation: Toward precision education. Learning and Individual Differences, 114, 102499.","journal-title":"Learning and Individual Differences"},{"issue":"7","key":"343_CR31","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1080\/0142159X.2017.1309376","volume":"39","author":"M Saqr","year":"2017","unstructured":"Saqr, M., Fors, U., & Tedre, M. (2017). How learning analytics can early predict under-achieving students in a blended medical education course. Medical Teacher, 39(7), 757\u2013767.","journal-title":"Medical Teacher"},{"key":"343_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/03075079.2022.2061450","volume":"47","author":"M Saqr","year":"2022","unstructured":"Saqr, M., Jovanovic, J., Viberg, O., & Ga\u0161evi\u0107, D. (2022). Is there order in the mess? A single paper meta-analysis approach to identification of predictors of success in learning analytics. Studies in Higher Education, 47, 1\u201322.","journal-title":"Studies in Higher Education"},{"key":"343_CR33","doi-asserted-by":"crossref","unstructured":"Saqr, M., & L\u00f3pez-Pernas, S. (2021). Idiographic learning analytics: A definition and a case study. In 2021 International conference on advanced learning technologies (ICALT), 163\u2013165.","DOI":"10.1109\/ICALT52272.2021.00056"},{"key":"343_CR34","unstructured":"Strumbelj, E., & Kononenko, I. (2010). An efficient explanation of individual classifications using game theory. https:\/\/www.jmlr.org\/papers\/volume11\/strumbelj10a\/strumbelj10a.pdf?ref=https:\/\/githubhelp.com"},{"key":"343_CR35","first-page":"446","volume-title":"Advances in Intelligent Systems and Computing","author":"JJ Thomas","year":"2020","unstructured":"Thomas, J. J., & Ali, A. M. (2020). Dispositional learning analytics structure integrated with recurrent neural networks in predicting students performance. Advances in Intelligent systems and Computing (pp. 446\u2013456). Springer International Publishing."},{"key":"343_CR36","doi-asserted-by":"crossref","unstructured":"Yan, K. (2021). Student performance prediction using XGBoost method from A macro perspective. In  2021 2nd international conference on computing and data science (CDS), pp. 453\u2013459.","DOI":"10.1109\/CDS52072.2021.00084"}],"container-title":["Smart Learning Environments"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40561-024-00343-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40561-024-00343-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40561-024-00343-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T05:04:43Z","timestamp":1731906283000},"score":1,"resource":{"primary":{"URL":"https:\/\/slejournal.springeropen.com\/articles\/10.1186\/s40561-024-00343-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,18]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["343"],"URL":"https:\/\/doi.org\/10.1186\/s40561-024-00343-4","relation":{},"ISSN":["2196-7091"],"issn-type":[{"value":"2196-7091","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,18]]},"assertion":[{"value":"20 July 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 October 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 November 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"52"}}