{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T11:21:13Z","timestamp":1785410473889,"version":"3.56.0"},"reference-count":47,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T00:00:00Z","timestamp":1777593600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Higher Education of the Republic of Kazakhstan","award":["BR24993072"],"award-info":[{"award-number":["BR24993072"]}]},{"award":["BR24993072"],"award-info":[{"award-number":["BR24993072"]}],"id":[{"id":"https:\/\/ror.org\/05tyne317","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>The rapid digitalization of higher education has intensified the need for reliable methods to assess higher-order cognitive skills, particularly critical thinking, in adaptive learning environments. However, most existing assessment approaches rely primarily on test outcomes and academic performance indicators, which do not adequately capture the multidimensional and process-based nature of critical thinking. This study proposes a multi-criteria hierarchical model for identifying and quantitatively assessing students\u2019 critical thinking based on individual process indicators of learning activity in an intelligent educational environment. The model integrates cognitive, metacognitive, and behavioral indicators, including knowledge dynamics, task complexity, time characteristics, learning activity intensity, error rate, level of doubt, user interaction patterns, and system operating modes. These indicators are aggregated into a three-component structure representing metacognitive awareness, analytical depth, and strategic learning activity. The proposed model was empirically validated through a quasi-experimental longitudinal study involving 500 university students divided into control and experimental groups. The results demonstrate a statistically significant increase in all latent components of critical thinking and in the integral indicator within the experimental group. The model shows satisfactory internal consistency (Cronbach\u2019s \u03b1\u22650.77) and acceptable construct validity confirmed by confirmatory factor analysis. The findings indicate that the proposed model can serve as a practical analytical tool for monitoring critical thinking development and supporting personalized learning trajectories in adaptive digital educational systems.<\/jats:p>","DOI":"10.3390\/make8050120","type":"journal-article","created":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T09:33:17Z","timestamp":1777627997000},"page":"120","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Individual Indicators of the Learning Process for Identifying Critical Thinking in Students in Adaptive Learning"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5807-3873","authenticated-orcid":false,"given":"Vassiliy","family":"Serbin","sequence":"first","affiliation":[{"name":"Department of Information Systems, Institute of Automation and Information Technologies, Satbayev University, Almaty 050013, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4313-7966","authenticated-orcid":false,"given":"Mateus","family":"Mendes","sequence":"additional","affiliation":[{"name":"RCM2+ Research Centre for Asset Management and Systems Engineering, Rua Pedro Nunes, 3030-199 Coimbra, Portugal"},{"name":"Coimbra Institute of Engineering, Polytechnic University of Coimbra, Rua Pedro Nunes-Quinta da Nora, 3030-199 Coimbra, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6355-9544","authenticated-orcid":false,"given":"Aray","family":"Kassenkhan","sequence":"additional","affiliation":[{"name":"Department of Software Engineering, Institute of Automation and Information Technologies, Satbayev University, Almaty 050013, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Akbayan","family":"Bekarystankyzy","sequence":"additional","affiliation":[{"name":"School of Digital Technology, Narxoz University, Almaty 050035, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gulnur","family":"Ibragim","sequence":"additional","affiliation":[{"name":"Department of Information Systems, Institute of Automation and Information Technologies, Satbayev University, Almaty 050013, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Azamat","family":"Tolegenov","sequence":"additional","affiliation":[{"name":"School of Digital Technology, Narxoz University, Almaty 050035, Kazakhstan"},{"name":"Department of Computer Engineering, International Information Technology University, Almaty 050035, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,5,1]]},"reference":[{"key":"ref_1","unstructured":"Facione, P.A. (2015). Critical Thinking: What It Is and Why It Counts, Insight Assessment. Technical Report."},{"key":"ref_2","unstructured":"Ennis, R.H. (2011). Critical Thinking: A Streamlined Conception. The Palgrave Handbook of Critical Thinking in Higher Education, Palgrave Macmillan."},{"key":"ref_3","unstructured":"Halpern, D.F., and Dunn, D.S. (2014). Thought and Knowledge: An Introduction to Critical Thinking, Routledge."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zourmpakis, A.I., Kalogiannakis, M., and Papadakis, S. (2023). Adaptive gamification in science education: An analysis of the impact of implementation and adapted game elements on students\u2019 motivation. Computers, 12.","DOI":"10.20944\/preprints202306.1513.v1"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zourmpakis, A.I., Kalogiannakis, M., and Papadakis, S. (2024). The effects of adaptive gamification in science learning: A comparison between traditional inquiry-based learning and gender differences. Computers, 13.","DOI":"10.20944\/preprints202411.0766.v1"},{"key":"ref_6","unstructured":"Gardner, H. (1983). Frames of Mind: The Theory of Multiple Intelligences, Basic Books."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1108\/HESWBL-05-2023-0121","article-title":"Adaptive Learning through Technology: A Technical Review and Implementation","volume":"14","author":"Gransden","year":"2024","journal-title":"High. Educ. Skills Work-Based Learn."},{"key":"ref_8","first-page":"1568","article-title":"Enhancing Learning Outcomes through Adaptive Learning Systems in Education Technology","volume":"2","author":"Asmara","year":"2024","journal-title":"Glob. Int. J. Innov. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/TLT.2023.3314610","article-title":"Data-Driven Artificial Intelligence in Education: A Comprehensive Review","volume":"17","author":"Ahmad","year":"2024","journal-title":"IEEE Trans. Learn. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1921","DOI":"10.1002\/sd.3221","article-title":"AI-Driven Adaptive Learning for Sustainable Educational Transformation","volume":"33","author":"Strielkowski","year":"2024","journal-title":"Sustain. Dev."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Thompson, K., Farr, A.C., Saunders, T., and Winter, G. (2023). The Role of Adaptive Learning Technologies and Conditional Learning. Adaptive Learning Technologies, Springer.","DOI":"10.1007\/978-981-99-4170-4_26"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"124","DOI":"10.25082\/AMLER.2021.02.007","article-title":"Adaptive E-Learning Systems through Learning Styles: A Review of the Literature","volume":"1","author":"Katsaris","year":"2021","journal-title":"Adv. Mob. Learn. Educ. Res."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1186\/s41239-021-00289-4","article-title":"Adaptive E-Learning Environment Based on Learning Styles and Its Impact on Development Students\u2019 Engagement","volume":"18","year":"2021","journal-title":"Int. J. Educ. Technol. High. Educ."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"012033","DOI":"10.1088\/1742-6596\/1235\/1\/012033","article-title":"Adaptive User Interface of Learning Management Systems for Education 4.0: A Research Perspective","volume":"1235","author":"Bagustari","year":"2019","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Bahroun, Z., Anane, C., Ahmed, V., and Zacca, A. (2023). Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis. Sustainability, 15.","DOI":"10.3390\/su151712983"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Barker, L.A., Moore, J.D., and Cook, H.A. (2024). Generative Artificial Intelligence as a Tool for Teaching Communication in Nutrition and Dietetics Education: A Novel Education Innovation. Nutrients, 16.","DOI":"10.3390\/nu16070914"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1186\/s41239-019-0171-0","article-title":"Systematic Review of Research on Artificial Intelligence Applications in Higher Education: Where Are the Educators?","volume":"16","author":"Bond","year":"2019","journal-title":"Int. J. Educ. Technol. High. Educ."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/s41039-017-0062-8","article-title":"Exploring the Impact of Artificial Intelligence on Teaching and Learning in Higher Education","volume":"12","author":"Popenici","year":"2017","journal-title":"Res. Pract. Technol. Enhanc. Learn."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1186\/s40561-024-00350-5","article-title":"A Systematic Review of Literature Reviews on Artificial Intelligence in Education (AIED): A Roadmap to a Future Research Agenda","volume":"11","author":"Mustafa","year":"2024","journal-title":"Smart Learn. Environ."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Meng, B., and Qin, X. (2024). Construction of an Education Innovation Network Management System under Artificial Intelligence Technology. Proceedings of the International Conference on Computer and Multimedia Technology, Sanming, China, 24\u201326 May 2024, Association for Computing Machinery.","DOI":"10.1145\/3675249.3675280"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"George, B., and Wooden, O. (2023). Managing the Strategic Transformation of Higher Education through Artificial Intelligence. Adm. Sci., 13.","DOI":"10.3390\/admsci13090196"},{"key":"ref_22","first-page":"291","article-title":"Some Experimental Results in the Correlation of Mental Abilities","volume":"3","author":"Brown","year":"1910","journal-title":"Br. J. Psychol."},{"key":"ref_23","unstructured":"Ebel, R.L., and Frisbie, D.A. (1986). Essentials of Educational Measurement, Prentice-Hall. [4th ed.]."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"98699","DOI":"10.1109\/ACCESS.2025.3576147","article-title":"Gamification and Artificial Intelligence in Education: A Review of Innovative Approaches to Fostering Critical Thinking","volume":"13","author":"Kassenkhan","year":"2025","journal-title":"IEEE Access"},{"key":"ref_25","first-page":"34","article-title":"Behavioral Authentication Model for Smartphone Users Based on Gyroscope and Accelerometer Data for Mobile Educational Resources","volume":"17","author":"Serbin","year":"2026","journal-title":"J. Adv. Inf. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1108\/DLO-03-2013-0008","article-title":"Considering Mobile Learning? A Case Study from Jaguar Land Rover","volume":"27","author":"Wigley","year":"2013","journal-title":"Dev. Learn. Organ."},{"key":"ref_27","first-page":"83","article-title":"Learning in Hybrid Spaces: Designing a Mobile Technology Capacity Building Framework for Workplace Learning","volume":"Volume 32","author":"Trede","year":"2017","journal-title":"Work-Integrated Learning in the 21st Century; International Perspectives on Education and Society"},{"key":"ref_28","first-page":"275","article-title":"Exposing Policy Gaps: The Experience of Kazakhstan in Implementing Distance Learning during the COVID-19 Pandemic","volume":"15","author":"Bokayev","year":"2021","journal-title":"Transform. Gov. People Process Policy"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"12","DOI":"10.51889\/2021-4.2077-6861.02","article-title":"Online Education during the COVID-19 Pandemic: Academic Analytics in the Kazakh Context","volume":"49","author":"Abilkhairova","year":"2021","journal-title":"Pedagog. Psychol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ibrayeva, A., and Yegemberdiyeva, S. (2022). Assessment of Digital Transformation in the Education System of Kazakhstan. Econ. Ser. Bull. L.N. Gumilyov ENU, 152\u2013160.","DOI":"10.32523\/2789-4320-2022-4-152-160"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Thornhill-Miller, B., Camarda, A., Mercier, M., Burkhardt, J.M., Morisseau, T., Bourgeois-Bougrine, S., Vinchon, F., El Hayek, S., Augereau-Landais, M., and Mourey, F. (2023). Creativity, Critical Thinking, Communication, and Collaboration: Assessment, Certification, and Promotion of 21st Century Skills for the Future of Work and Education. J. Intell., 11.","DOI":"10.3390\/jintelligence11030054"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"101578","DOI":"10.1016\/j.tsc.2024.101578","article-title":"The Mediating Effects of Critical Thinking on the Motivation and Creativity of Business English Learners in the Age of AI: Cognitive Flexibility Theory","volume":"53","author":"Hao","year":"2024","journal-title":"Think. Skills Creat."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"485","DOI":"10.48161\/qaj.v4n3a919","article-title":"Evaluation of Problem Based Gamification Learning (PBGL) Model on Critical Thinking Ability with Artificial Intelligence Approach Integrated with ChatGPT API: An Experimental Study","volume":"4","author":"Naatonis","year":"2024","journal-title":"Qubahan Acad. J."},{"key":"ref_34","first-page":"7","article-title":"Integrating Gamification into Curriculum Design: Enhancing Adolescent Students\u2019 Decision Making, Problem Solving, and Critical Thinking Capacities","volume":"2","author":"Nazneen","year":"2024","journal-title":"MLAC J. Arts Commer. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1002\/cae.22227","article-title":"Adapting Gamified Learning Systems Using Educational Data Mining Techniques","volume":"28","author":"Daghestani","year":"2020","journal-title":"Comput. Appl. Eng. Educ."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"102100","DOI":"10.1016\/j.lmot.2025.102100","article-title":"Uncurtaining Windows of Motivation, Enjoyment, Critical Thinking, and Autonomy in AI-Integrated Education: Duolingo vs. ChatGPT","volume":"89","author":"Xu","year":"2025","journal-title":"Learn. Motiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Roopaei, M., and Roopaei, R. (2024). Gamifying AI Education for Young Minds: The TransAI Adventure in Learning. Proceedings of the 5th IEEE Annual World AI IoT Congress (AIIoT), IEEE.","DOI":"10.1109\/AIIoT61789.2024.10578991"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"104105","DOI":"10.1016\/j.im.2025.104105","article-title":"Integrating Technology in Physical Classrooms: The Impact of Game-Based Response Systems on Student Learning Experience","volume":"62","author":"Priante","year":"2025","journal-title":"Inf. Manag."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1186\/s40561-024-00316-7","article-title":"The Effects of Over-Reliance on AI Dialogue Systems on Students\u2019 Cognitive Abilities: A Systematic Review","volume":"11","author":"Zhai","year":"2024","journal-title":"Smart Learn. Environ."},{"key":"ref_40","first-page":"533","article-title":"Critical Thinking in the Management Classroom: Bloom\u2019s Taxonomy as a Learning Tool","volume":"27","author":"Athanassiou","year":"2003","journal-title":"J. Manag. Educ."},{"key":"ref_41","first-page":"020","article-title":"Development of a Theoretical Framework of MOOCs with Gamification Elements to Enhance Students\u2019 Higher-Order Thinking Skills: A Critical Review of the Literature","volume":"23","author":"Hao","year":"2024","journal-title":"J. Inf. Technol. Educ. Res."},{"key":"ref_42","first-page":"11","article-title":"Gamification Resources in Education: A Theoretical Approach","volume":"25","author":"Astashova","year":"2023","journal-title":"Educ. Sci. J."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Gerlich, M. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies, 15.","DOI":"10.2139\/ssrn.5082524"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Alam, A. (2021). Possibilities and Apprehensions in the Landscape of Artificial Intelligence in Education. Proceedings of the International Conference on Computational Intelligence and Computing Applications (ICCICA), IEEE.","DOI":"10.1109\/ICCICA52458.2021.9697272"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1016\/j.vrih.2022.08.001","article-title":"Gamification and Virtual Reality for Digital Twin Learning and Training: Architecture and Challenges","volume":"4","author":"Bucchiarone","year":"2022","journal-title":"Virtual Real. Intell. Hardw."},{"key":"ref_46","first-page":"101994","article-title":"Artificial Intelligence (AI): Multidisciplinary Perspectives on Emerging Challenges, Opportunities, and Agenda for Research, Practice and Policy","volume":"57","author":"Dwivedi","year":"2021","journal-title":"Int. J. Inf. Manag."},{"key":"ref_47","first-page":"44","article-title":"DARPA\u2019s Explainable Artificial Intelligence (XAI) Program","volume":"40","author":"Gunning","year":"2019","journal-title":"AI Mag."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/8\/5\/120\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T12:45:49Z","timestamp":1779194749000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/8\/5\/120"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,1]]},"references-count":47,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,5]]}},"alternative-id":["make8050120"],"URL":"https:\/\/doi.org\/10.3390\/make8050120","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,1]]}}}