{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T19:56:53Z","timestamp":1777406213143,"version":"3.51.4"},"reference-count":30,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T00:00:00Z","timestamp":1769558400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>This study synthesizes empirical evidence on AI-supported skill assessment systems in higher vocational education through a systematic review and meta-analysis. Despite growing interest in generative AI within higher education, empirical research on AI-enabled assessment remains fragmented and methodologically uneven, particularly in vocational contexts. Following PRISMA 2020 guidelines, 27 peer-reviewed empirical studies published between 2010 and 2024 were identified from major international and Chinese databases and included in the analysis. Using a random-effects model, the meta-analysis indicates a moderate positive association between AI-supported assessment systems and skill-related learning outcomes (Hedges\u2019 g = 0.72), alongside substantial heterogeneity across study designs, outcome measures, and implementation contexts. Subgroup analyses suggest variation across regional and institutional settings, which should be interpreted cautiously given small sample sizes and diverse methodological approaches. Based on the synthesized evidence, the study proposes a conceptual AI-supported skill assessment framework that distinguishes empirically grounded components from forward-looking extensions related to generative AI. Rather than offering prescriptive solutions, the framework provides an evidence-informed baseline to support future research, system design, and responsible integration of generative AI in higher education assessment. Overall, the findings highlight both the potential and the current empirical limitations of AI-enabled assessment, underscoring the need for more robust, theory-informed, and transparent studies as generative AI applications continue to evolve.<\/jats:p>","DOI":"10.3390\/informatics13020020","type":"journal-article","created":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T15:04:46Z","timestamp":1769612686000},"page":"20","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["AI-Enhanced Skill Assessment in Higher Vocational Education: A Systematic Review and Meta-Analysis"],"prefix":"10.3390","volume":"13","author":[{"given":"Xia","family":"Sun","sequence":"first","affiliation":[{"name":"School of Quality Education, Yibin Vocational and Technical College, Yibin 644003, China"}]},{"given":"Haoheng","family":"Tian","sequence":"additional","affiliation":[{"name":"School of Quality Education, Yibin Vocational and Technical College, Yibin 644003, China"}]}],"member":"1968","published-online":{"date-parts":[[2026,1,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Boud, D., and Falchikov, N. (2007). Rethinking Assessment in Higher Education: Learning for the Longer Term, Routledge.","DOI":"10.4324\/9780203964309"},{"key":"ref_2","first-page":"1","article-title":"Towards an understanding of artificial intelligence in education: A systematic review","volume":"24","author":"Zhai","year":"2021","journal-title":"Educ. Technol. Soc."},{"key":"ref_3","unstructured":"Ministry of Education of the People\u2019s Republic of China (2020). Implementation Plan for the National Vocational Education Reform."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"n71","DOI":"10.1136\/bmj.n71","article-title":"The PRISMA 2020 statement: An updated guideline for reporting systematic reviews","volume":"372","author":"Page","year":"2021","journal-title":"BMJ"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Borenstein, M., Hedges, L.V., Higgins, J.P.T., and Rothstein, H.R. (2009). Introduction to Meta-Analysis, John Wiley & Sons.","DOI":"10.1002\/9780470743386"},{"key":"ref_7","unstructured":"Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., and Welch, V.A. (2022). Introduction to systematic reviews. Cochrane Handbook for Systematic Reviews of Interventions, John Wiley & Sons. [2nd ed.]."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1186\/s40594-019-0198-8","article-title":"Five years of development in pursuing excellence in quality and global impact to become the first journal in STEM education covered in SSCI","volume":"6","author":"Li","year":"2019","journal-title":"Int. J. STEM Educ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/21582440221094821","article-title":"Synchronous online learning during COVID-19: Chinese university EFL students\u2019 perspectives","volume":"12","author":"Zhang","year":"2022","journal-title":"SAGE Open"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"5","DOI":"10.11648\/j.tecs.20210601.12","article-title":"A Novice Japanese Teacher\u2019s Identity Construction in Online Teaching Under COVID-19: Beliefs and Perceptions","volume":"6","author":"Chen","year":"2021","journal-title":"Teach. Educ. Curric. Stud."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wang, R., Cao, J., Xu, Y., and Li, Y. (2022). Learning engagement in massive open online courses: A systematic review. Front. Educ., 7.","DOI":"10.3389\/feduc.2022.1074435"},{"key":"ref_12","first-page":"100215","article-title":"Human-centred learning analytics and AI in education: A systematic literature review","volume":"10","author":"Alfredo","year":"2023","journal-title":"J. Learn. Anal."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"7","DOI":"10.54097\/adcxze34","article-title":"Fairness and effectiveness in AI-driven educational assessments: Challenges and mitigation strategies","volume":"11","author":"Zhang","year":"2025","journal-title":"J. Innov. Dev."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zidoun, Y., and El Mardi, A. (2024). Artificial intelligence (AI)-based simulators versus simulated patients in undergraduate programs: A protocol for a randomized controlled trial. BMC Med. Educ., 24.","DOI":"10.1186\/s12909-024-06236-x"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"100005","DOI":"10.1016\/j.caeai.2020.100005","article-title":"A multi-perspective study on artificial intelligence in education: Grants, conferences, journals, software tools, institutions, and researchers","volume":"1","author":"Chen","year":"2020","journal-title":"Comput. Educ. Artif. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Messer, M., Brown, N.C.C., K\u00f6lling, M., and Shi, M. (2023). Automated grading and feedback tools for programming education: A systematic review. arXiv, Available online: https:\/\/arxiv.org\/abs\/2306.11722.","DOI":"10.35542\/osf.io\/wpsgk"},{"key":"ref_17","first-page":"104875","article-title":"Intelligent assessment frameworks for vocational education in the era of artificial intelligence","volume":"205","author":"Zhang","year":"2025","journal-title":"Comput. Educ."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"52","DOI":"10.18608\/jla.2023.8093","article-title":"A systematic review of learning analytics-incorporated instructional interventions on learning management systems","volume":"11","author":"Pan","year":"2024","journal-title":"J. Learn. Anal."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sciarrone, F. (2018, January 26\u201328). Machine Learning and Learning Analytics: Integrating Data with Learning. Proceedings of the 2018 17th International Conference on Information Technology Based Higher Education and Training (ITHET), Olhao, Portugal.","DOI":"10.1109\/ITHET.2018.8424780"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"590","DOI":"10.18178\/ijlt.10.5.590-596","article-title":"Opportunities and challenges of artificial intelligence in vocational education","volume":"10","author":"Jiang","year":"2024","journal-title":"Int. J. Learn. Teach."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1258","DOI":"10.1016\/j.aej.2025.08.029","article-title":"Machine learning optimization for vocational literacy education evaluation: A big data-powered decision support system","volume":"129","author":"Li","year":"2025","journal-title":"Alex. Eng. J."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"937","DOI":"10.1007\/s10648-020-09586-2","article-title":"The Cognitive Affective Model of Immersive Learning (CAMIL): A Theoretical Research-Based Model of Learning in Immersive Virtual Reality","volume":"33","author":"Makransky","year":"2021","journal-title":"Educ. Psychol. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Emerson, A., Geden, M., Smith, A., Wiebe, E., Mott, B., Boyer, K.E., and Lester, J. (2020, January 13\u201317). Predictive Student Modeling in Block-Based Programming Environments with Bayesian Hierarchical Models. Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (UMAP 2020), Genoa, Italy.","DOI":"10.1145\/3340631.3394853"},{"key":"ref_24","first-page":"25","article-title":"Supporting Self-Regulated Learning and Academic Achievement with Personalized Peer-Comparison Feedback in Higher Education","volume":"10","author":"Fleur","year":"2023","journal-title":"J. Learn. Anal."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"100153","DOI":"10.1016\/j.caeai.2023.100153","article-title":"Using chatbots for English language learning in higher education","volume":"5","author":"Annamalai","year":"2024","journal-title":"Comput. Educ. Artif. Intell."},{"key":"ref_26","first-page":"1","article-title":"Artificial intelligence applications in competency-based assessment: A systematic review","volume":"22","year":"2025","journal-title":"Int. J. Educ. Technol. High. Educ."},{"key":"ref_27","first-page":"1","article-title":"Mapping research in learning analytics and artificial intelligence in education: A systematic review","volume":"27","author":"Bond","year":"2024","journal-title":"Educ. Technol. Soc."},{"key":"ref_28","unstructured":"(2026, January 05). Project. Report on the Perception of Teachers About the Future Impact of AI. Available online: https:\/\/www.paideiaproject.eu\/wp-content\/uploads\/2025\/06\/Report-on-the-perception-of-teachers-about-the-future-impact-of-AI.pdf."},{"key":"ref_29","unstructured":"Luckin, R., Holmes, W., Griffiths, M., and Forcier, L.B. (2016). Intelligence Unleashed: An Argument for AI in Education, Pearson."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/jcal.12172","article-title":"Review of computer-based assessment for learning in elementary and secondary education","volume":"33","author":"Shute","year":"2017","journal-title":"J. Comput. Assist. Learn."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/13\/2\/20\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T15:36:42Z","timestamp":1769614602000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/13\/2\/20"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,28]]},"references-count":30,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["informatics13020020"],"URL":"https:\/\/doi.org\/10.3390\/informatics13020020","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,28]]}}}