{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,28]],"date-time":"2026-05-28T00:43:18Z","timestamp":1779928998769,"version":"3.53.1"},"reference-count":71,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2024,7,16]],"date-time":"2024-07-16T00:00:00Z","timestamp":1721088000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Intelligenza Artificiale"],"published-print":{"date-parts":[[2024,7,31]]},"abstract":"<jats:p>\u00a0Nowadays, there is growing attention on enhancing the quality of teaching, learning and assessment processes. As a recent EU Report underlines, the assessment and feedback area remains a problematic issue regarding educational professionals training and adopting new practices. In fact, traditional summative assessment practices are predominantly used in European countries, against the recommendations of the Bologna Process guidelines that promote the implementation of alternative assessment practices that seem crucial in order to engage and provide lifelong learning skills for students, also with the use of technology. Looking at the literature, a series of sustainability problems arise when these requests meet real-world teaching, particularly when academic instructors face the assessment of extensive classes. With the fast advancement in Large Language Models (LLMs) and their increasing availability, affordability and capability, part of the solution to these problems might be at hand. In fact, LLMs can process large amounts of text, summarise and give feedback about it following predetermined criteria. The insights of that analysis can be used both for giving feedback to the student and helping the instructor assess the text. With the proper pedagogical and technological framework, LLMs can disengage instructors from some of the time-related sustainability issues and so from the only choice of the multiple-choice test and similar. For this reason, as a first step, we are designing and validating a theoretical framework and a teaching model for fostering the use of LLMs in assessment practice, with the approaches that can be most beneficial.<\/jats:p>","DOI":"10.3233\/ia-240033","type":"journal-article","created":{"date-parts":[[2024,7,19]],"date-time":"2024-07-19T11:46:05Z","timestamp":1721389565000},"page":"121-138","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":18,"title":["Large language models for sustainable assessment and feedback in higher education"],"prefix":"10.1177","volume":"18","author":[{"given":"Daniele","family":"Agostini","sequence":"first","affiliation":[{"name":"University of Trento","place":["Italy"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Federica","family":"Picasso","sequence":"additional","affiliation":[{"name":"University of Trento","place":["Italy"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2024,7,16]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"38","article-title":"Are large language models capable of assessing students\u2019 written products?","volume":"11","author":"Agostini D.","year":"2024","unstructured":"AgostiniD., Are large language models capable of assessing students\u2019 written products?Research Trends in Humanities Education & Philosophy11 (2024), 38\u201360.","journal-title":"Research Trends in Humanities Education & Philosophy"},{"key":"e_1_3_1_3_2","unstructured":"AshbyW.R. An introduction to cybernetics Chapman & Hall London 1956."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.mex.2021.101401"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1080\/713695728"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2024.1221177"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641289"},{"key":"e_1_3_1_8_2","article-title":"How is ChatGPT\u2019s behavior changing over time?","author":"Chen L.","year":"2023","unstructured":"ChenL., ZahariaM.andZouJ., How is ChatGPT\u2019s behavior changing over time?arXiv preprint arXiv:2307.09009 (2023).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_9_2","article-title":"How to conduct a Delphi study","author":"Chuenjitwongsa S.","year":"2017","unstructured":"ChuenjitwongsaS., How to conduct a Delphi study, Medical Education (2017).","journal-title":"Medical Education"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12124-022-09703-6"},{"key":"e_1_3_1_11_2","unstructured":"Cook-SatherA. BovillC.andFeltenP. Engaging students as partners in learning and teaching: A guide for faculty John Wiley & Sons Hoboken NJ 2014."},{"key":"e_1_3_1_12_2","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin J.","year":"2018","unstructured":"DevlinJ., ChangM.W., LeeK.andToutanovaK., BERT: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_13_2","article-title":"Augmenting human intellect: A conceptual framework","author":"Englebart D.C.","year":"1962","unstructured":"EnglebartD.C., Augmenting human intellect: A conceptual framework, SRI Summary Report AFOSR-3223 (1962).","journal-title":"SRI Summary Report"},{"key":"e_1_3_1_14_2","first-page":"3","volume-title":"Theories and practice of interaction design","author":"Engestr\u00f6m Y.","year":"2006","unstructured":"Engestr\u00f6mY., Activity theory and expansive design, in: Theories and practice of interaction design, S.Bagnara and G.Cramton Smith, eds., Lawrence Erlbaum Associates , Mahwah, NJ, 2006, pp. 3\u201323."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.techfore.2011.04.006"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.3390\/app11125467"},{"key":"e_1_3_1_17_2","unstructured":"GrionV.andSerbatiA. Valutazione sostenibile e feedback nei contesti universitari. Prospettive emergenti ricerche e pratiche PensaMultimedia Lecce 2019."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF02504676"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1080\/0729436980170304"},{"key":"e_1_3_1_20_2","first-page":"30016","volume-title":"Advances in Neural Information Processing Systems","author":"Hoffmann J.","year":"2022","unstructured":"HoffmannJ., BorgeaudS., MenschA., BuchatskayaE., CaiT., RutherfordE.andSifreL., An empirical analysis of compute-optimallarge language model training, in: Advances in Neural Information Processing Systems, vol. 35, Curran Associates, Inc., Red Hook, NY, 2022, pp 30016\u201330030."},{"key":"e_1_3_1_21_2","unstructured":"HolmesW. BialikM.andFadelC. Artificial intelligence in Education: Promises and implications for teaching & learning The Center for Curriculum Redesign Boston MA 2019."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1111\/ejed.12533"},{"key":"e_1_3_1_23_2","article-title":"Can large language models identify authorship?","author":"Huang B.","year":"2024","unstructured":"HuangB., ChenC.andShuK., Can large language models identify authorship?arXiv preprint , arXiv:2403.08213 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23817"},{"key":"e_1_3_1_25_2","unstructured":"JISC Assessment ideas for an AI-enabled world 2023. https:\/\/repository.jisc.ac.uk\/9234\/1\/assessment-ideas-foran-ai-enabled-word.pptx."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF02818172"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.3390\/su151612451"},{"key":"e_1_3_1_28_2","article-title":"Scaling laws forneural language models","author":"Kaplan J.","year":"2020","unstructured":"KaplanJ., McCandlishS., HenighanT., BrownT.B., ChessB., ChildR., GrayS., RadfordA., WuJ.andAmodeiD., Scaling laws forneural language models, arXiv preprint, arXiv:2001.08361 (2020).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_29_2","first-page":"2006","volume-title":"Acting with Technology: Activity Theory and interaction Design","author":"Kaptelinin V.","unstructured":"KaptelininV.andNardiB., Acting with Technology: Activity Theory and interaction Design, MIT Press, Cambridge, MA, 2006."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.lindif.2023.102274"},{"key":"e_1_3_1_31_2","first-page":"61","volume-title":"The Cambridge handbook of the learning sciences","author":"Koedinger K.R.","year":"2006","unstructured":"KoedingerK.R.andCorbettA., Cognitive tutors: Technology bringing learning sciences to the classroom, in: The Cambridge handbook of the learning sciences, R.K.Sawyer, ed., Cambridge University Press, New York, NY, 2006, pp. 61\u201377."},{"issue":"1","key":"e_1_3_1_32_2","article-title":"Teaching English in the age of AI: Embracing ChatGPT to optimize EFL materials and assessment","volume":"3","author":"Koraishi O.","year":"2023","unstructured":"KoraishiO., Teaching English in the age of AI: Embracing ChatGPT to optimize EFL materials and assessment, Language Education and Technology3(1) (2023).","journal-title":"Language Education and Technology"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40547-024-00143-4"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1111\/bjet.13424"},{"issue":"4","key":"e_1_3_1_35_2","first-page":"34","article-title":"How human-computer \u2019superminds\u2019 are redefining the future of work","volume":"59","author":"Malone T.W.","year":"2018","unstructured":"MaloneT.W., How human-computer \u2019superminds\u2019 are redefining the future of work, MIT Sloan Management Review ,59(4) (2018), 34\u201341.","journal-title":"MIT Sloan Management Review"},{"key":"e_1_3_1_36_2","first-page":"2018","volume-title":"Superminds: The surprising power of people and computers thinking together","author":"Malone T.W.","unstructured":"MaloneT.W., Superminds: The surprising power of people and computers thinking together, Little, Brown Spark, New York, NY, 2018."},{"key":"e_1_3_1_37_2","first-page":"1","article-title":"Exploring new depths: Applying machine learning for the analysis of student argumentation in chemistry","author":"Martin P.P.","year":"2023","unstructured":"MartinP.P., KranzD., WulffP.andGraulichN., Exploring new depths: Applying machine learning for the analysis of student argumentation in chemistry, Journal of Research in Science Teaching (2023), 1\u201336.","journal-title":"Journal of Research in Science Teaching"},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","unstructured":"MiaoF.andHolmesW. Guidance for generative AI in education and research 2023. https:\/\/doi.org\/10.54675\/EWZM9535.","DOI":"10.54675\/EWZM9535"},{"key":"e_1_3_1_39_2","first-page":"2021","volume-title":"AI and education: A guidance for policymakers","author":"Miao F.","unstructured":"MiaoF., HolmesW., HuangR.andZhangH., AI and education: A guidance for policymakers, UNESCO Publishing, Paris, 2021."},{"key":"e_1_3_1_40_2","doi-asserted-by":"crossref","unstructured":"MollickE.R.andMollickL. Assigning AI: Seven approaches for students with prompts 2023. https:\/\/dx.doi.org\/10.2139\/ssrn.4475995.","DOI":"10.2139\/ssrn.4475995"},{"key":"e_1_3_1_41_2","first-page":"2021","author":"Monarch R.M.","unstructured":"MonarchR.M., Human-in-the-Loop Machine Learning: Active Learning and Annotation for Human-centered AI, Simon and Schuster, New York, NY, 2021.","journal-title":"Human-in-the-Loop Machine Learning: Active Learning and Annotation for Human-centered AI"},{"issue":"3","key":"e_1_3_1_42_2","article-title":"\u201cKeeping it Real\u201d: A review of the benefits, challenges and steps towards implementing authentic assessment","volume":"9","author":"Murphy V.","year":"2017","unstructured":"MurphyV., FoxJ., FreemanS.andHughesN.\u201cKeeping it Real\u201d: A review of the benefits, challenges and steps towards implementing authentic assessment, All Ireland Journal of Higher Education9(3) (2017).","journal-title":"All Ireland Journal of Higher Education"},{"key":"e_1_3_1_43_2","author":"Nerantzi C.","year":"2023","unstructured":"NerantziC., AbegglenS., KaratsioriM.andMart\u0131nez-ArboledaA.(Eds.), 101 creative ideas to use AI in education, A crowdsourced collection, 2023. https:\/\/doi.org\/10.5281\/zenodo.8355454.","journal-title":"101 creative ideas to use AI in education, A crowdsourced collection"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1080\/03075070600572090"},{"key":"e_1_3_1_45_2","first-page":"34","volume-title":"Feedback in higher and professional education","author":"Nicol D.","year":"2012","unstructured":"NicolD., Resituating feedback from the reactive to the proactive, in: Feedback in higher and professional education, D.Boud and E.Molloy, eds., Routledge, London, 2012, pp. 34\u201349."},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1080\/02602938.2022.2089627"},{"key":"e_1_3_1_47_2","unstructured":"OpenAI Teaching with AI 2023. https:\/\/openai.com\/blog\/teaching-with-ai."},{"key":"e_1_3_1_48_2","first-page":"1991","volume-title":"Constructionism","author":"Papert S.","unstructured":"PapertS.andHarelI., Situating constructionism, in Constructionism, S.Papert and I.Harel, eds., Ablex Publishing Corporation, Norwood, NJ, 1991."},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-016-0110-3"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1080\/19488289.2018.1424059"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-77584-1_15"},{"issue":"1","key":"e_1_3_1_52_2","first-page":"77","article-title":"Formative assessment: Revisiting the territory","volume":"5","author":"Sadler D.R.","year":"1989","unstructured":"SadlerD.R., Formative assessment: Revisiting the territory, Assessment in Education ,5(1) (1989), 77\u201384.","journal-title":"Assessment in Education"},{"key":"e_1_3_1_53_2","first-page":"2013","volume-title":"Assessment for learning in higher education","author":"Sambell K.","unstructured":"SambellK., McDowellL.andMontgomeryC., Assessment for learning in higher education, Routledge, London, 2013."},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0191-491X(97)86215-3"},{"key":"e_1_3_1_55_2","article-title":"Improving the validity of automatically generated feedback via reinforcement learning","author":"Scarlatos A.","year":"2024","unstructured":"ScarlatosA., SmithD., WoodheadS.andLanA., Improving the validity of automatically generated feedback via reinforcement learning, arXiv preprint arXiv:2403.01304 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_56_2","article-title":"Sentiment analysis through LLM negotiations","author":"Sun X.","year":"2023","unstructured":"SunX., LiX., ZhangS., WangS., WuF., LiJ.andWangG., Sentiment analysis through LLM negotiations, arXiv preprint arXiv:2311.01876 (2023).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.caeai.2022.100075"},{"key":"e_1_3_1_58_2","article-title":"Understanding the capabilities, limitations and societal impact of large language models","author":"Tamkin A.","year":"2021","unstructured":"TamkinA., BrundageM., ClarkJ.andGanguliD., Understanding the capabilities, limitations and societal impact of large language models, arXiv preprint arXiv:2102.02503 (2021).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_59_2","first-page":"2018","volume-title":"The impact of artificial intelligence on learning, teaching and education","author":"Tuomi I.","unstructured":"TuomiI., The impact of artificial intelligence on learning, teaching and education, Publications Office of the European Union, Luxembourg, 2018."},{"key":"e_1_3_1_60_2","unstructured":"UK Department for Education Generative artificial intelligence (AI) in education 2023. Retrieved from https:\/\/www.gov.uk\/government\/publications\/generativeartificial-intelligence-ai-in-education\/generative-artificialintelligence-ai-in-education."},{"key":"e_1_3_1_61_2","unstructured":"UNICEF Policy guidance on AI for children 2021. https:\/\/www.unicef.org\/globalinsight\/media\/2356\/file\/UNICEFGlobal-Insight-policy-guidance-AI-children-2.0-2021.pdf."},{"key":"e_1_3_1_62_2","first-page":"2023","volume-title":"Artificial intelligence and future of teaching and learning: Insights and recommendations","author":"U.S. Department of Education","unstructured":"U.S. Department of Education, Office of Educational Technology, Artificial intelligence and future of teaching and learning: Insights and recommendations, Washington, DC, 2023."},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.3233\/IRG-2006-16(3)02"},{"key":"e_1_3_1_64_2","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani A.","year":"2017","unstructured":"VaswaniA., ShazeerN., ParmarN., UszkoreitJ., JonesL., GomezA.N., KaiserL.andPolosukhinI., Attention is all you need, in: Advances in Neural Information Processing Systems, vol. 30, Curran Associates, Inc., Red Hook, NY, 2017, 5998\u20136008."},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1080\/02602938.2017.1412396"},{"key":"e_1_3_1_66_2","first-page":"1978","volume-title":"Mind in Society: Development of HigherPsychological Processes","author":"Vygotsky L.S.","unstructured":"VygotskyL.S., Mind in Society: Development of HigherPsychological Processes, Harvard University Press, Cambridge, MA, 1978."},{"key":"e_1_3_1_67_2","unstructured":"WebbM. A Generative AI primer JISC 2023. https:\/\/nationalcentreforai.jiscinvolve.org\/wp\/2023\/05\/11\/generativeai-primer\/#3-1."},{"issue":"1","key":"e_1_3_1_68_2","first-page":"92","article-title":"Student-centered learning in higher education","volume":"23","author":"Wright G.B.","year":"2011","unstructured":"WrightG.B., Student-centered learning in higher education, International Journal of Teaching and Learning in Higher Education23(1) (2011), 92\u201397.","journal-title":"International Journal of Teaching and Learning in Higher Education"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2022.05.014"},{"key":"e_1_3_1_70_2","article-title":"Reasoning before comparison: LLM-enhanced semantic similarity metrics for domain specialized text analysis","author":"Xu S.","year":"2024","unstructured":"XuS., WuZ., ZhaoH., ShuP., LiuZ., LiaoW.andLiX., Reasoning before comparison: LLM-enhanced semantic similarity metrics for domain specialized text analysis, arXiv preprint arXiv:2402.11398 (2024).","journal-title":"arXiv preprint"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1111\/bjet.13370"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1631\/FITEE.1700053"}],"container-title":["Intelligenza Artificiale"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/IA-240033","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/IA-240033","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/IA-240033","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:51:52Z","timestamp":1777459912000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/IA-240033"}},"subtitle":["Towards a Pedagogical and Technological Framework"],"short-title":[],"issued":{"date-parts":[[2024,7,16]]},"references-count":71,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,7,31]]}},"alternative-id":["10.3233\/IA-240033"],"URL":"https:\/\/doi.org\/10.3233\/ia-240033","relation":{},"ISSN":["1724-8035","2211-0097"],"issn-type":[{"value":"1724-8035","type":"print"},{"value":"2211-0097","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,16]]}}}