{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T20:09:47Z","timestamp":1776110987703,"version":"3.50.1"},"reference-count":35,"publisher":"Elsevier BV","issue":"2","license":[{"start":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T00:00:00Z","timestamp":1720396800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T00:00:00Z","timestamp":1720396800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000915","name":"Richard King Mellon Foundation","doi-asserted-by":"publisher","award":["10851"],"award-info":[{"award-number":["10851"]}],"id":[{"id":"10.13039\/100000915","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Artif Intell Educ"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    One-on-one tutoring is widely acknowledged as an effective instructional method, conditioned on qualified tutors. However, the high demand for qualified tutors remains a challenge, often necessitating the training of novice tutors (i.e., trainees) to ensure effective tutoring. Research suggests that providing timely explanatory feedback can facilitate the training process for trainees. However, it presents challenges due to the time-consuming nature of assessing trainee performance by human experts. Inspired by the recent advancements of large language models (LLMs), our study employed the GPT-4 model to build an explanatory feedback system. This system identifies trainees\u2019 responses in binary form (i.e., correct\/incorrect) and automatically provides template-based feedback with responses appropriately rephrased by the GPT-4 model. We conducted our study using the responses of 383 trainees from three training lessons (\n                    <jats:italic>Giving Effective Praise<\/jats:italic>\n                    ,\n                    <jats:italic>Reacting to Errors<\/jats:italic>\n                    , and\n                    <jats:italic>Determining What Students Know<\/jats:italic>\n                    ). Our findings indicate that: 1) using a few-shot approach, the GPT-4 model effectively identifies correct\/incorrect trainees\u2019 responses from three training lessons with an average F1 score of 0.84 and AUC score of 0.85; and 2) using the few-shot approach, the GPT-4 model adeptly rephrases incorrect trainees\u2019 responses into desired responses, achieving performance comparable to that of human experts.\n                  <\/jats:p>","DOI":"10.1007\/s40593-024-00408-y","type":"journal-article","created":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T13:02:46Z","timestamp":1720443766000},"page":"482-508","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["How Can I Get It Right? Using GPT to Rephrase Incorrect Trainee Responses"],"prefix":"10.1016","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3320-3907","authenticated-orcid":false,"given":"Jionghao","family":"Lin","sequence":"first","affiliation":[]},{"given":"Zifei","family":"Han","sequence":"additional","affiliation":[]},{"given":"Danielle R.","family":"Thomas","sequence":"additional","affiliation":[]},{"given":"Ashish","family":"Gurung","sequence":"additional","affiliation":[]},{"given":"Shivang","family":"Gupta","sequence":"additional","affiliation":[]},{"given":"Vincent","family":"Aleven","sequence":"additional","affiliation":[]},{"given":"Kenneth R.","family":"Koedinger","sequence":"additional","affiliation":[]}],"member":"78","published-online":{"date-parts":[[2024,7,8]]},"reference":[{"issue":"1","key":"408_CR1","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1177\/0013164409332231","volume":"70","author":"Y Attali","year":"2010","unstructured":"Attali, Y., & Powers, D. (2010). Immediate feedback and opportunity to revise answers to open-ended questions. Educational and Psychological Measurement, 70(1), 22\u201335.","journal-title":"Educational and Psychological Measurement"},{"key":"408_CR2","doi-asserted-by":"crossref","unstructured":"Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Gianinazzi, L., Gajda, J., Lehmann, T., Podstawski, M., Niewiadomski, H., Nyczyk, P., et al. (2023). Graph of thoughts: Solving elaborate problems with large language models. arXiv:2308.09687","DOI":"10.1609\/aaai.v38i16.29720"},{"issue":"2","key":"408_CR3","doi-asserted-by":"publisher","first-page":"290","DOI":"10.1037\/a0031026","volume":"105","author":"AC Butler","year":"2013","unstructured":"Butler, A. C., Godbole, N., & Marsh, E. J. (2013). Explanation feedback is better than correct answer feedback for promoting transfer of learning. Journal of Educational Psychology, 105(2), 290.","journal-title":"Journal of Educational Psychology"},{"key":"408_CR4","unstructured":"Cao, T., Law, M. T., & Fidler, S. (2019). A theoretical analysis of the number of shots in few-shot learning. In International Conference on Learning Representations"},{"issue":"4","key":"408_CR5","doi-asserted-by":"publisher","first-page":"13128","DOI":"10.1111\/cogs.13128","volume":"46","author":"PF Carvalho","year":"2022","unstructured":"Carvalho, P. F., & Goldstone, R. L. (2022). A computational model of context-dependent encodings during category learning. Cognitive Science, 46(4), 13128.","journal-title":"Cognitive Science"},{"key":"408_CR6","unstructured":"Chhabra, P., Chine, D., Adeniran, A., Gupta, S., & Koedinger, K. (2022). An evaluation of perceptions regarding mentor competencies for technology-based personalized learning. In Society for Information Technology & Teacher Education International Conference (pp. 1620\u20131625). Association for the Advancement of Computing in Education (AACE)"},{"key":"408_CR7","doi-asserted-by":"crossref","unstructured":"Dai, W., Lin, J., Jin, F., Li, T., Tsai, Y.-S., Gasevic, D., & Chen, G. (2023). Can large language models provide feedback to students? a case study on chatgpt","DOI":"10.35542\/osf.io\/hcgzj"},{"key":"408_CR8","doi-asserted-by":"crossref","unstructured":"Demszky, D., Liu, J., Hill, H. C., Jurafsky, D., & Piech, C. (2021). Can automated feedback improve teachers\u2019 uptake of student ideas? evidence from a randomized controlled trial in a large-scale online course. edworkingpaper (no. 21-483). Annenberg Institute for School Reform at Brown University.","DOI":"10.3102\/01623737231169270"},{"issue":"1","key":"408_CR9","doi-asserted-by":"publisher","first-page":"81","DOI":"10.3102\/003465430298487","volume":"77","author":"J Hattie","year":"2007","unstructured":"Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81\u2013112.","journal-title":"Review of Educational Research"},{"key":"408_CR10","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-25112-3","volume-title":"The Impact of Feedback in Higher Education: Improving Assessment Outcomes for Learners","author":"M Henderson","year":"2019","unstructured":"Henderson, M., Ajjawi, R., Boud, D., & Molloy, E. (2019). The Impact of Feedback in Higher Education: Improving Assessment Outcomes for Learners. Switzerland AG: Springer."},{"key":"408_CR11","unstructured":"Hirunyasiri, D., Thomas, D. R., Lin, J., Koedinger, K. R., & Aleven, V. (2023). Comparative analysis of gpt-4 and human graders in evaluating praise given to students in synthetic dialogues. arXiv:2307.02018"},{"issue":"13","key":"408_CR12","doi-asserted-by":"publisher","first-page":"2221311120","DOI":"10.1073\/pnas.2221311120","volume":"120","author":"KR Koedinger","year":"2023","unstructured":"Koedinger, K. R., Carvalho, P. F., Liu, R., & McLaughlin, E. A. (2023). An astonishing regularity in student learning rate. Proceedings of the National Academy of Sciences, 120(13), 2221311120.","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"408_CR13","doi-asserted-by":"publisher","first-page":"233285842110428","DOI":"10.1177\/23328584211042858","volume":"7","author":"MA Kraft","year":"2021","unstructured":"Kraft, M. A., & Falken, G. T. (2021). A blueprint for scaling tutoring and mentoring across public schools. AERA Open, 7, 23328584211042856.","journal-title":"AERA Open"},{"key":"408_CR14","unstructured":"Levonian, Z., Li, C., Zhu, W., Gade, A., Henkel, O., Postle, M.-E., & Xing, W. (2023). Retrieval-augmented generation to improve math question-answering: Trade-offs between groundedness and human preference. arXiv:2310.03184"},{"key":"408_CR15","doi-asserted-by":"crossref","unstructured":"Lin, J., Dai, W., Lim, L.-A., Tsai, Y.-S., Mello, R. F., Khosravi, H., Gasevic, D., & Chen, G. (2023). Learner-centred analytics of feedback content in higher education. In LAK23: 13th International Learning Analytics and Knowledge Conference (pp. 100\u2013110)","DOI":"10.1145\/3576050.3576064"},{"key":"408_CR16","doi-asserted-by":"crossref","unstructured":"Lin, J., Rakovic, M., Lang, D., Gasevic, D., & Chen, G. (2022). Exploring the politeness of instructional strategies from human-human online tutoring dialogues. In LAK22: 12th International Learning Analytics and Knowledge Conference (pp. 282\u2013293)","DOI":"10.1145\/3506860.3506904"},{"key":"408_CR17","doi-asserted-by":"crossref","unstructured":"Lin, J., Rakovi\u0107, M., Xie, H., Lang, D., Ga\u0161evi\u0107, D., Chen, G., & Li, Y. (2023). On the role of politeness in online human\u2013human tutoring. British Journal of Educational Technology","DOI":"10.1111\/bjet.13333"},{"key":"408_CR18","unstructured":"Lin, J., Thomas, D. R., Han, F., Gupta, S., Tan, W., Nguyen, N. D., & Koedinger, K. R. (2023). Using large language models to provide explanatory feedback to human tutors. arXiv:2306.15498"},{"key":"408_CR19","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1016\/j.future.2021.09.001","volume":"127","author":"J Lin","year":"2022","unstructured":"Lin, J., Singh, S., Sha, L., Tan, W., Lang, D., Ga\u0161evi\u0107, D., & Chen, G. (2022). Is it a good move? mining effective tutoring strategies from human-human tutorial dialogues. Future Generation Computer Systems, 127, 194\u2013207.","journal-title":"Future Generation Computer Systems"},{"key":"408_CR20","first-page":"100140","volume":"4","author":"Y Li","year":"2023","unstructured":"Li, Y., Sha, L., Yan, L., Lin, J., Rakovi\u0107, M., Galbraith, K., Lyons, K., Ga\u0161evi\u0107, D., & Chen, G. (2023). Can large language models write reflectively. Computers and Education: Artificial Intelligence, 4, 100140.","journal-title":"Computers and Education: Artificial Intelligence"},{"key":"408_CR21","doi-asserted-by":"crossref","unstructured":"MacNeil, S., Tran, A., Mogil, D., Bernstein, S., Ross, E., & Huang, Z. (2022). Generating diverse code explanations using the gpt-3 large language model. In Proceedings of the 2022 ACM Conference on International Computing Education Research-Volume 2 (pp. 37\u201339).","DOI":"10.1145\/3501709.3544280"},{"key":"408_CR22","unstructured":"McNichols, H., Feng, W., Lee, J., Scarlatos, A., Smith, D., Woodhead, S., & Lan, A. (2023). Exploring automated distractor and feedback generation for math multiple-choice questions via in-context learning. arXiv:2308.03234"},{"key":"408_CR23","doi-asserted-by":"publisher","DOI":"10.4135\/9781071802878","volume-title":"The Content Analysis Guidebook","author":"KA Neuendorf","year":"2017","unstructured":"Neuendorf, K. A. (2017). The Content Analysis Guidebook. Los Angeles, CA: SAGE Publications."},{"key":"408_CR24","doi-asserted-by":"crossref","unstructured":"Nickow, A., Oreopoulos, P., & Quan, V. (2020). The impressive effects of tutoring on prek-12 learning: A systematic review and meta-analysis of the experimental evidence.","DOI":"10.3386\/w27476"},{"key":"408_CR25","unstructured":"OpenAI (2023). GPT-4 Technical Report."},{"issue":"3","key":"408_CR26","doi-asserted-by":"publisher","first-page":"235","DOI":"10.18608\/jla.2018.53.15","volume":"5","author":"A Pardo","year":"2018","unstructured":"Pardo, A., Bartimote, K., Shum, S. B., Dawson, S., Gao, J., Ga\u0161evi\u0107, D., Leichtweis, S., Liu, D., Mart\u00ednez-Maldonado, R., Mirriahi, N., et al. (2018). Ontask: Delivering data-informed, personalized learning support actions. Journal of Learning Analytics, 5(3), 235\u2013249.","journal-title":"Journal of Learning Analytics"},{"key":"408_CR27","unstructured":"Penedo, G., Malartic, Q., Hesslow, D., Cojocaru, R., Cappelli, A., Alobeidli, H., Pannier, B., Almazrouei, E., & Launay, J. (2023). The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only."},{"key":"408_CR28","doi-asserted-by":"crossref","unstructured":"Pourpanah, F., Abdar, M., Luo, Y., Zhou, X., Wang, R., Lim, C. P., Wang, X.-Z., & Wu, Q. J. (2022). A review of generalized zero-shot learning methods. IEEE transactions on pattern analysis and machine intelligence.","DOI":"10.1109\/TPAMI.2022.3191696"},{"key":"408_CR29","doi-asserted-by":"crossref","unstructured":"Thomas, D., Yang, X., Gupta, S., Adeniran, A., Mclaughlin, E., & Koedinger, K. (2023). When the tutor becomes the student: Design and evaluation of efficient scenario-based lessons for tutors. In LAK23: 13th International Learning Analytics and Knowledge Conference (pp. 250\u2013261).","DOI":"10.1145\/3576050.3576089"},{"issue":"3","key":"408_CR30","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1080\/21532974.2019.1587727","volume":"35","author":"M Thompson","year":"2019","unstructured":"Thompson, M., Owho-Ovuakporie, K., Robinson, K., Kim, Y. J., Slama, R., & Reich, J. (2019). Teacher moments: A digital simulation for preservice teachers to approximate parent-teacher conversations. Journal of Digital Learning in Teacher Education, 35(3), 144\u2013164.","journal-title":"Journal of Digital Learning in Teacher Education"},{"issue":"1","key":"408_CR31","doi-asserted-by":"publisher","first-page":"81","DOI":"10.14434\/josotl.v22i1.31232","volume":"22","author":"J Torres","year":"2022","unstructured":"Torres, J. (2022). Feedback as open-ended conversation: Inviting students to coregulate and metacognitively reflect during assessment. Journal of the Scholarship of Teaching and Learning, 22(1), 81\u201394.","journal-title":"Journal of the Scholarship of Teaching and Learning"},{"key":"408_CR32","unstructured":"Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi\u00e8re, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., & Lample, G. (2023). LLaMA: Open and Efficient Foundation Language Models."},{"issue":"3","key":"408_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3386252","volume":"53","author":"Y Wang","year":"2020","unstructured":"Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys (csur), 53(3), 1\u201334.","journal-title":"ACM Computing Surveys (csur)"},{"key":"408_CR34","unstructured":"Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023). Tree of thoughts: Deliberate problem solving with large language models. arXiv:2305.10601"},{"key":"408_CR35","doi-asserted-by":"crossref","unstructured":"Yu, X., Peng, B., Galley, M., Gao, J., & Yu, Z. (2023). Teaching language models to self-improve through interactive demonstrations. arXiv:2310.13522","DOI":"10.18653\/v1\/2024.naacl-long.287"}],"container-title":["International Journal of Artificial Intelligence in Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40593-024-00408-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40593-024-00408-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40593-024-00408-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T18:12:51Z","timestamp":1772647971000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40593-024-00408-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,8]]},"references-count":35,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["408"],"URL":"https:\/\/doi.org\/10.1007\/s40593-024-00408-y","relation":{},"ISSN":["1560-4292","1560-4306"],"issn-type":[{"value":"1560-4292","type":"print"},{"value":"1560-4306","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,8]]},"assertion":[{"value":"25 April 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 July 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The study presented in this paper obtained the Institutional Review Boards (IRB) approval from Carnegie Mellon University.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"The authors have no relevant financial or non-financial interests to disclose, nor conflicting interests nor competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of Interest"}}]}}