{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T17:00:57Z","timestamp":1771261257000,"version":"3.50.1"},"publisher-location":"Cham","reference-count":36,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783031426810","type":"print"},{"value":"9783031426827","type":"electronic"}],"license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,28]],"date-time":"2023-08-28T00:00:00Z","timestamp":1693180800000},"content-version":"vor","delay-in-days":239,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    In numerous studies, intelligent tutoring systems (ITSs) have proven effective in helping students learn mathematics. Prior work posits that their effectiveness derives from efficiently providing eventually-correct practice opportunities. Yet, there is little empirical evidence on how learning processes with ITSs compare to other forms of instruction. The current study compares problem-solving with an ITS versus solving the same problems on paper. We analyze the learning process and pre-post gain data from N\u2009=\u200997 middle school students practicing linear graphs in three curricular units. We find that (i) working with the ITS, students had more than twice the number of eventually-correct practice opportunities than when working on paper and (ii) omission errors on paper were associated with lower learning gains. Yet, contrary to our hypothesis, tutor practice did not yield greater learning gains, with tutor and paper comparing differently across curricular units. These findings align with tutoring allowing students to grapple with challenging steps through tutor assistance but not with eventually-correct opportunities driving learning gains. Gaming-the-system, lack of transfer to an unfamiliar test format, potentially ineffective tutor design, and learning affordances of paper can help explain this gap. This study provides first-of-its-kind quantitative evidence that ITSs yield more learning\n                    <jats:italic>opportunities<\/jats:italic>\n                    than equivalent paper-and-pencil practice and reveals that the relation between opportunities and learning gains emerges only when the instruction is effective.\n                  <\/jats:p>","DOI":"10.1007\/978-3-031-42682-7_4","type":"book-chapter","created":{"date-parts":[[2023,8,29]],"date-time":"2023-08-29T15:01:46Z","timestamp":1693321306000},"page":"44-59","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["What Makes Problem-Solving Practice Effective? Comparing Paper and AI Tutoring"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3437-8979","authenticated-orcid":false,"given":"Conrad","family":"Borchers","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0449-3733","authenticated-orcid":false,"given":"Paulo F.","family":"Carvalho","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2676-9032","authenticated-orcid":false,"given":"Meng","family":"Xia","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3842-1017","authenticated-orcid":false,"given":"Pinyang","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5850-4768","authenticated-orcid":false,"given":"Kenneth R.","family":"Koedinger","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1581-6657","authenticated-orcid":false,"given":"Vincent","family":"Aleven","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,8,28]]},"reference":[{"issue":"5","key":"4_CR1","doi-asserted-by":"publisher","first-page":"315","DOI":"10.3390\/educsci12050315","volume":"12","author":"M Kerres","year":"2022","unstructured":"Kerres, M., Buchner, J.: Education after the pandemic: what we have (not) learned about learning. Educ. Sci. 12(5), 315 (2022)","journal-title":"Educ. Sci."},{"key":"4_CR2","doi-asserted-by":"publisher","first-page":"227","DOI":"10.3233\/IRG-2006-16(3)02","volume":"16","author":"K VanLehn","year":"2006","unstructured":"VanLehn, K.: The behavior of tutoring systems. Int. J. Artif. Intell. Educ. 16, 227\u2013265 (2006)","journal-title":"Int. J. Artif. Intell. Educ."},{"key":"4_CR3","unstructured":"Ritter, S., Fancsali, S.: MATHia X: the next generation cognitive tutor. In: EDM, pp. 624\u2013625. ERIC (2016)"},{"key":"4_CR4","doi-asserted-by":"publisher","first-page":"1278","DOI":"10.1080\/01443410.2018.1495829","volume":"39","author":"Y Fang","year":"2019","unstructured":"Fang, Y., Ren, Z., Hu, X., Graesser, A.C.: A meta-analysis of the effectiveness of ALEKS on learning. Educ. Psychol. 39, 1278\u20131292 (2019)","journal-title":"Educ. Psychol."},{"key":"4_CR5","doi-asserted-by":"publisher","first-page":"470","DOI":"10.1007\/s40593-014-0024-x","volume":"24","author":"NT Heffernan","year":"2014","unstructured":"Heffernan, N.T., Heffernan, C.L.: The ASSISTments ecosystem: building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. Int. J. Artif. Intell. Educ. 24, 470\u2013497 (2014)","journal-title":"Int. J. Artif. Intell. Educ."},{"key":"4_CR6","unstructured":"Silva, T.B.: The effects of the i-Ready computer assisted instruction program on the reading and fluency achievement of first graders (2016)"},{"key":"4_CR7","doi-asserted-by":"publisher","first-page":"536","DOI":"10.1007\/s40593-015-0060-1","volume":"26","author":"B du Boulay","year":"2016","unstructured":"du Boulay, B.: Recent meta-reviews and meta\u2013analyses of AIED systems. Int. J. Artif. Intell. Educ. 26, 536\u2013537 (2016)","journal-title":"Int. J. Artif. Intell. Educ."},{"key":"4_CR8","doi-asserted-by":"publisher","first-page":"42","DOI":"10.3102\/0034654315581420","volume":"86","author":"JA Kulik","year":"2016","unstructured":"Kulik, J.A., Fletcher, J.D.: Effectiveness of intelligent tutoring systems: a meta-analytic review. Rev. Educ. Res. 86, 42\u201378 (2016)","journal-title":"Rev. Educ. Res."},{"key":"4_CR9","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1080\/00461520.2011.611369","volume":"46","author":"K VanLehn","year":"2011","unstructured":"VanLehn, K.: The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educ. Psychol. 46, 197\u2013221 (2011)","journal-title":"Educ. Psychol."},{"key":"4_CR10","unstructured":"Aleven, V., McLaughlin, E.A., Glenn, R.A., Koedinger, K.R.: Instruction based on adaptive learning technologies. In: Handbook of Research on Learning and Instruction, vol. 2, pp. 522\u2013560 (2016)"},{"key":"4_CR11","doi-asserted-by":"crossref","unstructured":"Corbett, A.T., Anderson, J.R.: Locus of feedback control in computer-based tutoring: impact on learning rate, achievement and attitudes. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 245\u2013252 (2001)","DOI":"10.1145\/365024.365111"},{"key":"4_CR12","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1207\/s15327809jls0402_2","volume":"4","author":"JR Anderson","year":"1995","unstructured":"Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4, 167\u2013207 (1995)","journal-title":"J. Learn. Sci."},{"key":"4_CR13","unstructured":"Cabalo, J.V., Ma, B., Jaciw, A.: Comparative Effectiveness of Carnegie Learning\u2019s \u201cCognitive Tutor Bridge to Algebra\u201d Curriculum: A Report of a Randomized Experiment in the Maui School District. Research Report. Empirical Education Inc. (2007)"},{"key":"4_CR14","first-page":"13","volume-title":"Supporting learning flow through integrative technologies","author":"S Ritter","year":"2007","unstructured":"Ritter, S., Kulikowich, J., Lei, P., McGuire, C.L., Morgan, P.: What evidence matters? A randomized field trial of Cognitive Tutor Algebra I. In: Hirashima, T., Hoppe, U., Young, U.U. (eds.) Supporting Learning Flow Through Integrative Technologies, vol. 162, pp. 13\u201320. IOS Press, Amsterdam (2007)"},{"issue":"4","key":"4_CR15","doi-asserted-by":"publisher","first-page":"970","DOI":"10.1037\/a0032447","volume":"105","author":"S Steenbergen-Hu","year":"2013","unstructured":"Steenbergen-Hu, S., Cooper, H.: A meta-analysis of the effectiveness of intelligent tutoring systems on K\u201312 students\u2019 mathematical learning. J. Educ. Psychol. 105(4), 970 (2013)","journal-title":"J. Educ. Psychol."},{"issue":"1","key":"4_CR16","first-page":"30","volume":"8","author":"KR Koedinger","year":"1997","unstructured":"Koedinger, K.R., Anderson, J.R., Hadley, W.H., Mark, M.A.: Intelligent tutoring goes to school in the big city. Int. J. Artif. Intell. Educ. 8(1), 30\u201343 (1997)","journal-title":"Int. J. Artif. Intell. Educ."},{"issue":"3","key":"4_CR17","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1080\/15391523.2009.10782534","volume":"41","author":"M Mendicino","year":"2009","unstructured":"Mendicino, M., Razzaq, L., Heffernan, N.T.: A comparison of traditional homework to computer-supported homework. J. Res. Technol. Educ. 41(3), 331\u2013359 (2009)","journal-title":"J. Res. Technol. Educ."},{"key":"4_CR18","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2020.103869","volume":"152","author":"P Magalh\u00e3es","year":"2020","unstructured":"Magalh\u00e3es, P., Ferreira, D., Cunha, J., Ros\u00e1rio, P.: Online vs traditional homework: a systematic review on the benefits to students\u2019 performance. Comput. Educ. 152, 103869 (2020)","journal-title":"Comput. Educ."},{"key":"4_CR19","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1007\/s10648-007-9049-0","volume":"19","author":"KR Koedinger","year":"2007","unstructured":"Koedinger, K.R., Aleven, V.: Exploring the assistance dilemma in experiments with cognitive tutors. Educ. Psychol. Rev. 19, 239\u2013264 (2007)","journal-title":"Educ. Psychol. Rev."},{"key":"4_CR20","doi-asserted-by":"publisher","unstructured":"Baker, R.S., Corbett, A.T., Koedinger, K.R.: Detecting student misuse of intelligent tutoring systems. In: Lester, J.C., Vicari, R.M., Paragua\u00e7u, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 531\u2013540. Springer, Heidelberg (2004). https:\/\/doi.org\/10.1007\/978-3-540-30139-4_50","DOI":"10.1007\/978-3-540-30139-4_50"},{"key":"4_CR21","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1207\/s15516709cog1304_1","volume":"13","author":"JR Anderson","year":"1989","unstructured":"Anderson, J.R., Conrad, F.G., Corbett, A.T.: Skill acquisition and the LISP tutor. Cogn. Sci. 13, 467\u2013505 (1989)","journal-title":"Cogn. Sci."},{"key":"4_CR22","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1111\/j.1551-6709.2012.01245.x","volume":"36","author":"KR Koedinger","year":"2012","unstructured":"Koedinger, K.R., Corbett, A.T., Perfetti, C.: The knowledge-learning-instruction framework: bridging the science-practice chasm to enhance robust student learning. Cogn. Sci. 36, 757\u2013798 (2012)","journal-title":"Cogn. Sci."},{"key":"4_CR23","unstructured":"Koedinger, K.R., Baker, R.Sj., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J.: A data repository for the EDM community: the PSLC DataShop. In: Handbook of Educational Data Mining, vol. 43, pp. 43\u201356 (2010)"},{"key":"4_CR24","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1109\/TLT.2009.22","volume":"2","author":"V Aleven","year":"2009","unstructured":"Aleven, V., McLaren, B.M., Sewall, J.: Scaling up programming by demonstration for intelligent tutoring systems development: an open-access web site for middle school mathematics learning. IEEE Trans. Learn. Technol. 2, 64\u201378 (2009)","journal-title":"IEEE Trans. Learn. Technol."},{"key":"4_CR25","doi-asserted-by":"crossref","unstructured":"Sibbald, B., Roberts, C.: Understanding controlled trials crossover trials. Bmj 316, 1719\u20131720 (1998)","DOI":"10.1136\/bmj.316.7146.1719"},{"key":"4_CR26","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1111\/tops.12035","volume":"5","author":"KR Koedinger","year":"2013","unstructured":"Koedinger, K.R., Stamper, J.C., Leber, B., Skogsholm, A.: LearnLab\u2019s DataShop: a data repository and analytics tool set for cognitive science. Top. Cogn. Sci. 5, 668\u2013669 (2013)","journal-title":"Top. Cogn. Sci."},{"key":"4_CR27","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1080\/1049482980050111","volume":"5","author":"KR Koedinger","year":"1998","unstructured":"Koedinger, K.R., Anderson, J.R.: Illustrating principled design: the early evolution of a cognitive tutor for algebra symbolization. Interact. Learn. Environ. 5, 161\u2013179 (1998)","journal-title":"Interact. Learn. Environ."},{"key":"4_CR28","unstructured":"Paquette, L., de Carvahlo, A., Baker, R., Ocumpaugh, J.: Reengineering the feature distillation process: a case study in detection of gaming the system. In: Educational Data Mining 2014 (2014)"},{"key":"4_CR29","doi-asserted-by":"publisher","unstructured":"Cen, H., Koedinger, K., Junker, B.: Learning factors analysis \u2013 a general method for cognitive model evaluation and improvement. In: Ikeda, M., Ashley, K.D., Chan, TW. (eds.) ITS 2006. LNCS, vol. 4053, pp. 164\u2013175. Springer, Heidelberg (2006). https:\/\/doi.org\/10.1007\/11774303_17","DOI":"10.1007\/11774303_17"},{"issue":"3","key":"4_CR30","first-page":"254","volume":"3","author":"JF Pane","year":"2010","unstructured":"Pane, J.F., McCaffrey, D.F., Slaughter, M.E., Steele, J.L., Ikemoto, G.S.: An experiment to evaluate the efficacy of cognitive tutor geometry. J. Res. Educ. Effect. 3(3), 254\u2013281 (2010)","journal-title":"J. Res. Educ. Effect."},{"key":"4_CR31","doi-asserted-by":"crossref","unstructured":"Anthony, L., Yang, J., Koedinger, K.R.: Towards the application of a handwriting interface for mathematics learning. In: 2006 IEEE International Conference on Multimedia and Expo, pp. 2077\u20132080 (2006)","DOI":"10.1109\/ICME.2006.262643"},{"key":"4_CR32","unstructured":"Huang, Y., et al.: Item response theory-based gaming detection. Int. Educ. Data Min. Soc. (2022)"},{"issue":"2","key":"4_CR33","first-page":"147","volume":"26","author":"VA Aleven","year":"2002","unstructured":"Aleven, V.A., Koedinger, K.R.: An effective metacognitive strategy: learning by doing and explaining with a computer-based cognitive tutor. Cogn. Sci. 26(2), 147\u2013179 (2002)","journal-title":"Cogn. Sci."},{"issue":"4","key":"4_CR34","doi-asserted-by":"publisher","first-page":"774","DOI":"10.1037\/0022-0663.95.4.774","volume":"95","author":"RK Atkinson","year":"2003","unstructured":"Atkinson, R.K., Renkl, A., Merrill, M.M.: Transitioning from studying examples to solving problems: effects of self-explanation prompts and fading worked-out steps. J. Educ. Psychol. 95(4), 774 (2003)","journal-title":"J. Educ. Psychol."},{"key":"4_CR35","doi-asserted-by":"crossref","unstructured":"Huang, Y., et al.: A general multi-method approach to data-driven redesign of tutoring systems. In: LAK21: 11th International Learning Analytics and Knowledge Conference, pp. 161\u2013172 (2021)","DOI":"10.1145\/3448139.3448155"},{"key":"4_CR36","doi-asserted-by":"crossref","unstructured":"Koedinger, K.R., McLaughlin, E.A., Jia, J.Z., Bier, N.L.: Is the doer effect a causal relationship? How can we tell and why it\u2019s important. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 388\u2013397 (2016)","DOI":"10.1145\/2883851.2883957"}],"updated-by":[{"DOI":"10.1007\/978-3-031-42682-7_75","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2023,8,28]],"date-time":"2023-08-28T00:00:00Z","timestamp":1693180800000}}],"container-title":["Lecture Notes in Computer Science","Responsive and Sustainable Educational Futures"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-42682-7_4","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T13:58:15Z","timestamp":1762091895000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-42682-7_4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"ISBN":["9783031426810","9783031426827"],"references-count":36,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-42682-7_4","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023]]},"assertion":[{"value":"28 August 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"28 August 2023","order":2,"name":"change_date","label":"Change Date","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"Correction","order":3,"name":"change_type","label":"Change Type","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"A correction has been published.","order":4,"name":"change_details","label":"Change Details","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"EC-TEL","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"European Conference on Technology Enhanced Learning","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Aveiro","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Portugal","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2023","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"4 September 2023","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"8 September 2023","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"18","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"ectel2023","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/ea-tel.eu\/ectel2023","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}