{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T03:12:11Z","timestamp":1782184331741,"version":"3.54.5"},"publisher-location":"New York, NY, USA","reference-count":57,"publisher":"ACM","funder":[{"DOI":"10.13039\/501100006374","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1931523"],"award-info":[{"award-number":["1931523"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,7,17]]},"DOI":"10.1145\/3698205.3729547","type":"proceedings-article","created":{"date-parts":[[2025,7,17]],"date-time":"2025-07-17T14:35:08Z","timestamp":1752762908000},"page":"176-185","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Encouraging Metacognitive Reflection through Prompts in a Computer-Based Learning Platform: Failure to Find a Benefit in a Large-Scale Randomized Trial"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4166-0705","authenticated-orcid":false,"given":"Allison","family":"Zengilowski","sequence":"first","affiliation":[{"name":"Lehigh University, Bethlehem, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5210-6785","authenticated-orcid":false,"given":"Brendan A","family":"Schuetze","sequence":"additional","affiliation":[{"name":"University of Utah, Salt Lake City, UT, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-9973-8511","authenticated-orcid":false,"given":"Abubakir","family":"Siedahmed","sequence":"additional","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3988-3184","authenticated-orcid":false,"given":"Veronica X","family":"Yan","sequence":"additional","affiliation":[{"name":"The University of Texas at Austin, Austin, TX, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3280-288X","authenticated-orcid":false,"given":"Neil T","family":"Heffernan","sequence":"additional","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,7,17]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Teaching Machines: The History of Personalized Learning","author":"Watters Audrey","year":"2023","unstructured":"Audrey Watters. Teaching Machines: The History of Personalized Learning. MIT Press, 2023."},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0032447"},{"key":"e_1_3_2_2_3_1","first-page":"41","volume-title":"Towards a motivationally intelligent pedagogy: how should an intelligent tutor respond to the unmotivated or the demotivated? In New Perspectives on Affect and Learning Technologies","author":"du Boulay Benedict","year":"2011","unstructured":"Benedict du Boulay. Towards a motivationally intelligent pedagogy: how should an intelligent tutor respond to the unmotivated or the demotivated? In New Perspectives on Affect and Learning Technologies, pages 41-52. Springer, 2011."},{"issue":"2","key":"e_1_3_2_2_4_1","first-page":"185","article-title":"Why students engage","volume":"19","author":"Baker Ryan","year":"2008","unstructured":"Ryan Baker, Jason Walonoski, Neil Heffernan, Ido Roll, Albert Corbett, and Kenneth Koedinger. Why students engage in ''gaming the system'' behavior in interactive learning environments. Journal of Interactive Learning Research, 19(2):185-224, 2008.","journal-title":"Journal of Interactive Learning Research"},{"key":"e_1_3_2_2_5_1","volume-title":"Proceedings of the 15th International Conference on Educational Data Mining, International Educational Data Mining Society","author":"Levin Nathan","year":"2022","unstructured":"Nathan Levin, Ryan Baker, Nidhi Nasiar, Fancsali Stephen, and Stephen Hutt. Evaluating gaming detector model robustness over time. In Proceedings of the 15th International Conference on Educational Data Mining, International Educational Data Mining Society, 2022."},{"key":"e_1_3_2_2_6_1","volume-title":"Incentivizing supplemental math assignments and using AI-generated hints improve exam performance, especially for racially minoritized students. arXiv preprint","author":"Lu Yifan","year":"2024","unstructured":"Yifan Lu, K. Supriya, Shanna Shaked, Elizabeth H. Simmons, and Alexander Kusenko. Incentivizing supplemental math assignments and using AI-generated hints improve exam performance, especially for racially minoritized students. arXiv preprint, 2024."},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/jcal.12650"},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"crossref","unstructured":"Arief Darmanegara Liem Shun Lau and Youyan Nie. The role of self-efficacy task value and achievement goals in predicting learning strategies task disengagement peer relationship and achievement outcome. Contemporary educational psychology 33(4):486-512 2008.","DOI":"10.1016\/j.cedpsych.2007.08.001"},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13394-020-00313-2"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-007-9054-3"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.4324\/9780203878842-27"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0028389"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.96.3.558"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-008-9080-9"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-012109890-2\/50043-3"},{"key":"e_1_3_2_2_16_1","first-page":"1","volume-title":"Theories of self-regulated learning and academic achievement: An overview and analysis","author":"Zimmerman Barry J.","year":"2001","unstructured":"Barry J. Zimmerman. Theories of self-regulated learning and academic achievement: An overview and analysis. In Barry J. Zimmerman and Dale H. Schunk, editors, Self-Regulated Learning and Academic Achievement, chapter 1, pages 1-36. Routledge, 2nd. edition, 2001."},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-023-09767-9"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tate.2017.03.001"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1111\/cogs.13136"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2021.104315"},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1093\/applin\/19.4.515"},{"key":"e_1_3_2_2_22_1","volume-title":"The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41(4):219-225","author":"Pintrich Paul R","year":"2002","unstructured":"Paul R Pintrich. The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41(4):219-225, 2002."},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1086\/461578"},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.system.2008.08.003"},{"key":"e_1_3_2_2_25_1","volume-title":"Self-regulated learning: Beliefs, techniques, and illusions. Annual review of psychology, 64(1):417-444","author":"Bjork Robert A","year":"2013","unstructured":"Robert A Bjork, John Dunlosky, and Nate Kornell. Self-regulated learning: Beliefs, techniques, and illusions. Annual review of psychology, 64(1):417-444, 2013."},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066X.34.10.906"},{"key":"e_1_3_2_2_27_1","volume-title":"Confusion and confusion regulation: An empirical investigation of the emotion regulation in achievement situations model. Contemporary Educational Psychology, page 102350","author":"Muis Krista R","year":"2025","unstructured":"Krista R Muis, Martina Kohatsu, Reinhard Pekrun, and Shasha Li. Confusion and confusion regulation: An empirical investigation of the emotion regulation in achievement situations model. Contemporary Educational Psychology, page 102350, 2025."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2011.10.001"},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2012.05.003"},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/2336131.2336134"},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11409-010-9060-6"},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.iheduc.2012.01.002"},{"issue":"1","key":"e_1_3_2_2_33_1","first-page":"52","article-title":"Exploring the effect of student confusion in massive open online courses","volume":"8","author":"Yang Diyi","year":"2016","unstructured":"Diyi Yang, Robert E Kraut, and Carolyn P Ros\u00e9. Exploring the effect of student confusion in massive open online courses. Journal of Educational Data Mining, 8(1):52-83, 2016.","journal-title":"Journal of Educational Data Mining"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-19773-9_22"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2019.04.001"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.iheduc.2019.100690"},{"key":"e_1_3_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10758-020-09446-6"},{"key":"e_1_3_2_2_38_1","volume-title":"Self-regulated learning strategies and non-academic outcomes in higher education blended learning environments: A one decade review. Education and information technologies, 25(5):3677-3704","author":"Anthonysamy Lilian","year":"2020","unstructured":"Lilian Anthonysamy, Ah-Choo Koo, and Soon-Hin Hew. Self-regulated learning strategies and non-academic outcomes in higher education blended learning environments: A one decade review. Education and information technologies, 25(5):3677-3704, 2020."},{"key":"e_1_3_2_2_39_1","volume-title":"Misinformed and unaware? Metacognition and the influence of inaccurate information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 47(4):608-624","author":"Salovich Nikita A","year":"2021","unstructured":"Nikita A Salovich and David N Rapp. Misinformed and unaware? Metacognition and the influence of inaccurate information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 47(4):608-624, 2021."},{"key":"e_1_3_2_2_40_1","first-page":"58","article-title":"The benefit of reflection prompts for encouraging learning with hints in an online programming course","author":"Choi Heeryung","year":"2023","unstructured":"Heeryung Choi, Jelena Jovanovic, Oleksandra Poquet, Christopher Brooks, Sre\u0107ko Joksimovi\u0107, and Joseph Jay Williams. The benefit of reflection prompts for encouraging learning with hints in an online programming course. The Internet and Higher Education, 58, 2023.","journal-title":"The Internet and Higher Education"},{"key":"e_1_3_2_2_41_1","volume-title":"The influence of self-efficacy and metacognitive prompting on math problem-solving efficiency. Contemporary educational psychology, 33(4):875-893","author":"Hoffman Bobby","year":"2008","unstructured":"Bobby Hoffman and Alexandru Spatariu. The influence of self-efficacy and metacognitive prompting on math problem-solving efficiency. Contemporary educational psychology, 33(4):875-893, 2008."},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.3102\/00346543073003277"},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2010.07.004"},{"key":"e_1_3_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0027696"},{"key":"e_1_3_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2012.08.003"},{"key":"e_1_3_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11409-019-09203-w"},{"key":"e_1_3_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1080\/00461520.2014.965823"},{"key":"e_1_3_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.89.2.329"},{"key":"e_1_3_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/11774303_36"},{"key":"e_1_3_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2003.10.001"},{"key":"e_1_3_2_2_51_1","first-page":"515","volume-title":"FLAIRS","author":"Heffernan Neil T","year":"2006","unstructured":"Neil T Heffernan, Terrence E Turner, Abraao LN Lourenco, Michael A Macasek, Goss Nuzzo-Jones, and Kenneth R Koedinger. The assistment builder: Towards an analysis of cost effectiveness of its creation. In FLAIRS, pages 515-520, 2006."},{"key":"e_1_3_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-014-0024-x"},{"key":"e_1_3_2_2_53_1","volume-title":"Behavioural science is unlikely to change the world without a heterogeneity revolution. Nature human behaviour, 5(8):980-989","author":"Bryan Christopher J","year":"2021","unstructured":"Christopher J Bryan, Elizabeth Tipton, and David S Yeager. Behavioural science is unlikely to change the world without a heterogeneity revolution. Nature human behaviour, 5(8):980-989, 2021."},{"issue":"2","key":"e_1_3_2_2_54_1","first-page":"1","article-title":"von Hippel and Brendan A. Schuetze. How not to fool ourselves about the heterogeneity of treatment effects","volume":"8","author":"Paul","year":"2025","unstructured":"Paul T. von Hippel and Brendan A. Schuetze. How not to fool ourselves about the heterogeneity of treatment effects. Advances in Methods and Practices in Psychological Science, 8(2):1-30, 2025.","journal-title":"Advances in Methods and Practices in Psychological Science"},{"key":"e_1_3_2_2_55_1","volume-title":"Projections of education statistics to","author":"Hussar William J","year":"2020","unstructured":"William J Hussar and Tabitha M Bailey. Projections of education statistics to 2020. nces 2011-026. National Center for Education Statistics, 2011."},{"key":"e_1_3_2_2_56_1","volume-title":"Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:1406.5823","author":"Bates Douglas","year":"2014","unstructured":"Douglas Bates, Martin M\u00e4chler, Ben Bolker, and Steve Walker. Fitting linear mixed-effects models using lme4. arXiv preprint arXiv:1406.5823, 2014."},{"key":"e_1_3_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1177\/0272431614529365"}],"event":{"name":"L@S '25: Twelfth ACM Conference on Learning @ Scale","location":"Palermo Italy","acronym":"L@S '25"},"container-title":["Proceedings of the Twelfth ACM Conference on Learning @ Scale"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3698205.3729547","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T13:17:03Z","timestamp":1755868623000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3698205.3729547"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,17]]},"references-count":57,"alternative-id":["10.1145\/3698205.3729547","10.1145\/3698205"],"URL":"https:\/\/doi.org\/10.1145\/3698205.3729547","relation":{},"subject":[],"published":{"date-parts":[[2025,7,17]]},"assertion":[{"value":"2025-07-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}