{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,31]],"date-time":"2026-10-31T14:40:11Z","timestamp":1793457611652,"version":"4.5.0"},"reference-count":28,"publisher":"National Academy of Sciences","issue":"26","license":[{"start":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T00:00:00Z","timestamp":1750809600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"name":"Wharton AI and Analytics Initiative","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}]},{"name":"Fishman Davidson Center at Wharton","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}]},{"name":"Wharton Global Initiatives","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}]}],"content-domain":{"domain":["www.pnas.org"],"crossmark-restriction":true},"short-container-title":["Proc. Natl. Acad. Sci. U.S.A."],"published-print":{"date-parts":[[2025,7]]},"abstract":"<jats:p>Generative AI is poised to revolutionize how humans work, and has already demonstrated promise in significantly improving human productivity. A key question is how generative AI affects learning\u2014namely, how humans acquire new skills as they perform tasks. Learning is critical to long-term productivity, especially since generative AI is fallible and users must check its outputs. We study this question via a field experiment where we provide nearly a thousand high school math students with access to generative AI tutors. To understand the differential impact of tool design on learning, we deploy two generative AI tutors: one that mimics a standard ChatGPT interface (\u201cGPT Base\u201d) and one with prompts designed to safeguard learning (\u201cGPT Tutor\u201d). Consistent with prior work, our results show that having GPT-4 access while solving problems significantly improves performance (48% improvement in grades for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction in grades for GPT Base)\u2014i.e., unfettered access to GPT-4 can harm educational outcomes. These negative learning effects are largely mitigated by the safeguards in GPT Tutor. Without guardrails, students attempt to use GPT-4 as a \u201ccrutch\u201d during practice problem sessions, and subsequently perform worse on their own. Thus, decision-makers must be cautious about design choices underlying generative AI deployments to preserve skill learning and long-term productivity.<\/jats:p>","DOI":"10.1073\/pnas.2422633122","type":"journal-article","created":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T11:38:07Z","timestamp":1750851487000},"update-policy":"https:\/\/doi.org\/10.1073\/pnas.cm10313","source":"Crossref","is-referenced-by-count":327,"title":["Generative AI without guardrails can harm learning: Evidence from high school mathematics"],"prefix":"10.1073","volume":"122","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8793-4732","authenticated-orcid":false,"given":"Hamsa","family":"Bastani","sequence":"first","affiliation":[{"name":"Department of Operations, Information, and Decisions","place":["Philadelphia"]},{"name":"Wharton School","place":["Philadelphia"]},{"name":"University of Pennsylvania","place":["Philadelphia"]},{"name":"Wharton AI & Analytics","place":["Philadelphia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Osbert","family":"Bastani","sequence":"additional","affiliation":[{"name":"Department of Computer and Information Science","place":["Philadelphia"]},{"name":"School of Engineering and Applied Science","place":["Philadelphia"]},{"name":"University of Pennsylvania","place":["Philadelphia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1130-677X","authenticated-orcid":false,"given":"Alp","family":"Sungu","sequence":"additional","affiliation":[{"name":"Department of Operations, Information, and Decisions","place":["Philadelphia"]},{"name":"Wharton School","place":["Philadelphia"]},{"name":"University of Pennsylvania","place":["Philadelphia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2443-3949","authenticated-orcid":false,"given":"Haosen","family":"Ge","sequence":"additional","affiliation":[{"name":"Wharton AI & Analytics","place":["Philadelphia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"\u00d6zge","family":"Kabakc\u0131","sequence":"additional","affiliation":[{"name":"Department of Mathematics","place":["Budapest, Hungary"]},{"name":"Budapest British International School","place":["Budapest, Hungary"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rei","family":"Mariman","sequence":"additional","affiliation":[{"name":"Independent","place":["Philadelphia"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"341","published-online":{"date-parts":[[2025,6,25]]},"reference":[{"key":"e_1_3_5_1_2","unstructured":"J. Achiam Gpt-4 technical report. arXiv [Preprint] (2023). https:\/\/arxiv.org\/abs\/2303.08774 (Accessed 5 March 2025)."},{"key":"e_1_3_5_2_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06291-2"},{"key":"e_1_3_5_3_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.abq1158"},{"key":"e_1_3_5_4_2","first-page":"233","article-title":"ChatGPT has aced the test of understanding in college economics: Now what?","volume":"68","author":"Geerling W.","year":"2023","unstructured":"W. Geerling, G. D. Mateer, J. Wooten, N. Damodaran, ChatGPT has aced the test of understanding in college economics: Now what? Am. Econ. 68, 233\u2013245 (2023).","journal-title":"Am. Econ."},{"key":"e_1_3_5_5_2","volume-title":"Mack Institute White Paper","author":"Terwiesch C.","year":"2023","unstructured":"C. Terwiesch, \u201cWould chat GPT get a Wharton MBA\u201d in Mack Institute White Paper (Wharton School, University of Pennsylvania, 2023)."},{"key":"e_1_3_5_6_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.adj0998"},{"key":"e_1_3_5_7_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.adh2586"},{"key":"e_1_3_5_8_2","doi-asserted-by":"crossref","unstructured":"E. Brynjolfsson D. Li L. R. Raymond \u201cGenerative AI at work\u201d in National Bureau of Economic Research Working Paper (2023).","DOI":"10.3386\/w31161"},{"key":"e_1_3_5_9_2","doi-asserted-by":"crossref","unstructured":"F. Dell\u2019Acqua \u201cNavigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and quality\u201d in Harvard Business School Technology & Operations Management Unit Working Paper (2023).","DOI":"10.2139\/ssrn.4573321"},{"key":"e_1_3_5_10_2","volume-title":"Human Capital: A Theoretical and Empirical Analysis, with Special Reference to Education","author":"Becker G. S.","year":"2009","unstructured":"G. S. Becker, Human Capital: A Theoretical and Empirical Analysis, with Special Reference to Education (University of Chicago press, 2009)."},{"key":"e_1_3_5_11_2","unstructured":"Safety alert for operators: Manual flight operations (Tech. Rep. SAFO 13002 Federal Aviation Administration 2013)."},{"key":"e_1_3_5_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.173"},{"key":"e_1_3_5_13_2","doi-asserted-by":"crossref","unstructured":"E. R. Mollick L. Mollick New modes of learning enabled by AI chatbots: Three methods and assignments. http:\/\/dx.doi.org\/10.2139\/ssrn.4300783. Accessed 5 March 2025.","DOI":"10.2139\/ssrn.4300783"},{"key":"e_1_3_5_14_2","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-023-03507-3"},{"key":"e_1_3_5_15_2","doi-asserted-by":"crossref","unstructured":"S. Ruan et al. \u201cSupporting children\u2019s math learning with feedback-augmented narrative technology\u201d in Proceedings of the Interaction Design and Children Conference (Association for Computing Machinery New York NY 2020) pp. 567\u2013580.","DOI":"10.1145\/3392063.3394400"},{"key":"e_1_3_5_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-023-06423-9"},{"key":"e_1_3_5_17_2","doi-asserted-by":"publisher","DOI":"10.1257\/app.20150385"},{"key":"e_1_3_5_18_2","doi-asserted-by":"publisher","DOI":"10.18404\/ijemst.408936"},{"key":"e_1_3_5_19_2","doi-asserted-by":"publisher","DOI":"10.1257\/jel.20191507"},{"key":"e_1_3_5_20_2","unstructured":"E. L. Bjork R. A. Bjork \u201cMaking things hard on yourself but in a good way: Creating desirable difficulties to enhance learning\u201d in Psychology and the Real World: Essays Illustrating Fundamental Contributions to Society M. A. Gernsbacher R. W. Pew L. M. Hough J. R. Pomerantz Eds. (Worth Publishers 2011) pp. 56\u201364."},{"key":"e_1_3_5_21_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1821936116"},{"key":"e_1_3_5_22_2","first-page":"13867","article-title":"Mathematical capabilities of chatGPT","volume":"36","author":"Frieder S.","year":"2024","unstructured":"S. Frieder , Mathematical capabilities of chatGPT. Adv. Neural Inf. Process. Syst. 36, 13867 (2024).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_5_23_2","unstructured":"N. Singer In classrooms teachers put A.I. tutoring bots to the test. N. Y. Times (2023)."},{"key":"e_1_3_5_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-019-00187-x"},{"key":"e_1_3_5_25_2","unstructured":"S. Yao \u201cReact: Synergizing reasoning and acting in language models\u201d in The Eleventh International Conference on Learning Representations (ICLR) (2023)."},{"key":"e_1_3_5_26_2","volume-title":"Forty-First International Conference on Machine Learning","author":"Wang X.","year":"2024","unstructured":"X. Wang , \u201cExecutable code actions elicit better LLM agents\u201d in Forty-First International Conference on Machine Learning (JMLR.org, Vienna, Austria, 2024)."},{"key":"e_1_3_5_27_2","doi-asserted-by":"crossref","unstructured":"R. E. Wang A. T. Ribeiro C. D. Robinson S. Loeb D. Demszky Tutor copilot: A human-AI approach for scaling real-time expertise. arXiv [Preprint] (2024). https:\/\/arxiv.org\/abs\/2410.03017 (Accessed 5 March 2025).","DOI":"10.21203\/rs.3.rs-5363154\/v1"},{"key":"e_1_3_5_28_2","doi-asserted-by":"crossref","unstructured":"H. Bastani Data and Code for \u201cGenerative AI without guardrails can harm learning: Evidence from high school mathematics.\u201d GitHub. https:\/\/github.com\/obastani\/GenAICanHarmLearning. Deposited 11 June 2025.","DOI":"10.1073\/pnas.2422633122"}],"updated-by":[{"DOI":"10.1073\/pnas.2518204122","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T00:00:00Z","timestamp":1755648000000}}],"container-title":["Proceedings of the National Academy of Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pnas.org\/doi\/pdf\/10.1073\/pnas.2422633122","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T14:42:50Z","timestamp":1755700970000},"score":1,"resource":{"primary":{"URL":"https:\/\/pnas.org\/doi\/10.1073\/pnas.2422633122"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,25]]},"references-count":28,"journal-issue":{"issue":"26","published-print":{"date-parts":[[2025,7]]}},"alternative-id":["10.1073\/pnas.2422633122"],"URL":"https:\/\/doi.org\/10.1073\/pnas.2422633122","relation":{},"ISSN":["0027-8424","1091-6490"],"issn-type":[{"value":"0027-8424","type":"print"},{"value":"1091-6490","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,25]]},"assertion":[{"value":"2024-11-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-25","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2422633122"}}