{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T04:27:49Z","timestamp":1785472069296,"version":"3.56.0"},"reference-count":84,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"DOI":"10.13039\/501100006464","name":"Birla Institute of Technology And Science - Pilani","doi-asserted-by":"crossref","award":["C2\/24\/317"],"award-info":[{"award-number":["C2\/24\/317"]}],"id":[{"id":"10.13039\/501100006464","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM J. Comput. Sustain. Soc."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>The advent of Large Language Models (LLMs) is reshaping education, particularly in programming, by enhancing problem-solving, enabling personalized feedback, and supporting adaptive learning. Existing AI tools for programming education struggle with key challenges, including the lack of Socratic guidance, direct code generation, limited context retention, minimal adaptive feedback, and the need for prompt engineering. To address these challenges, we introduce Sakshm AI, an intelligent tutoring system for learners across undergraduate and advanced secondary learners. It fosters Socratic learning through Disha, its inbuilt AI chatbot, which provides context-aware hints, structured feedback, and adaptive guidance while maintaining conversational memory and supporting language flexibility. This study examines 1,170 registered participants, analyzing platform logs, engagement trends, and problem-solving behavior to assess Sakshm AI\u2019s impact. Additionally, a structured survey with 45 active users and 25 in-depth interviews was conducted, using thematic encoding to extract qualitative insights. Our findings reveal how AI-driven Socratic guidance influences problem-solving behaviors and engagement, offering key recommendations for optimizing AI-based coding platforms. This research combines quantitative and qualitative insights to inform AI-assisted education, providing a framework for scalable, intelligent tutoring systems that improve learning outcomes. Furthermore, Sakshm AI represents a significant step toward Sustainable Development Goal 4: Quality Education, providing an accessible and structured learning tool for undergraduate students, even without expert guidance. This is one of the first large-scale studies examining AI-assisted programming education across multiple institutions and demographics.<\/jats:p>","DOI":"10.1145\/3788679","type":"journal-article","created":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T21:15:00Z","timestamp":1769030100000},"page":"1-38","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Sakshm AI: Advancing AI-Assisted Coding Education for Engineering Students in India Through Socratic Tutoring and Comprehensive Feedback"],"prefix":"10.1145","volume":"4","author":[{"given":"Harshita","family":"Goyal","sequence":"first","affiliation":[{"name":"BITS Pilani","place":["Pilani, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-2819-9090","authenticated-orcid":false,"given":"Raj","family":"Gupta","sequence":"additional","affiliation":[{"name":"IIIT-Delhi","place":["New Delhi, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sanchit","family":"Sharma","sequence":"additional","affiliation":[{"name":"IIIT-Delhi","place":["New Delhi, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kashish","family":"Mittal","sequence":"additional","affiliation":[{"name":"Microsoft Research India","place":["Bengaluru, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Apurv","family":"Mehra","sequence":"additional","affiliation":[{"name":"IIIT-Delhi","place":["New Delhi, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jagat","family":"Challa","sequence":"additional","affiliation":[{"name":"BITS Pilani","place":["Pilani, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dhruv","family":"Kumar","sequence":"additional","affiliation":[{"name":"BITS Pilani","place":["Pilani, India"]},{"name":"IIITD","place":["Pilani, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,4,13]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1002\/jac5.1441"},{"key":"e_1_3_2_3_2","unstructured":"Sakshm AI. 2025. Sakshm Dashboard: AI-Powered Coding Practice Platform. Retrieved January 30 2025 from https:\/\/sakshm.com"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3626252.3630799"},{"key":"e_1_3_2_5_2","unstructured":"Bashar Alhafni Sowmya Vajjala Stefano Bann\u00f2 Kaushal Kumar Maurya and Ekaterina Kochmar. 2024. LLMs in Education: Novel Perspectives Challenges and Opportunities. arXiv:2409.11917. Retrieved from https:\/\/arxiv.org\/abs\/2409.11917"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.5430\/elr.v3n1p39"},{"key":"e_1_3_2_7_2","unstructured":"amazon. 1995. Amazon Shooping. Retrieved January 31 2025 from https:\/\/www.amazon.in\/"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","unstructured":"Hannah McLean Babe Sydney Nguyen Yangtian Zi Arjun Guha Molly Q. Feldman and Carolyn Jane Anderson. 2024. StudentEval: A benchmark of student-written prompts for large language models of code. In Findings of the Association for Computational Linguistics: (ACL\u201924). Association for Computational Linguistics 8452\u20138474. DOI:10.18653\/v1\/2024.findings-acl.501","DOI":"10.18653\/v1\/2024.findings-acl.501"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/AERO.2017.7943959"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3545945.3569759"},{"key":"e_1_3_2_11_2","unstructured":"James R. Bilodeau and William A. Rhodes. 2010. Data logging collection and analysis techniques. US Patent 7 698 028."},{"key":"e_1_3_2_12_2","first-page":"133","article-title":"Deductive and inductive approaches to qualitative data analysis","volume":"1","author":"Bingham Andrea J.","year":"2021","unstructured":"Andrea J. Bingham and Patricia Witkowsky. 2021. Deductive and inductive approaches to qualitative data analysis. Analyzing and Interpreting Qualitative Data: After the Interview 1 (2021), 133\u2013146.","journal-title":"Analyzing and Interpreting Qualitative Data: After the Interview"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3699538.3699556"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.58496\/MJCSC\/2023\/002"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3143434.3143443"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3636243.3636257"},{"key":"e_1_3_2_17_2","unstructured":"Chinedu Wilfred Okonkwo and Abejide Ade-Ibijola. 2020. Python-bot: A chatbot for teaching python programming. Engineering Letters 29 1 (2020)."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1080\/10494820.2023.2172044"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.3390\/digital4010005"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3661167.3661221"},{"key":"e_1_3_2_21_2","unstructured":"Wiktionary contributors. 2025. methodical. Retrieved January 31 2025 from https:\/\/en.wiktionary.org\/wiki\/methodical"},{"key":"e_1_3_2_22_2","unstructured":"Wikipedia contributors. 2025. Socrates. Retrieved January 31 2025 from https:\/\/en.wikipedia.org\/wiki\/Socrates"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1177\/0894439305278972"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbusres.2020.12.058"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671458"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587102.3588815"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.4324\/9780203843123"},{"key":"e_1_3_2_28_2","unstructured":"DeepMind. 2022. AlphaCode. Retrieved January 31 2025 from https:\/\/alphacode.deepmind.com\/"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","unstructured":"Paul Denny James Prather Brett A. Becker James Finnie-Ansley Arto Hellas Juho Leinonen Andrew Luxton-Reilly Brent N. Reeves Eddie Antonio Santos and Sami Sarsa. 2024. Computing Education in the Era of Generative AI. Commun. ACM 67 2 (January 2024) 56\u201367. DOI:10.1145\/3624720","DOI":"10.1145\/3624720"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315015484"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1080\/14703297.2023.2195846"},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","unstructured":"Lucile Favero Juan Antonio P\u00e9rez-Ortiz Tanja K\u00e4ser and Nuria Oliver. 2025. Enhancing critical thinking in education by means of a Socratic chatbot. In AI in Education and Educational Research. Springer Nature Switzerland. Cham 17\u201332.","DOI":"10.1007\/978-3-031-93409-4_2"},{"key":"e_1_3_2_33_2","unstructured":"Flipkart. 1998. Flipkart. Retrieved January 31 2025 from https:\/\/www.flipkart.com\/"},{"key":"e_1_3_2_34_2","unstructured":"GitHub. 2025. GitHub Copilot. Retrieved January 31 2025 from https:\/\/github.com\/features\/copilot"},{"key":"e_1_3_2_35_2","unstructured":"Google. 1998. Google. Retrieved January 31 2025 from www.google.com"},{"key":"e_1_3_2_36_2","unstructured":"Google. 2023. Gemini. Retrieved January 31 2026 from https:\/\/gemini.google.com\/?hl=en-IN"},{"key":"e_1_3_2_37_2","unstructured":"Sudeesh Goriparthi and Research Scholar II. 2024. Advancing conversational AI: Best practices in prompt engineering for enhanced chatbot performance. International Journal of Dataanalytics 4(2024) 26\u201335."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.21125\/iceri.2023.2245"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1021\/acs.jchemed.3c00505"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.3102\/003465430298487"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.apnr.2018.06.015"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-023-04134-2"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.lisr.2006.06.005"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"Junfeng Jiao Saleh Afroogh Yiming Xu and Connor Phillips. 2025. Navigating llm ethics: Advancements challenges and future directions. AI and Ethics (2025) 1\u201325.","DOI":"10.1007\/s43681-025-00814-5"},{"key":"e_1_3_2_45_2","unstructured":"Ishika Joshi Ritvik Budhiraja Pranav Deepak Tanna Lovenya Jain Mihika Deshpande Arjun Srivastava Srinivas Rallapalli Harshal D. Akolekar Jagat Sesh Challa and Dhruv Kumar. 2023. \u201cWith Great Power Comes Great Responsibility!\u201d: Student and instructor perspectives on the influence of LLMs on undergraduate engineering education. Retrieved from https:\/\/arxiv.org\/abs\/2309.10694"},{"key":"e_1_3_2_46_2","doi-asserted-by":"crossref","unstructured":"Priyanka Kargupta Ishika Agarwal Dilek Hakkani-Tur and Jiawei Han. 2024. Instruct Not Assist: LLM-based Multi-Turn Planning and Hierarchical Questioning for Socratic Code Debugging. arXiv:2406.11709. Retrieved from https:\/\/arxiv.org\/abs\/2406.11709","DOI":"10.18653\/v1\/2024.findings-emnlp.553"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.lindif.2023.102274"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642773"},{"key":"e_1_3_2_49_2","unstructured":"Khan Academy. 2024. Khanmigo: AI-Powered Teaching and Tutoring Assistant. Retrieved November 15 2025 from https:\/\/www.khanacademy.org\/khan-labs"},{"key":"e_1_3_2_50_2","unstructured":"Oliver Knill Johnny Carlsson Andrew Chi and Mark Lezama. 2004. An artificial intelligence experiment in college math education. http:\/\/www.math.harvard.edu\/knill\/preprints\/sofia.pdf"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587102.3588785"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3631802.3631830"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3626252.3630938"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641554.3701945"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3649165.3690094"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.5958\/2347-7202.2022.00006.8"},{"key":"e_1_3_2_57_2","unstructured":"Meta. 2023. LLaMA: Large Language Model Meta AI. Retrieved January 31 2026 from https:\/\/www.llama.com\/"},{"key":"e_1_3_2_58_2","unstructured":"Manas Mhasakar Shikhar Sharma Apurv Mehra Utkarsh Venaik Ujjwal Singhal Dhruv Kumar and Kashish Mittal. 2024. Comuniqa: Exploring large language models for improving speaking skills. arXiv:2401.15595. Retrieved from https:\/\/arxiv.org\/abs\/2401.15595"},{"key":"e_1_3_2_59_2","unstructured":"Ministry of Education Government of India. 2025. National Institutional Ranking Framework (NIRF) 2025. Retrieved October 06 2025 from https:\/\/www.nirfindia.org\/Rankings\/2025\/OverallRanking.html"},{"issue":"133","key":"e_1_3_2_60_2","first-page":"14","article-title":"Codeforces as an educational platform for learning programming in digitalization","volume":"14","author":"Mirzayanov Mike","year":"2020","unstructured":"Mike Mirzayanov, Oksana Pavlova, Pavel MAVRIN, Roman Melnikov, Andrew Plotnikov, Vladimir Parfenov, and Andrew Stankevich. 2020. Codeforces as an educational platform for learning programming in digitalization. Olympiads in Informatics 14, 133-142 (2020), 14.","journal-title":"Olympiads in Informatics"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3688094"},{"key":"e_1_3_2_62_2","unstructured":"OpenAI. 2023. GPT-3.5. Retrieved January 31 2026 from https:\/\/chatgpt.com\/g\/g-F00faAwkE-open-a-i-gpt-3-5"},{"key":"e_1_3_2_63_2","unstructured":"OpenAI. 2024. GPT-4. Retrieved January 31 2026 from https:\/\/openai.com\/index\/gpt-4\/"},{"key":"e_1_3_2_64_2","unstructured":"OpenAI. 2025. GPT-X. Retrieved January 31 2025 from https:\/\/chatgpt.com\/g\/g-od9m9nZl3-gpt-x"},{"key":"e_1_3_2_65_2","unstructured":"OpenAI. 2025. OpenAI Codex. Retrieved January 31 2025 from https:\/\/openai.com\/index\/openai-codex\/"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.21125\/inted.2018.0854"},{"key":"e_1_3_2_67_2","doi-asserted-by":"crossref","unstructured":"Maciej Pankiewicz and Ryan S. Baker. 2023. Large language models (GPT) for automating feedback on programming assignments. arXiv:2307.00150. Retrieved from https:\/\/arxiv.org\/abs\/2307.00150","DOI":"10.58459\/icce.2023.950"},{"key":"e_1_3_2_68_2","volume-title":"The Thinker\u2019s Guide to Socratic Questioning","author":"Paul Richard","year":"2019","unstructured":"Richard Paul and Linda Elder. 2019. The Thinker\u2019s Guide to Socratic Questioning. Rowman and Littlefield."},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2021.102200"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2024.103407"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587103.3594206"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1177\/15586898221126816"},{"issue":"10","key":"e_1_3_2_73_2","first-page":"851","article-title":"ChatGPT is not capable of serving as an author: Ethical concerns and challenges of large language models in education","volume":"5","author":"Rane Nitin Liladhar","year":"2023","unstructured":"Nitin Liladhar Rane, Saurabh P. Choudhary, Abhijeet Tawde, and Jayesh Rane. 2023. ChatGPT is not capable of serving as an author: Ethical concerns and challenges of large language models in education. International Research Journal of Modernization in Engineering Technology and Science 5, 10 (2023), 851\u2013874.","journal-title":"International Research Journal of Modernization in Engineering Technology and Science"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA51224.2020.00036"},{"key":"e_1_3_2_75_2","unstructured":"Amazon Web Services. 2025. What is AWS CodeWhisperer? Retrieved January 31 2025 from https:\/\/docs.aws.amazon.com\/codewhisperer\/latest\/userguide\/what-is-cwspr.html"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.52214\/salt.v24i1.12869"},{"key":"e_1_3_2_77_2","volume-title":"The age of AI: Artificial Intelligence and the Future of Humanity","author":"Thacker Jason","year":"2020","unstructured":"Jason Thacker. 2020. The age of AI: Artificial Intelligence and the Future of Humanity. HarperChristian+ ORM."},{"key":"e_1_3_2_78_2","unstructured":"Haoye Tian Weiqi Lu Tsz On Li Xunzhu Tang Shing-Chi Cheung Jacques Klein and Tegawend\u00e9 F. Bissyand\u00e9. 2023. Is ChatGPT the ultimate programming assistant\u2013how far is it? arXiv:2304.11938. Retrieved from https:\/\/arxiv.org\/abs\/2304.11938"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-24322-7_39"},{"key":"e_1_3_2_80_2","unstructured":"James Walden Nicholas Caporusso and Ludiana Atnafu. [n. d.]. A Chatbot for teaching secure programming. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:271433014"},{"key":"e_1_3_2_81_2","unstructured":"Guanzhi Wang Yuqi Xie Yunfan Jiang Ajay Mandlekar Chaowei Xiao Yuke Zhu Linxi Fan and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models 2023. arXiv:2305.16291. Retrieved from https:\/\/arxiv.org\/abs\/2305.16291"},{"key":"e_1_3_2_82_2","unstructured":"Chen Xu Wenjie Wang Yuxin Li Liang Pang Jun Xu and Tat-Seng Chua. 2024. A study of implicit ranking unfairness in large language models. Retrieved from https:\/\/arxiv.org\/abs\/2311.07054"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1111\/bjet.13370"},{"issue":"6","key":"e_1_3_2_84_2","first-page":"256","article-title":"The use of Socratic method as a teaching\/learning tool to develop students\u2019 critical thinking: A review of Literature","volume":"15","author":"Zare Pezhman","year":"2015","unstructured":"Pezhman Zare and Jayakaran Mukundan. 2015. The use of Socratic method as a teaching\/learning tool to develop students\u2019 critical thinking: A review of Literature. Language in India 15, 6 (2015), 256\u2013265.","journal-title":"Language in India"},{"key":"e_1_3_2_85_2","unstructured":"Jingjing Zhao Tao Song and Yuxia Sun. 2020. APIHelper: Helping junior android programmers learn API usage. IAENG International Journal of Computer Science 47 1 (2020). Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:211550382"}],"container-title":["ACM Journal on Computing and Sustainable Societies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3788679","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T15:42:13Z","timestamp":1776094933000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3788679"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,13]]},"references-count":84,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3788679"],"URL":"https:\/\/doi.org\/10.1145\/3788679","relation":{},"ISSN":["2834-5533"],"issn-type":[{"value":"2834-5533","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,13]]},"assertion":[{"value":"2025-03-25","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-23","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}