{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,9]],"date-time":"2026-02-09T08:44:17Z","timestamp":1770626657946,"version":"3.49.0"},"reference-count":118,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2025,12,31]]},"abstract":"<jats:p>Programming demands a variety of cognitive skills, and mastering these competencies is essential for success in computer science education. The importance of formative feedback is well acknowledged in programming education, and thus, a diverse range of techniques has been proposed to generate and enhance formative feedback for programming exercises. This article reviews state-of-the-art automated feedback generation techniques and categorizes the various approaches based on the underlying computational techniques, programming languages, the kind of programming errors they deal with, and the type of feedback they provide. It covers data-driven techniques, those which use program repair methods, machine learning-based techniques, and techniques based around the use of large language models, particularly noting the rapid uptake of the latter. The article provides a summary of key findings and challenges, alongside recommendations for future work. The findings reveal that although there exist numerous tools for automated programming feedback, many studies depend on non-public benchmarks, which limits reproducibility and independent evaluation of the tools and their datasets. Additionally, tools are not always language agnostic and in some cases involve complex configuration steps. Large language models have demonstrated transformative potential in generating feedback. However, most research has focused on introductory courses (CS1 and CS2) indicating the need to apply them in advanced fields like machine learning and image processing. Although large language models have outperformed traditional approaches, challenges related to hallucinations and incorrect responses still need to be addressed as precision is critical in a pedagogical setting. Most of the studies use proprietary models that lack transparency and customization options, emphasizing the need for further research into open LLM alternatives.<\/jats:p>","DOI":"10.1145\/3764593","type":"journal-article","created":{"date-parts":[[2025,9,2]],"date-time":"2025-09-02T15:22:11Z","timestamp":1756826531000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Navigating the Landscape of Automated Feedback Generation Techniques for Programming Exercises"],"prefix":"10.1145","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7583-1795","authenticated-orcid":false,"given":"Smitha S.","family":"Kumar","sequence":"first","affiliation":[{"name":"School of Mathematical and Computer Sciences, Heriot-Watt University, Dubai, United Arab Emirates"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2745-9896","authenticated-orcid":false,"given":"Michael A.","family":"Lones","sequence":"additional","affiliation":[{"name":"Heriot-Watt University, Edinburgh, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6233-6341","authenticated-orcid":false,"given":"Manuel","family":"Maarek","sequence":"additional","affiliation":[{"name":"Heriot-Watt University, Edinburgh, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3804-0513","authenticated-orcid":false,"given":"Hind","family":"Zantout","sequence":"additional","affiliation":[{"name":"School of Mathematical and Computer Sciences, Heriot-Watt University, Dubai, United Arab Emirates"}]}],"member":"320","published-online":{"date-parts":[[2025,10,22]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510418"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3183377.3183383"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00039"},{"key":"e_1_3_1_5_2","unstructured":"Desai Ankur and Deo Atul. 2022. Amazon CodeWhisperer. Retrieved August 21 2023 from https:\/\/aws.amazon.com\/blogs\/machine-learning\/introducing-amazon-codewhisperer-the-ml-powered-coding-companion\/\/"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3627217.3627233"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587102.3588852"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3344429.3372508"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287324.3287432"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324888"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2538862.2538924"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/2351676.2351752"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.caeai.2021.100027"},{"key":"e_1_3_1_14_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374. Retrieved from https:\/\/arxiv.org\/abs\/2107.03374"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-52237-7_9"},{"key":"e_1_3_1_16_2","first-page":"83","volume-title":"Intelligent Tutoring Systems: At the Crossroads of Artificial Intelligence and Education","author":"Corbett Albert T.","year":"1990","unstructured":"Albert T. Corbett, John R. Anderson, and Eric G. Patterson. 1990. Student modeling and tutoring flexibility in the Lisp Intelligent Tutoring System. In Intelligent Tutoring Systems: At the Crossroads of Artificial Intelligence and Education. C. Frasson and G. Gauthier (Eds.), Ablex Norwood, NJ, 83\u2013106."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366860"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","unstructured":"Rajdeep Das Umair Z. Ahmed Amey Karkare and Sumit Gulwani. 2016. Prutor: A system for tutoring CS1 and collecting student programs for analysis. arXiv:1608.03828. Retrieved from 10.48550\/arXiv.1608.03828","DOI":"10.48550\/arXiv.1608.03828"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CogMI48466.2019.00018"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/2593735.2593740"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3291279.3339402"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2325296.2325318"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/1163405.1163409"},{"key":"e_1_3_1_24_2","unstructured":"Thomas Durieux and Martin Monperrus. 2016. IntroClassJava: A benchmark of 297 small and buggy Java programs. Retrieved from https:\/\/hal.archives-ouvertes.fr\/hal-01272126"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287324.3287474"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3408877.3437225"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/1384271.1384371"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3160489.3160493"},{"key":"e_1_3_1_29_2","unstructured":"Daniel Fried Armen Aghajanyan Jessy Lin Sida Wang Eric Wallace Freda Shi Ruiqi Zhong Wen Tau Yih Luke Zettlemoyer and Mike Lewis. 2023. InCoder: A generative model for code infilling and synthesis. arXiv:2204.05999. Retrieved from https:\/\/arxiv.org\/abs\/2204.05999"},{"key":"e_1_3_1_30_2","unstructured":"Nat Friedman. 2021. Introducing GitHub Copilot: Your AI Pair Programmer. GitHub. Retrieved from https:\/\/github.blog\/news-insights\/product-news\/introducing-github-copilot-ai-pair-programmer\/"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/1352135.1352226"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3192366.3192387"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.3301930"},{"key":"e_1_3_1_34_2","first-page":"1345","volume-title":"Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI \u201917)","author":"Gupta Rahul","year":"2017","unstructured":"Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish Shevade. 2017. DeepFix: Fixing common C language errors by deep learning. In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI \u201917). AAAI Press, San Francisco, California, 1345\u20131351."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3578527.3578535"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1002\/cl2.1230"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.5555\/AAI28771074"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","unstructured":"Qiang Hao Jack P. Wilson Camille Ottaway Naitra Iriumi Kai Arakawa and David H. Smith. 2019. Investigating the essential of meaningful automated formative feedback for programming assignments. In Proceedings of the 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL\/HCC). IEEE Memphis TN 151\u2013155. DOI: 10.1109\/VLHCC.2019.8818922","DOI":"10.1109\/VLHCC.2019.8818922"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3568813.3600139"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/1268784.1268805"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.3991\/ijet.v13i09.8023"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00044"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/1930464.1930480"},{"issue":"8","key":"e_1_3_1_44_2","first-page":"1049","article-title":"A critical review on recent proposed automated programming assessment tool","volume":"57","author":"Ismail Huzaifah","year":"2020","unstructured":"Huzaifah Ismail and Muhammad Modi Lakulu. 2020. A critical review on recent proposed automated programming assessment tool. Psychology and Education 57, 8 (2020), 1049\u20131060.","journal-title":"Psychology and Education"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/FIE.2005.1611967"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33263-0_42"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571785.3574124"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580919"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3231711"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-36272-9_74"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3622780.3623648"},{"issue":"2","key":"e_1_3_1_52_2","first-page":"109","article-title":"A review of techniques in automatic programming assessment for practical skill test","volume":"10","author":"Lajis Adidah","year":"2018","unstructured":"Adidah Lajis, Shahidatul Arfah Baharudin, Diyana Ab Kadir, Nadilah Mohd Ralim, Haidawati Mohd Nasir, and Normaziah Abdul Aziz. 2018. A review of techniques in automatic programming assessment for practical skill test. Journal of Telecommunication, Electronic and Computer Engineering (JTEC) 10, 2\u20135, (Jul. 2018), 109\u2013113. Retrieved from https:\/\/jtec.utem.edu.my\/jtec\/article\/view\/4394","journal-title":"Journal of Telecommunication, Electronic and Computer Engineering (JTEC)"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-00293-4_20"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2011.104"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2021.3072577"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3408877.3432463"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3276528"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3545945.3569770"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-15-1899-7_4"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3631802.3631830"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3135932.3135941"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/2914770.2837617"},{"key":"e_1_3_1_63_2","unstructured":"Yunlong Lu Na Meng and Wenxin Li. 2021. FAPR: Fast and accurate program repair for introductory programming courses. Retrieved from http:\/\/arxiv.org\/abs\/2107.06550"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293881.3295779"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2948705"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/TLT.2022.3180984"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372782.3406264"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3304221.3319759"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3469885"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3335814"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/FIE.2014.7044420"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884807"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1109\/TE.2018.2864133"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3636515"},{"key":"e_1_3_1_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3105906"},{"key":"e_1_3_1_76_2","first-page":"125","volume-title":"Handbook of Research on Educational Communications and Technology","author":"Narciss Susanne","year":"2008","unstructured":"Susanne Narciss. 2008. Feedback strategies for interactive learning tasks. In Handbook of Research on Educational Communications and Technology. J. Michael Spector, M. David Merrill, Jeroen van Merrienboer, and Marcy P. Driscoll (Eds.), Routledge, Milton Park, 125\u2013143."},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICALT.2014.38"},{"key":"e_1_3_1_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3430665.3456344"},{"key":"e_1_3_1_79_2","unstructured":"OpenAI. 2022. Introducing ChatGPT. OpenAI. Retrieved August 21 2023 from https:\/\/openai.com\/blog\/chatgpt"},{"key":"e_1_3_1_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587102.3588794"},{"key":"e_1_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics12102254"},{"key":"e_1_3_1_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3513140"},{"key":"e_1_3_1_83_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dib.2023.108887"},{"key":"e_1_3_1_84_2","doi-asserted-by":"publisher","DOI":"10.58459\/ICCE.2023.950"},{"key":"e_1_3_1_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3059009.3059026"},{"key":"e_1_3_1_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/3059009.3059058"},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.8115653"},{"key":"e_1_3_1_88_2","first-page":"192","volume-title":"Proceedings of the 10th International Conference on Educational Data Mining","author":"Price Thomas","year":"2017","unstructured":"Thomas Price, Rui Zhi, and Tiffany Barnes. 2017. Evaluation of a data-driven feedback algorithm for open-ended programming. In Proceedings of the 10th International Conference on Educational Data Mining. International Educational Data Mining Society, Wuhan, China, 192\u2013197."},{"key":"e_1_3_1_89_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-019-00177-z"},{"key":"e_1_3_1_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/3430895.3460159"},{"key":"e_1_3_1_91_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-015-0070-z"},{"key":"e_1_3_1_92_2","doi-asserted-by":"publisher","DOI":"10.1017\/9781108654555.013"},{"key":"e_1_3_1_93_2","doi-asserted-by":"publisher","DOI":"10.15388\/infedu.2023.09"},{"key":"e_1_3_1_94_2","first-page":"1186","volume-title":"Proceedings of the 2010 International Symposium on Information Technology","volume":"3","author":"Romli Rohaida","year":"2010","unstructured":"Rohaida Romli, Shahida Sulaiman, and Kamal Zuhairi Zamli. 2010. Automatic programming assessment and test data generation a review on its approaches. In Proceedings of the 2010 International Symposium on Information Technology, Vol. 3, 1186\u20131192. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:18222797"},{"key":"e_1_3_1_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/3385412.3386005"},{"key":"e_1_3_1_96_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2018.8330219"},{"key":"e_1_3_1_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/3576882.3617909"},{"key":"e_1_3_1_98_2","doi-asserted-by":"publisher","DOI":"10.1109\/APR59189.2023.00012"},{"key":"e_1_3_1_99_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468598"},{"key":"e_1_3_1_100_2","doi-asserted-by":"publisher","DOI":"10.1109\/CSEET.2016.48"},{"key":"e_1_3_1_101_2","first-page":"71","volume-title":"Proceedings of the 9th International Conference on Intelligent Tutoring Systems Young Researchers Track","author":"Stamper John","year":"2008","unstructured":"John Stamper, Tiffany Barnes, Lorrie Lehmann, and Marvin Croy. 2008. The hint factory: Automatic generation of contextualized help for existing computer aided instruction. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems Young Researchers Track. Montreal, Canada, 71\u201378. Retrieved from https:\/\/dev.stamper.org\/publications\/ITS2008Stamper.pdf"},{"key":"e_1_3_1_102_2","unstructured":"Andrew Taylor Alexandra Vassar Jake Renzella and Hammond Pearce. 2023. Dcc\u2013help: Generating context-aware compiler error explanations with large language models. arXiv:2308.11873. Retrieved from https:\/\/arxiv.org\/abs\/2308.11873"},{"key":"e_1_3_1_103_2","doi-asserted-by":"publisher","DOI":"10.1145\/3626252.3630822"},{"key":"e_1_3_1_104_2","doi-asserted-by":"publisher","DOI":"10.1109\/FIE44824.2020.9274053"},{"key":"e_1_3_1_105_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10639-017-9611-4"},{"key":"e_1_3_1_106_2","doi-asserted-by":"publisher","DOI":"10.1080\/00461520.2011.611369"},{"key":"e_1_3_1_107_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-022-14053-x"},{"key":"e_1_3_1_108_2","doi-asserted-by":"publisher","DOI":"10.1145\/3192366.3192384"},{"key":"e_1_3_1_109_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341525.3387411"},{"key":"e_1_3_1_110_2","doi-asserted-by":"publisher","DOI":"10.1145\/2591708.2591749"},{"key":"e_1_3_1_111_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2013.6693094"},{"key":"e_1_3_1_112_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00129"},{"key":"e_1_3_1_113_2","unstructured":"Chunqiu Steven Xia and Lingming Zhang. 2023. Conversational automated program repair. arXiv:2301.13246. Retrieved from https:\/\/arxiv.org\/abs\/2301.13246"},{"key":"e_1_3_1_114_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680323"},{"key":"e_1_3_1_115_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623337"},{"key":"e_1_3_1_116_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106262"},{"key":"e_1_3_1_117_2","unstructured":"Jialu Zhang Jos\u00e9 Cambronero Sumit Gulwani Vu Le Ruzica Piskac Gustavo Soares and Gust Verbruggen. 2022. Repairing bugs in python assignments using large language models. arXiv:2209.14876. Retrieved from https:\/\/arxiv.org\/abs\/2209.14876"},{"key":"e_1_3_1_118_2","doi-asserted-by":"publisher","DOI":"10.1145\/3649850"},{"key":"e_1_3_1_119_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.54"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3764593","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T14:48:07Z","timestamp":1761144487000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3764593"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,22]]},"references-count":118,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12,31]]}},"alternative-id":["10.1145\/3764593"],"URL":"https:\/\/doi.org\/10.1145\/3764593","relation":{},"ISSN":["1946-6226"],"issn-type":[{"value":"1946-6226","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,22]]},"assertion":[{"value":"2023-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-31","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}