{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T16:38:13Z","timestamp":1784392693278,"version":"3.55.0"},"reference-count":100,"publisher":"Association for Computing Machinery (ACM)","issue":"5","funder":[{"name":"Australian Research Council\u2019s Discovery Early Career Researcher Award","award":["DE210101091"],"award-info":[{"award-number":["DE210101091"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,5,31]]},"abstract":"<jats:p>Modern code review is a ubiquitous software quality assurance process aimed at identifying and resolving potential issues (e.g., functional, evolvability) within newly written code. Despite its effectiveness, the process demands large amounts of effort from the human reviewers involved. To help alleviate this workload, researchers have trained various deep learning-based language models to imitate human reviewers in providing natural language code reviews for submitted code. Formally, this automation task is known as code review comment generation. Prior work has demonstrated improvements in code review comment generation by leveraging machine learning techniques and neural models, such as transfer learning and the transformer architecture. However, the quality of the model-generated reviews remains sub-optimal due to the quality of the open-source code review data used in model training. This is in part due to the data obtained from open-source projects where code reviews are conducted in a public forum, and reviewers possess varying levels of software development experience, potentially affecting the quality of their feedback. To accommodate this variation, we propose a suite of experience-aware training methods that utilise the reviewers\u2019 past authoring and reviewing experiences as signals for review quality. Specifically, we propose experience-aware loss functions (ELF), which use the reviewers\u2019 authoring and reviewing ownership of a project as weights in the model\u2019s loss function. Through this method, experienced reviewers\u2019 code reviews yield larger influence over the model\u2019s behaviour. Compared to the SOTA model, ELF was able to generate higher quality reviews in terms of accuracy (e.g., +29% applicable comments), informativeness (e.g., +56% suggestions), and issue types discussed (e.g., +129% functional issues identified). The key contribution of this work is the demonstration of how traditional software engineering concepts such as reviewer experience can be integrated into the design of AI-based automated code review models.<\/jats:p>","DOI":"10.1145\/3762183","type":"journal-article","created":{"date-parts":[[2025,8,19]],"date-time":"2025-08-19T16:02:18Z","timestamp":1755619338000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Leveraging Reviewer Experience in Code Review Comment Generation"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-5368-8897","authenticated-orcid":false,"given":"Hong Yi","family":"Lin","sequence":"first","affiliation":[{"name":"The University of Melbourne, Melbourne, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6328-8839","authenticated-orcid":false,"given":"Patanamon","family":"Thongtanunam","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6919-2149","authenticated-orcid":false,"given":"Christoph","family":"Treude","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5500-025X","authenticated-orcid":false,"given":"Michael W.","family":"Godfrey","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-6778-0172","authenticated-orcid":false,"given":"Chunhua","family":"Liu","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9814-3514","authenticated-orcid":false,"given":"Wachiraphan","family":"Charoenwet","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,4,24]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3212695"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2009.59"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2016.7503718"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2013.6606617"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.2981898"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2003.1166590"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950323"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597082"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025119"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2015.21"},{"key":"e_1_3_2_12_2","first-page":"1003","volume-title":"NeurIPS","author":"Chang Haw-Shiuan","year":"2017","unstructured":"Haw-Shiuan Chang, Erik Learned-Miller, and Andrew McCallum. 2017. Active bias: training more accurate neural networks by emphasizing high variance samples. In NeurIPS. Curran Associates Inc., Red Hook, NY, 1003\u20131013. Retrieved from https:\/\/papers.nips.cc\/paper_files\/paper\/2017\/hash\/2f37d10131f2a483a8dd005b3d14b0d9-Abstract.html"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597108"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER50967.2021.00060"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.131"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1998.738485"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/N19-1423"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00093"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.APACOUST.2020.107740"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2018.00061"},{"key":"e_1_3_2_22_2","volume-title":"Statistical Methods for Rates and Proportions","author":"Fleiss Joseph L.","year":"2013","unstructured":"Joseph L. Fleiss, Bruce Levin, and Myunghee Cho Paik. 2013. Statistical Methods for Rates and Proportions. John Wiley & Sons, Hoboken, NJ."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/1287624.1287673"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3639477.3639746"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3632746"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3528452"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549098"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3528435"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616291"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3528228.3528406"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.JSS.2021.110911"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10664-022-10123-8"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623306"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10664-021-10038-W"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2012.6227193"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473134"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/2902362"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00107"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3613892"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10664-015-9393-5"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884840"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2015.7332457"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/52.469759"},{"key":"e_1_3_2_44_2","volume-title":"NeurIPS","author":"Lewis Patrick","year":"2020","unstructured":"Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\u00fcttler, Mike Lewis, Wen-Tau Yih, Tim Rockt\u00e4schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In NeurIPS. Curran Associates Inc., Red Hook, NY, Article 793, 16 pages. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/hash\/6b493230205f780e1bc26945df7481e5-Abstract.html"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597207"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549099"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-16440-8_55"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549081"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616339"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00075"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644910"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE59848.2023.00026"},{"key":"e_1_3_2_53_2","volume-title":"NeurIPS Datasets","author":"Lu Shuai","year":"2021","unstructured":"Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. 2021. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. In NeurIPS Datasets. Curran Associates Inc., Red Hook, NY. Retrieved from https:\/\/datasets-benchmarks-proceedings.neurips.cc\/paper\/2021\/hash\/c16a5320fa475530d9583c34fd356ef5-Abstract-round1.html"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.265100500"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597076"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2013.36"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2006.31"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/222124.222136"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2008.71"},{"key":"e_1_3_2_60_2","first-page":"1017","volume-title":"NeurIPS","author":"Needell Deanna","year":"2014","unstructured":"Deanna Needell, Nathan Srebro, and Rachel Ward. 2014. Stochastic gradient descent, weighted sampling, and the randomized kaczmarz algorithm. In NeurIPS. MIT Press, Cambridge, MA, 1017\u20131025. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2014\/hash\/b3310bba2be31e673a7ded3386994599-Abstract.html"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/78607.78611"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073135"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00036"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/2377656.2377660"},{"key":"e_1_3_2_65_2","volume-title":"Improving Language Understanding by Generative Pre-Training","author":"Radford Alec","year":"2018","unstructured":"Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving Language Understanding by Generative Pre-Training. Technical Report. OpenAI."},{"issue":"140","key":"e_1_3_2_66_2","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21, 140 (2020), 1\u201367.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985860"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.17"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544902.3546639"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884848"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/2491411.2491444"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1987.233165"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643775"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3264598"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(98)10055-9"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","unstructured":"Patanamon Thongtanunam Shane McIntosh Ahmed E. Hassan and Hajimu Iida. 2015. Investigating code review practices in defective files: An empirical study of the Qt system. In MSR. ACM New York NY 168\u2013179. DOI: 10.1109\/MSR.2015.23","DOI":"10.1109\/MSR.2015.23"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884852"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510067"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE62328.2024.00044"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2015.7081824"},{"key":"e_1_3_2_81_2","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re Naman Goyal Eric Hambro Faisal Azhar et al. 2023. LLaMA: Open and efficient foundation language models. arXiv:2302.13971. Retrieved from https:\/\/arxiv.org\/abs\/2302.13971"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00021"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340544"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3348172"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510621"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00027"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10664-023-10411-X"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM56168.2023.10304851"},{"key":"e_1_3_2_89_2","first-page":"5998","volume-title":"NeurIPS","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. Curran Associates Inc., Red Hook, NY, 5998\u20136008. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/3664646.3665664"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32226-7_35"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP43922.2022.9747797"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33015361"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2023.3249754"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/3274451"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.1997.624574"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616245"},{"key":"e_1_3_2_98_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2015.2500238"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534390"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-024-10484-2"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639222"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3762183","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T06:33:33Z","timestamp":1777098813000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3762183"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,24]]},"references-count":100,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,5,31]]}},"alternative-id":["10.1145\/3762183"],"URL":"https:\/\/doi.org\/10.1145\/3762183","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,24]]},"assertion":[{"value":"2024-09-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-10","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}