{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T21:37:04Z","timestamp":1770241024209,"version":"3.49.0"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2024,7,12]]},"abstract":"<jats:p>Code reviews are an ubiquitous and essential part of the software development process. They also offer a unique, at-scale opportunity for teaching developers in the context of their day-to-day development activities versus something more removed and formal, like a class. Yet there is little research on effective teaching through code reviews: focusing on learning for the author and not just changes to the code. We address this gap through a case study at Google: interviews with 14 developers revealed 12 patterns and 15 anti-patterns in code reviews that impact learning. For instance, explanatory rationale, sample solutions backed by standards, and a constructive tone facilitates learning, whereas harsh comments, excessive shallow critiques, and nonpragmatic reviewing that ignores authors\u2019 constraints hinders learning. We validated our qualitative findings through member checking, interviews with code reviewers, a literature review, and a survey of 324 developers. This comprehensive study provides an empirical evidence of how social dynamics in code reviews impact learning. Based on our findings, we provide practical recommendations on how to frame constructive reviews to create a supportive learning environment.<\/jats:p>","DOI":"10.1145\/3660764","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T10:22:09Z","timestamp":1720779729000},"page":"1262-1283","source":"Crossref","is-referenced-by-count":1,"title":["Effective Teaching through Code Reviews: Patterns and Anti-patterns"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1859-1692","authenticated-orcid":false,"given":"Anita","family":"Sarma","sequence":"first","affiliation":[{"name":"Oregon State University, Corvallis, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8980-2779","authenticated-orcid":false,"given":"Nina","family":"Chen","sequence":"additional","affiliation":[{"name":"Google, Seattle, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,7,12]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2017.8115623"},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","unstructured":"Alberto Bacchelli and Christian Bird. 2013. Expectations outcomes and challenges of modern code review. In 2013 35th International Conference on Software Engineering (ICSE). IEEE 712\u2013721.","DOI":"10.1109\/ICSE.2013.6606617"},{"key":"e_1_3_1_4_2","doi-asserted-by":"crossref","unstructured":"Shaowen Bardzell and Jeffrey Bardzell. 2011. Towards a feminist HCI methodology: social science feminism and HCI. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 675\u2013684.","DOI":"10.1145\/1978942.1979041"},{"key":"e_1_3_1_5_2","doi-asserted-by":"crossref","unstructured":"Tobias Baum Olga Liskin Kai Niklas and Kurt Schneider. 2016. Factors influencing code review processes in industry. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. 85\u201396.","DOI":"10.1145\/2950290.2950323"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9366-8"},{"key":"e_1_3_1_7_2","volume-title":"Research Methods in Anthropology: Qualitative and Quantitative Approaches","author":"Bernard H Russell","year":"2017","unstructured":"H Russell Bernard. 2017. Research Methods in Anthropology: Qualitative and Quantitative Approaches. Rowman & Littlefield."},{"key":"e_1_3_1_8_2","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1109\/MSR.2015.25","volume-title":"2015 IEEE\/ACM 12th Working Conference on Mining Software Repositories","author":"Bird Christian","year":"2015","unstructured":"Christian Bird, Trevor Carnahan, and Michaela Greiler. 2015. Lessons learned from building and deploying a code review analytics platform. In 2015 IEEE\/ACM 12th Working Conference on Mining Software Repositories. IEEE, 191\u2013201."},{"key":"e_1_3_1_9_2","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1109\/ESEM.2013.23","volume-title":"2013 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement","author":"Bosu Amiangshu","year":"2013","unstructured":"Amiangshu Bosu and Jeffrey C Carver. 2013. Impact of peer code review on peer impression formation: A survey. In 2013 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE, 133\u2013142."},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2576451"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2015.21"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1037\/qup0000196"},{"key":"e_1_3_1_14_2","volume-title":"Software Engineering at Google: Lessons Learned from Programming Over Time","author":"Chen Nina","year":"2020","unstructured":"Nina Chen and Mark Barolak. 2020. Knowledge Sharing. In Software Engineering at Google: Lessons Learned from Programming Over Time, Titus Winters, Tom Manshreck, and Hyrum Wright (Eds.). O\u2019Reilly Media, Chapter 3."},{"key":"e_1_3_1_15_2","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1109\/SANER50967.2021.00060","volume-title":"2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","author":"Chouchen Moataz","year":"2021","unstructured":"Moataz Chouchen, Ali Ouni, Raula Gaikovina Kula, Dong Wang, Patanamon Thongtanunam, Mohamed Wiem Mkaouer, and Kenichi Matsumoto. 2021. Anti-patterns in modern code review: Symptoms and prevalence. In 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 531\u2013535."},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.110951"},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1007\/978-1-84800-044-5_11","volume-title":"In Guide to Advanced Empirical Software Engineering","author":"Easterbrook Steve","year":"2008","unstructured":"Steve Easterbrook, Janice Singer, Margaret-Anne Storey, and Daniela Damian. 2008. Selecting empirical methods for software engineering research. In Guide to Advanced Empirical Software Engineering. Springer, 285\u2013311."},{"key":"e_1_3_1_18_2","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1109\/SANER.2019.8668024","volume-title":"2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER)","author":"Ebert Felipe","year":"2019","unstructured":"Felipe Ebert, Fernando Castor, Nicole Novielli, and Alexander Serebrenik. 2019. Confusion in code reviews: Reasons, impacts, and coping strategies. In 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 49\u201360."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2019.06.005"},{"key":"e_1_3_1_20_2","doi-asserted-by":"crossref","first-page":"797","DOI":"10.1145\/3540250.3549167","article-title":"A Case Study of Implicit Mentoring, Its Prevalence, and Impact in Apache","author":"Feng Zixuan","year":"2022","unstructured":"Zixuan Feng, Amreeta Chatterjee, Anita Sarma, and Iftekhar Ahmed. 2022. A Case Study of Implicit Mentoring, Its Prevalence, and Impact in Apache. Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 797\u2013809.","journal-title":"Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering"},{"key":"e_1_3_1_21_2","doi-asserted-by":"crossref","unstructured":"Daniel M German Gregorio Robles Germ\u00e1n Poo-Caama\u00f1o Xin Yang Hajimu Iida and Katsuro Inoue. 2018. \u201cWas my contribution fairly reviewed?\u201d: a framework to study the perception of fairness in modern code reviews. In Proceedings of the 40th International Conference on Software Engineering. 523\u2013534.","DOI":"10.1145\/3180155.3180217"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","unstructured":"Daniel Graziotin Fabian Fagerholm Xiaofeng Wang and Pekka Abrahamsson. 2017. Unhappy Developers: Bad for Themselves Bad for Process and Bad for Software Product. In 2017 IEEE\/ACM 39th International Conference on Software Engineering Companion (ICSE-C). 362\u2013364. https:\/\/doi.org\/10.1109\/ICSE-C.2017.104 10.1109\/ICSE-C.2017.104","DOI":"10.1109\/ICSE-C.2017.104"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3555183"},{"key":"e_1_3_1_24_2","first-page":"1028","article-title":"Code review quality: How developers see it","author":"Kononenko Oleksii","year":"2016","unstructured":"Oleksii Kononenko, Olga Baysal, and Michael W Godfrey. 2016. Code review quality: How developers see it. In Proceedings of the 38th International Conference on Software Engineering. 1028\u20131038.","journal-title":"Proceedings of the 38th International Conference on Software Engineering"},{"key":"e_1_3_1_25_2","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1109\/ICSM.2015.7332457","volume-title":"2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Kononenko Oleksii","year":"2015","unstructured":"Oleksii Kononenko, Olga Baysal, Latifa Guerrouj, Yaxin Cao, and Michael W Godfrey. 2015. Investigating code review quality: Do people and participation matter?. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 111\u2013120."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.265100500"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9381-9"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474097"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1002\/ejsp.2420150303"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3274404"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2019.8667987"},{"key":"e_1_3_1_32_2","doi-asserted-by":"crossref","unstructured":"Huilian Sophie Qiu Bogdan Vasilescu Christian K\u00e4stner Carolyn Egelman Ciera Jaspan and Emerson Murphy-Hill. 2022. Detecting interpersonal conflict in issues and code review: cross pollinating open-and closed-source approaches. In Proceedings of the 2022 ACM\/IEEE 44th International Conference on Software Engineering: Software Engineering in Society. 41\u201355.","DOI":"10.1145\/3510458.3513019"},{"key":"e_1_3_1_33_2","doi-asserted-by":"crossref","unstructured":"Achyudh Ram Anand Ashok Sawant Marco Castelluccio and Alberto Bacchelli. 2018. What makes a code change easier to review: an empirical investigation on code change reviewability. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 201\u2013212.","DOI":"10.1145\/3236024.3236080"},{"key":"e_1_3_1_34_2","doi-asserted-by":"crossref","unstructured":"Peter C Rigby and Christian Bird. 2013. Convergent contemporary software peer review practices. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. 202\u2013212.","DOI":"10.1145\/2491411.2491444"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-018-9646-1"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","unstructured":"Caitlin Sadowski Emma S\u00f6derberg Luke Church Michal Sipko and Alberto Bacchelli. 2018. Modern Code Review: A Case Study at Google. In International Conference on Software Engineering Software Engineering in Practice track (ICSE SEIP).","DOI":"10.1145\/3183519.3183525"},{"key":"e_1_3_1_37_2","doi-asserted-by":"crossref","unstructured":"Jaydeb Sarker. 2022. Identification and Mitigation of Toxic Communications Among Open Source Software Developers. In Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering. 1\u20135.","DOI":"10.1145\/3551349.3559570"},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","unstructured":"Jaydeb Sarker Sayma Sultana Steven R Wilson and Amiangshu Bosu. 2023. ToxiSpanSE: An Explainable Toxicity Detection in Code Review Comments. The 17th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) (2023) to appear.","DOI":"10.1109\/ESEM56168.2023.10304855"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","unstructured":"Anita Sarma and Nina Chen. 2024. Effective Teaching through Code Reviews: Patterns and Anti-Patterns. https:\/\/doi.org\/10.5281\/zenodo.11212083 10.5281\/zenodo.11212083","DOI":"10.5281\/zenodo.11212083"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09858-z"},{"key":"e_1_3_1_41_2","doi-asserted-by":"crossref","first-page":"768","DOI":"10.1007\/s10664-016-9452-6","article-title":"Review participation in modern code review: An empirical study of the android, Qt, and OpenStack projects.","volume":"22","author":"Thongtanunam Patanamon","year":"2017","unstructured":"Patanamon Thongtanunam, Shane McIntosh, Ahmed E Hassan, and Hajimu Iida. 2017. Review participation in modern code review: An empirical study of the android, Qt, and OpenStack projects. Empirical Software Engineering 22 (2017), 768\u2013817.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_1_42_2","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1145\/3422392.3422459","volume-title":"Proceedings of the 34th Brazilian symposium on software engineering","author":"Wessel Mairieli","year":"2020","unstructured":"Mairieli Wessel, Alexander Serebrenik, Igor Wiese, Igor Steinmacher, and Marco A Gerosa. 2020. What to Expect from Code Review Bots on GitHub? A Survey with OSS Maintainers. In Proceedings of the 34th Brazilian symposium on software engineering. Association for Computing Machinery, New York, NY, USA, 457\u2013462."},{"key":"e_1_3_1_43_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3579471","article-title":"Competencies for code review","volume":"7","author":"Gon\u00e7alves Pavl\u00edna Wurzel","year":"2023","unstructured":"Pavl\u00edna Wurzel Gon\u00e7alves, G\u00fcl Calikli, Alexander Serebrenik, and Alberto Bacchelli. 2023. Competencies for code review. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1\u201333.","journal-title":"Proceedings of the ACM on Human-Computer Interaction"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3660764","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3660764","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T08:06:36Z","timestamp":1770192396000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3660764"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,12]]},"references-count":42,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2024,7,12]]}},"alternative-id":["10.1145\/3660764"],"URL":"https:\/\/doi.org\/10.1145\/3660764","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,12]]}}}