{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:37:05Z","timestamp":1785537425322,"version":"3.56.0"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2023,4,14]],"date-time":"2023-04-14T00:00:00Z","timestamp":1681430400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"crossref","award":["PP00P2_170529"],"award-info":[{"award-number":["PP00P2_170529"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2023,4,14]]},"abstract":"<jats:p>Peer code review is a widely practiced software engineering process in which software developers collaboratively evaluate and improve source code quality. Whether developers can perform good reviews depends on whether they have sufficient competence and experience. However, the knowledge of what competencies developers need to execute code review is currently limited, thus hindering, for example, the creation of effective support tools and training strategies. To address this gap, we firstly identified 27 competencies relevant to performing code review through expert validation. Later, we conducted an online survey with 105 reviewers to rank these competencies along four dimensions: frequency of usage, importance, proficiency, and desire of reviewers to improve in that competency. The survey shows that technical competencies are considered essential to performing reviews and that respondents feel generally confident in their technical proficiency. Moreover, reviewers feel less confident in how to communicate clearly and give constructive feedback - competencies they consider like-wise an essential part of reviewing. Therefore, research and education should focus in more detail on how to support and develop reviewers' potential to communicate effectively during reviews. In the paper, we also discuss further implications for training, code review performance assessment, and reviewers of different experience level.<\/jats:p>\n          <jats:p>Data and materials: https:\/\/doi.org\/10.5281\/zenodo.7401313<\/jats:p>","DOI":"10.1145\/3579471","type":"journal-article","created":{"date-parts":[[2023,4,16]],"date-time":"2023-04-16T17:23:07Z","timestamp":1681665787000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Competencies for Code Review"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2231-054X","authenticated-orcid":false,"given":"Pavl\u00edna","family":"Wurzel Gon\u00e7alves","sequence":"first","affiliation":[{"name":"University of Zurich, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4578-1747","authenticated-orcid":false,"given":"G\u00fcl","family":"Calikli","sequence":"additional","affiliation":[{"name":"University of Glasgow, Glasgow, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1418-0095","authenticated-orcid":false,"given":"Alexander","family":"Serebrenik","sequence":"additional","affiliation":[{"name":"Eindhoven University of Technology, Eindhoven, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0193-6823","authenticated-orcid":false,"given":"Alberto","family":"Bacchelli","sequence":"additional","affiliation":[{"name":"University of Zurich, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,4,16]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Software engineering body of knowledge","author":"Abran Alain","year":"2004","unstructured":"Alain Abran, James W Moore, Pierre Bourque, Robert Dupuis, and L Tripp. 2004. Software engineering body of knowledge. IEEE Computer Society, Angela Burgess (2004)."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.28121"},{"key":"e_1_2_2_3_1","unstructured":"Hiyam Al-Kilidar Tor Stalhane Cat Kutay and Ross Jeffery. 2003. Teaching the Process of Code Review. (2003)."},{"key":"e_1_2_2_4_1","volume-title":"The state of research on software engineering competencies: A systematic mapping study. Journal of Systems and Software","author":"Assyne Nana","year":"2021","unstructured":"Nana Assyne, Hadi Ghanbari, and Mirja Pulkkinen. 2021. The state of research on software engineering competencies: A systematic mapping study. Journal of Systems and Software (2021), 111183."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2013.6606617"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-018-9310-8"},{"key":"e_1_2_2_7_1","volume-title":"The SHL universal competency framework. SHL White paper","author":"Bartram Dave","year":"2006","unstructured":"Dave Bartram. 2006. The SHL universal competency framework. SHL White paper (2006), 1--8."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","unstructured":"Tobias Baum. 2019. Cognitive-support code review tools: improved efficiency of change-based code review by guiding and assisting reviewers. Ph. D. Dissertation. Hannover: Institutionelles Repositorium der Universit\"at Hannover. https:\/\/doi.org\/10.15488\/9164","DOI":"10.15488\/9164"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1080\/13636820903194682"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/1049732316654870"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2015.21"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-018-9642-5"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.2156"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1177\/1523422305286156"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0883-0355(97)81235-9"},{"key":"e_1_2_2_16_1","volume-title":"Modern Code Review","author":"Cohen Jason","unstructured":"Jason Cohen. 2010. Modern Code Review. In Making Software, Andy Oram and Greg Wilson (Eds.). O'Reilly, Chapter 18, 329--338."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2012.04.021"},{"key":"e_1_2_2_18_1","volume-title":"Designing and Evaluating Screening Questions for Online Surveys with Programmers. In 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 537--548","author":"Danilova Anastasia","year":"2021","unstructured":"Anastasia Danilova, Alena Naiakshina, Stefan Horstmann, and Matthew Smith. 2021. Do you really code? Designing and Evaluating Screening Questions for Online Surveys with Programmers. In 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 537--548."},{"key":"e_1_2_2_19_1","volume-title":"A systematic literature review and taxonomy of modern code review. Journal of Systems and Software","author":"Davila Nicole","year":"2021","unstructured":"Nicole Davila and Ingrid Nunes. 2021. A systematic literature review and taxonomy of modern code review. Journal of Systems and Software (2021), 110951."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00094"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2015.7081844"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09909-5"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380414"},{"key":"e_1_2_2_24_1","volume-title":"Collin Green, Ciera Jaspan, and James Lin.","author":"Egelman Carolyn D","year":"2020","unstructured":"Carolyn D Egelman, Emerson Murphy-Hill, Elizabeth Kammer, Maggie Morrow Hodges, Collin Green, Ciera Jaspan, and James Lin. 2020b. Pushbacacterizing and Detecting Negative Interpersonal Interactions in Code Review. (2020)."},{"key":"e_1_2_2_25_1","volume-title":"Expert and exceptional performance: Evidence of maximal adaptation to task constraints. Annual review of psychology","author":"Anders Ericsson K","year":"1996","unstructured":"K Anders Ericsson and Andreas C Lehmann. 1996. Expert and exceptional performance: Evidence of maximal adaptation to task constraints. Annual review of psychology, Vol. 47, 1 (1996), 273--305."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICETAS48360.2019.9117268"},{"key":"e_1_2_2_27_1","volume-title":"https:\/\/www.onetcenter.org\/ Retrieved","author":"National Center for O*NET Development. 2020. O*Net Resource Center.","year":"2020","unstructured":"National Center for O*NET Development. 2020. O*Net Resource Center. https:\/\/www.onetcenter.org\/ Retrieved October, 2020 from"},{"key":"e_1_2_2_28_1","doi-asserted-by":"crossref","unstructured":"Mina Ghomi and Christine Redecker. 2019. Digital Competence of Educators (DigCompEdu): Development and Evaluation of a Self-assessment Instrument for Teachers' Digital Competence.. In CSEDU (1). 541--548.","DOI":"10.5220\/0007679005410548"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568260"},{"key":"e_1_2_2_30_1","volume-title":"Selection and use of content experts for instrument development. Research in nursing & health","author":"Grant Joan S","year":"1997","unstructured":"Joan S Grant and Linda L Davis. 1997. Selection and use of content experts for instrument development. Research in nursing & health, Vol. 20, 3 (1997), 269--274."},{"key":"e_1_2_2_31_1","volume-title":"Software engineering at Google. arXiv preprint arXiv:1702.01715","author":"Henderson Fergus","year":"2017","unstructured":"Fergus Henderson. 2017. Software engineering at Google. arXiv preprint arXiv:1702.01715 (2017)."},{"key":"e_1_2_2_32_1","volume-title":"Guide to advanced empirical software engineering","author":"Kitchenham Barbara A","unstructured":"Barbara A Kitchenham and Shari L Pfleeger. 2008. Personal opinion surveys. In Guide to advanced empirical software engineering. Springer, 63--92."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1027796"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2575.2011.00395.x"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2015.7332457"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3195836.3195842"},{"key":"e_1_2_2_37_1","volume-title":"Content analysis: An introduction to its methodology","author":"Krippendorff Klaus","unstructured":"Klaus Krippendorff. 2018. Content analysis: An introduction to its methodology. Sage publications."},{"key":"e_1_2_2_38_1","volume-title":"Competency and individual performance: Modelling the world of work. Organizational effectiveness: The role of psychology","author":"Kurz Rainer","year":"2002","unstructured":"Rainer Kurz and Dave Bartram. 2002. Competency and individual performance: Modelling the world of work. Organizational effectiveness: The role of psychology, Vol. 227 (2002), 255."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1080\/10919390903482382"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2003.1232286"},{"key":"e_1_2_2_41_1","unstructured":"Xiaosong Li. 2007. Incorporating a code review process into the assessment. (2007)."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/EduCon.2013.6530090"},{"key":"e_1_2_2_43_1","volume-title":"SYMPOSIUM PROCEEDINGS OCCUPATIONAL RESEARCH AND THE NAVY-PROSPECTUS","author":"McCormick Ernest J","year":"1980","unstructured":"Ernest J McCormick, PR Jeanneret, and S Gael. 1988. Position analysis questionnaire (PAQ). In SYMPOSIUM PROCEEDINGS OCCUPATIONAL RESEARCH AND THE NAVY-PROSPECTUS 1980. ERIC, 78."},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9381-9"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2012.01.060"},{"key":"e_1_2_2_46_1","doi-asserted-by":"crossref","unstructured":"Frederick P Morgeson Adela S Garza and Michael A Campion. 2013. Work design. (2013).","DOI":"10.1002\/9781118133880.hop212020"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1468-2389.2009.00467.x"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2983929"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274404"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274404"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.316.7139.1236"},{"key":"e_1_2_2_52_1","volume-title":"The content validity index: are you sure you know what's being reported? Critique and recommendations. Research in nursing & health","author":"Polit Denise F","year":"2006","unstructured":"Denise F Polit and Cheryl Tatano Beck. 2006. The content validity index: are you sure you know what's being reported? Critique and recommendations. Research in nursing & health, Vol. 29, 5 (2006), 489--497."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597121"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.17"},{"key":"e_1_2_2_55_1","volume-title":"Designing and conducting survey research: A comprehensive guide","author":"Rea Louis M","unstructured":"Louis M Rea and Richard A Parker. 2014. Designing and conducting survey research: A comprehensive guide. John Wiley & Sons."},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491411.2491444"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/CSEET.2010.21"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/CSEET.2012.22"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183519.3183525"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1049\/iet-sen.2011.0182"},{"key":"e_1_2_2_61_1","volume-title":"Scott Ed Highhouse, and Irving B Weiner","author":"Schmitt Neal W","year":"2013","unstructured":"Neal W Schmitt, Scott Ed Highhouse, and Irving B Weiner. 2013. Handbook of psychology: Industrial and organizational psychology, Vol. 12. John Wiley & Sons Inc."},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/FIE.2014.7044331"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/EDUCON.2014.6826125"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/2695664.2695856"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00110"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/CSEET.2015.8"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.69.5.797"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2584050"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2794977"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/2393596.2393656"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/197530.197637"},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1080\/01449290410001673036"},{"key":"e_1_2_2_73_1","volume-title":"What is meant by a competency? Leadership & organization development journal","author":"Woodruffe Charles","year":"1993","unstructured":"Charles Woodruffe. 1993. What is meant by a competency? Leadership & organization development journal (1993)."},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.5851633"},{"key":"e_1_2_2_75_1","unstructured":"FARIDEH YAGHMAEI. 2003. Content validity and its estimation. (2003)."}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3579471","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3579471","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:24Z","timestamp":1750182684000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3579471"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,14]]},"references-count":75,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2023,4,14]]}},"alternative-id":["10.1145\/3579471"],"URL":"https:\/\/doi.org\/10.1145\/3579471","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,14]]},"assertion":[{"value":"2023-04-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}