{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T00:26:29Z","timestamp":1768695989185,"version":"3.49.0"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2021,10,13]],"date-time":"2021-10-13T00:00:00Z","timestamp":1634083200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,10,13]]},"abstract":"<jats:p>Pull Requests (PRs) are a frequently used method for proposing changes to source code repositories. When discussing proposed changes in a PR discussion, stakeholders often reference a wide variety of information objects for establishing shared awareness and common ground. Previous work has not considered how the referential behavior impacts collaborative software development via PRs. This knowledge gap is the major barrier in evaluating the current support for referencing in PRs and improving them. We conducted an explorative analysis of \\textasciitilde7K references, collected from 450 public PRs on GitHub, and constructed taxonomies of referent types and expressions. Using our annotated dataset, we identified several patterns in the use of references. Referencing source code elements was prevalent but the authoring interface lacks support for it. Three classes of contextual factors influence referencing behaviors: referent type, discussion thread, and project attributes. Referencing patterns may indicate PR outcomes (e.g., merged PRs frequently reference issues, users, and tests). We conclude with design implications to support more effective referencing in PR discussion interfaces.<\/jats:p>","DOI":"10.1145\/3479529","type":"journal-article","created":{"date-parts":[[2021,10,19]],"date-time":"2021-10-19T02:32:07Z","timestamp":1634610727000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["\"@alex, this fixes #9\": Analysis of Referencing Patterns in Pull Request Discussions"],"prefix":"10.1145","volume":"5","author":[{"given":"Ashish","family":"Chopra","sequence":"first","affiliation":[{"name":"University of British Columbia, Vancouver, BC, Canada"}]},{"given":"Morgan","family":"Mo","sequence":"additional","affiliation":[{"name":"University of British Columbia, Vancouver, BC, Canada"}]},{"given":"Samuel","family":"Dodson","sequence":"additional","affiliation":[{"name":"University at Buffalo, Buffalo, NY, USA"}]},{"given":"Ivan","family":"Beschastnikh","sequence":"additional","affiliation":[{"name":"University of British Columbia, Vancouver, BC, Canada"}]},{"given":"Sidney S.","family":"Fels","sequence":"additional","affiliation":[{"name":"University of British Columbia, Vancouver, BC, Canada"}]},{"given":"Dongwook","family":"Yoon","sequence":"additional","affiliation":[{"name":"University of British Columbia, Vancouver, BC, Canada"}]}],"member":"320","published-online":{"date-parts":[[2021,10,18]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"The state of the Octoverse: An overview on community and platform growth. https:\/\/octoverse.github.com\/2019\/ Accessed","year":"2019","unstructured":"2019. The state of the Octoverse: An overview on community and platform growth. https:\/\/octoverse.github.com\/2019\/ Accessed Nov. 6, 2019."},{"key":"e_1_2_2_2_1","volume-title":"Autolinked references and URLs - GitHub Docs. https:\/\/docs.github.com\/en\/github\/writing-on-github\/autolinked-references-and-urls Accessed","year":"2020","unstructured":"2020. Autolinked references and URLs - GitHub Docs. https:\/\/docs.github.com\/en\/github\/writing-on-github\/autolinked-references-and-urls Accessed Oct 29, 2020."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","unstructured":"2021. Public Dataset: \"@alex this fixes #9\": Analysis of Referencing Patterns in Pull Request Discussions. http:\/\/doi.org\/10.5281\/zenodo.5081029 Accessed Jul. 7 2021.","DOI":"10.5281\/zenodo.5081029"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2013.6606617"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/246110.246113"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597082"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2670534"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998258"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134669"},{"key":"e_1_2_2_10_1","volume-title":"Using Language","author":"Clark Herbert","unstructured":"Herbert Clark. 1996. Using Language. Cambridge University Press, Cambridge, United Kingdom."},{"key":"e_1_2_2_11_1","unstructured":"Herbert Clark and Susan E Brennan. 1991. Grounding in communication. (1991)."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0022-5371(83)90189-5"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2901739.2901776"},{"key":"e_1_2_2_15_1","volume-title":"Context in information behavior research. Annual review of information science and technology","author":"Courtright Christina","year":"2007","unstructured":"Christina Courtright. 2007. Context in information behavior research. Annual review of information science and technology, Vol. 41, 1 (2007), 273--306."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145396"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2018.00061"},{"key":"e_1_2_2_18_1","volume-title":"2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER). 49--60","author":"Ebert F.","unstructured":"F. Ebert, F. Castor, N. Novielli, and A. Serebrenik. 2019. Confusion in Code Reviews: Reasons, Impacts, and Coping Strategies. In 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER). 49--60."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183399.3183411"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/358916.358947"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568260"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/2818754.2818800"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1177\/0165551519837174"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1031607.1031621"},{"key":"e_1_2_2_25_1","unstructured":"Francc ois Husson Julie Josse and J\u00e9r\u00f4me Pag\u00e8s. 2010. Principal Component Methods-Hierarchical Clustering-Partitional Clustering: Why Would We Need to Choose for Visualizing Data? 17 pages."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884840"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/345966.346005"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.4135\/9781071878781"},{"key":"e_1_2_2_29_1","first-page":"1125","article-title":"Transclusions in the 21st Century","volume":"7","author":"Krottmaier Harald","year":"2001","unstructured":"Harald Krottmaier and Hermann A. Maurer. 2001. Transclusions in the 21st Century. Journal of Universal Computer Science, Vol. 7, 12 (2001), 1125--1136.","journal-title":"Journal of Universal Computer Science"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531727"},{"key":"e_1_2_2_31_1","volume-title":"Critical Theory and New Media in an Era of Globalization","author":"Landow George","unstructured":"George Landow. 2006. Hypertext 3.0: Critical Theory and New Media in an Era of Globalization .Johns Hopkins University Press, Baltimore, MD ."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2018.00053"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-017-1783-2"},{"key":"e_1_2_2_34_1","volume-title":"Determinants of information retweeting in microblogging. Internet Research","author":"Liu Zhiming","year":"2012","unstructured":"Zhiming Liu, Lu Liu, and Hong Li. 2012. Determinants of information retweeting in microblogging. Internet Research (2012)."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1108\/JD-06-2012-0078"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1108\/00220410310506303"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/276627.276632"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.2498\/cit.2006.02.07"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359174"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9381-9"},{"key":"e_1_2_2_41_1","doi-asserted-by":"crossref","unstructured":"Martin M\u00fchlpfordt and Martin Wessner. 2005. Explicit referencing in chat supports collaborative learning. (2005).","DOI":"10.3115\/1149293.1149353"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2008.71"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491411.2491415"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2012.6405249"},{"key":"e_1_2_2_45_1","volume-title":"Computer Lib\/dream Machine","author":"Nelson Theodor","unstructured":"Theodor Nelson. 1987. Computer Lib\/dream Machine .Tempus Books of Microsoft Press, Redmond, WA ."},{"key":"e_1_2_2_46_1","volume-title":"Distance matters. Human-computer interaction","author":"Olson Gary M","year":"2000","unstructured":"Gary M Olson and Judith S Olson. 2000. Distance matters. Human-computer interaction, Vol. 15, 2--3 (2000), 139--178."},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274404"},{"key":"e_1_2_2_48_1","volume-title":"Dakuo Wang, Michael Muller, and Felix Portnoy.","author":"Piorkowski David","year":"2021","unstructured":"David Piorkowski, Soya Park, April Yi Wang, Dakuo Wang, Michael Muller, and Felix Portnoy. 2021. How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study. arXiv preprint arXiv:2101.06098 (2021)."},{"key":"e_1_2_2_49_1","volume-title":"Save time linking resources with autolink references. https:\/\/github.blog\/2019--10--14-introducing-autolink-references\/ Accessed","author":"Schneider Lars","year":"2020","unstructured":"Lars Schneider. 2019. Save time linking resources with autolink references. https:\/\/github.blog\/2019--10--14-introducing-autolink-references\/ Accessed Oct 29, 2020."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2695664.2695856"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-012-9164-4"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568315"},{"key":"e_1_2_2_53_1","volume-title":"Psychological Scaling: Theory and Applications","author":"Tucker Ledyard R","unstructured":"Ledyard R Tucker. 1960. Intra-Individual and Inter-Individual Multidimensionality. In Psychological Scaling: Theory and Applications, Samuel Messick and Harold Gulliksen (Eds.). Wiley, New York, NY, 155--167."},{"key":"e_1_2_2_54_1","volume-title":"Exploratory Data Analysis","author":"Tukey John","unstructured":"John Tukey. 1977. Exploratory Data Analysis. Addison-Wesley, Reading, MA ."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3195836.3195841"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274451"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300719"},{"key":"e_1_2_2_58_1","volume-title":"Wait for It: Determinants of Pull Request Evaluation Latency on GitHub. In 2015 IEEE\/ACM 12th Working Conference on Mining Software Repositories. 367--371","author":"Yu Y.","unstructured":"Y. Yu, H. Wang, V. Filkov, P. Devanbu, and B. Vasilescu. 2015. Wait for It: Determinants of Pull Request Evaluation Latency on GitHub. In 2015 IEEE\/ACM 12th Working Conference on Mining Software Repositories. 367--371."},{"key":"e_1_2_2_59_1","volume-title":"How Developers Document Pull Requests With External References. In 2017 IEEE\/ACM 25th International Conference on Program Comprehension. IEEE","author":"Zampetti Fiorella","year":"2017","unstructured":"Fiorella Zampetti, Luca Ponzanelli, Gabriele Bavota, Andrea Mocci, Massimiliano Di Penta, and Michele Lanza. 2017. How Developers Document Pull Requests With External References. In 2017 IEEE\/ACM 25th International Conference on Program Comprehension. IEEE, Piscataway, NJ, 23--33."},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2875913.2875914"},{"key":"e_1_2_2_61_1","volume-title":"Qualitative analysis of content 2 ed.). Libraries Unlimited","author":"Zhang Yan","unstructured":"Yan Zhang and Barbara Wildemuth. 2017. Qualitative analysis of content 2 ed.). Libraries Unlimited, Santa Barbara, CA, 318--330."},{"key":"e_1_2_2_62_1","volume-title":"The Impact of Continuous Integration on Other Software Development Practices: A Large-Scale Empirical Study. In IEEE\/ACM International Conference on Automated Software Engineering. IEEE","author":"Zhao Yangyang","year":"2017","unstructured":"Yangyang Zhao, Alexander Serebrenik, Yuming Zhou, Vladimir Filkov, and Bogdan Vasilescu. 2017. The Impact of Continuous Integration on Other Software Development Practices: A Large-Scale Empirical Study. In IEEE\/ACM International Conference on Automated Software Engineering. IEEE, Piscataway, NJ, 60--71."},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208326"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479529","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3479529","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:59:29Z","timestamp":1752469169000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479529"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,13]]},"references-count":63,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2021,10,13]]}},"alternative-id":["10.1145\/3479529"],"URL":"https:\/\/doi.org\/10.1145\/3479529","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,13]]},"assertion":[{"value":"2021-10-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}