{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T16:25:13Z","timestamp":1792254313867,"version":"4.4.0"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,5,27]],"date-time":"2023-05-27T00:00:00Z","timestamp":1685145600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001381","name":"National Research Foundation, Singapore","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001381","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2023,7,31]]},"abstract":"<jats:p>\n            Many Duplicate Bug Report Detection (DBRD) techniques have been proposed in the research literature. The industry uses some other techniques. Unfortunately, there is insufficient comparison among them, and it is unclear how far we have been. This work fills this gap by comparing the aforementioned techniques. To compare them, we first need a benchmark that can estimate how a tool would perform if applied in a realistic setting today. Thus, we first investigated potential biases that affect the fair comparison of the accuracy of DBRD techniques. Our experiments suggest that data age and issue tracking system (ITS) choice cause a significant difference. Based on these findings, we prepared a new benchmark. We then used it to evaluate DBRD techniques to estimate better how far we have been. Surprisingly, a simpler technique outperforms recently proposed sophisticated techniques on most projects in our benchmark. In addition, we compared the DBRD techniques proposed in research with those used in\n            <jats:monospace>Mozilla<\/jats:monospace>\n            and\n            <jats:monospace>VSCode<\/jats:monospace>\n            . Surprisingly, we observe that a simple technique already adopted in practice can achieve comparable results as a recently proposed research tool. Our study gives reflections on the current state of DBRD, and we share our insights to benefit future DBRD research.\n          <\/jats:p>","DOI":"10.1145\/3576042","type":"journal-article","created":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T14:49:46Z","timestamp":1670856586000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":40,"title":["Duplicate Bug Report Detection: How Far Are We?"],"prefix":"10.1145","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6001-1372","authenticated-orcid":false,"given":"Ting","family":"Zhang","sequence":"first","affiliation":[{"name":"Singapore Management University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8599-2197","authenticated-orcid":false,"given":"Donggyun","family":"Han","sequence":"additional","affiliation":[{"name":"Royal Holloway, University of London"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7666-1524","authenticated-orcid":false,"given":"Venkatesh","family":"Vinayakarao","sequence":"additional","affiliation":[{"name":"Chennai Mathematical Institute"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6350-2700","authenticated-orcid":false,"given":"Ivana Clairine","family":"Irsan","sequence":"additional","affiliation":[{"name":"Singapore Management University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1006-8493","authenticated-orcid":false,"given":"Bowen","family":"Xu","sequence":"additional","affiliation":[{"name":"Singapore Management University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5566-3819","authenticated-orcid":false,"given":"Ferdian","family":"Thung","sequence":"additional","affiliation":[{"name":"Singapore Management University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4367-7201","authenticated-orcid":false,"given":"David","family":"Lo","sequence":"additional","affiliation":[{"name":"Singapore Management University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4336-8548","authenticated-orcid":false,"given":"Lingxiao","family":"Jiang","sequence":"additional","affiliation":[{"name":"Singapore Management University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,5,27]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"2022. GraphQL. Retrieved from https:\/\/docs.github.com\/en\/graphql. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_3_2","unstructured":"2022. hadoop. Retrieved from https:\/\/issues.apache.org\/jira\/projects\/HADOOP\/issues. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_4_2","unstructured":"2022. Jira. Retrieved from https:\/\/www.atlassian.com\/software\/jira. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_5_2","unstructured":"2022. kibana. Retrieved from https:\/\/github.com\/elastic\/kibana. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_6_2","unstructured":"2022. spark. Retrieved from https:\/\/issues.apache.org\/jira\/projects\/SPARK\/issues. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_7_2","unstructured":"2022. vscode repository on GitHub. Retrieved from https:\/\/github.com\/microsoft\/vscode. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_8_2","unstructured":"2022. vscodebot on GitHub. Retrieved from https:\/\/github.com\/apps\/vscodebot. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2013.6624026"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2013.6624025"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377816.3381738"},{"key":"e_1_3_2_12_2","unstructured":"Atlassian. 2022. Open Source Project License Request | Atlassian. Retrieved from https:\/\/www.atlassian.com\/software\/views\/open-source-license-request. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.14569\/IJACSA.2019.0100836"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-71301-2_10"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2008.4658082"},{"key":"e_1_3_2_16_2","unstructured":"Jake D. Brutlag Hilary Hutchinson and Maria Stone. 2008. User preference and search engine latency. (2008)."},{"key":"e_1_3_2_17_2","unstructured":"Bugzilla. 2022. Eclipse. Retrieved from https:\/\/bugs.eclipse.org\/bugs\/. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_18_2","unstructured":"Bugzilla. 2022. Mozilla. Retrieved from https:\/\/bugzilla.mozilla.org\/home. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2017.69"},{"key":"e_1_3_2_20_2","unstructured":"Bugzilla Wiki FAQ. 2022. How to Mark a Bug Report as a Duplicate? Retrieved from https:\/\/wiki.documentfoundation.org\/QA\/Bugzilla\/FAQ##How_to_Mark_a_Bug_Report_as_a_Duplicate. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132953"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106256"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32489-6_14"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/52.1-2.203"},{"key":"e_1_3_2_25_2","unstructured":"GitHub. Marking issues or pull requests as a duplicate - GitHub Docs. Retrieved from https:\/\/docs.github.com\/en\/issues\/tracking-your-work-with-issues\/marking-issues-or-pull-requests-as-a-duplicate. (Accessed on 02\/10\/2022)."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/BotSE52550.2021.00012"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.5555\/2487085.2487132"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387904.3389263"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/CSMR.2012.78"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-09946-8"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931051"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER50967.2021.00084"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597128"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCI-CC.2015.7259377"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1142\/S0218194018500158"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271811"},{"key":"e_1_3_2_37_2","first-page":"554","volume-title":"Proceedings of the 2018 IEEE\/ACM 15th International Conference on Mining Software Repositories (MSR)","author":"Menzies Tim","year":"2018","unstructured":"Tim Menzies, Suvodeep Majumder, Nikhila Balaji, Katie Brey, and Wei Fu. 2018. 500+ times faster than deep learning:(A case study exploring faster methods for text mining stackoverflow). In Proceedings of the 2018 IEEE\/ACM 15th International Conference on Mining Software Repositories (MSR). IEEE, 554\u2013563."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3528486"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.5555\/2821575"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395363.3397356"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9590-5"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9404-6"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/1031171.1031181"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387470"},{"key":"e_1_3_2_45_2","volume-title":"Proceedings of the Annual Meeting of the Florida Association of Institutional Research","volume":"13","author":"Romano Jeanine","year":"2006","unstructured":"Jeanine Romano, Jeffrey D. Kromrey, Jesse Coraggio, and Jeff Skowronek. 2006. Appropriate statistics for ordinal level data: Should we really be using t-test and cohen\u2019sd for evaluating group differences on the NSSE and other surveys. In Proceedings of the Annual Meeting of the Florida Association of Institutional Research, Vol. 13."},{"key":"e_1_3_2_46_2","volume-title":"CoRR","author":"Rong Xin","year":"2014","unstructured":"Xin Rong. 2014. word2vec parameter learning explained. CoRR abs\/1411.2738 (2014). arXiv:1411.2738. http:\/\/arxiv.org\/abs\/1411.2738"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2007.32"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-017-2053-3_5"},{"key":"e_1_3_2_49_2","volume-title":"Introduction to Information Retrieval","author":"Sch\u00fctze Hinrich","year":"2008","unstructured":"Hinrich Sch\u00fctze, Christopher D. Manning, and Prabhakar Raghavan. 2008. Introduction to Information Retrieval. Vol. 39. Cambridge University Press Cambridge."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678574"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2011.6100061"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806811"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236054"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/1368088.1368151"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2007.13"},{"key":"e_1_3_2_56_2","unstructured":"Eric W. Weisstein. 2004. Bonferroni correction. https:\/\/mathworld.wolfram.com\/."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/CSMR-WCRE.2014.6747167"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE5003.2020.00027"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2018.00056"},{"key":"e_1_3_2_60_2","first-page":"209","volume-title":"Proceedings of the 22nd International Conference on Software Engineering & Knowledge Engineering (SEKE.2010)","author":"Xuan Jifeng","year":"2010","unstructured":"Jifeng Xuan, He Jiang, Zhilei Ren, Jun Yan, and Zhongxuan Luo. 2010. Automatic bug triage using semi-supervised text classification. In Proceedings of the 22nd International Conference on Software Engineering & Knowledge Engineering (SEKE.2010). Redwood City, San Francisco Bay, CA, 209\u2013214."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464819"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/2396761.2396869"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-26989-4_4"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2870414"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3576042","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3576042","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:39Z","timestamp":1750182579000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3576042"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,27]]},"references-count":63,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,7,31]]}},"alternative-id":["10.1145\/3576042"],"URL":"https:\/\/doi.org\/10.1145\/3576042","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,27]]},"assertion":[{"value":"2022-02-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-26","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-05-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}