{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T00:06:31Z","timestamp":1777421191941,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":60,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,11,8]],"date-time":"2020-11-08T00:00:00Z","timestamp":1604793600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,11,8]]},"DOI":"10.1145\/3368089.3417048","type":"proceedings-article","created":{"date-parts":[[2020,11,8]],"date-time":"2020-11-08T06:03:47Z","timestamp":1604815427000},"page":"1308-1319","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["Effort-aware just-in-time defect identification in practice: a case study at Alibaba"],"prefix":"10.1145","author":[{"given":"Meng","family":"Yan","sequence":"first","affiliation":[{"name":"Chongqing University, China"}]},{"given":"Xin","family":"Xia","sequence":"additional","affiliation":[{"name":"Monash University, Australia"}]},{"given":"Yuanrui","family":"Fan","sequence":"additional","affiliation":[{"name":"Zhejiang University, China"}]},{"given":"David","family":"Lo","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore"}]},{"given":"Ahmed E.","family":"Hassan","sequence":"additional","affiliation":[{"name":"Queen's University, Canada"}]},{"given":"Xindong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Alibaba Group, China"}]}],"member":"320","published-online":{"date-parts":[[2020,11,8]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Bonferroni and \u0160id\u00e1k corrections for multiple comparisons. Encyclopedia of measurement and statistics 3 ( 2007 ), 103-107","author":"Abdi Herv\u00e9","unstructured":"Herv\u00e9 Abdi . 2007. Bonferroni and \u0160id\u00e1k corrections for multiple comparisons. Encyclopedia of measurement and statistics 3 ( 2007 ), 103-107 . Herv\u00e9 Abdi. 2007. Bonferroni and \u0160id\u00e1k corrections for multiple comparisons. Encyclopedia of measurement and statistics 3 ( 2007 ), 103-107."},{"key":"e_1_3_2_2_2_1","volume-title":"M Henry H Stevens, and Jada-Simone S White","author":"Bolker Benjamin M","year":"2009","unstructured":"Benjamin M Bolker , Mollie E Brooks , Connie J Clark , Shane W Geange , John R Poulsen , M Henry H Stevens, and Jada-Simone S White . 2009 . Generalized linear mixed models: a practical guide for ecology and evolution. Trends in ecology & evolution 24, 3 ( 2009 ), 127-135. Benjamin M Bolker, Mollie E Brooks, Connie J Clark, Shane W Geange, John R Poulsen, M Henry H Stevens, and Jada-Simone S White. 2009. Generalized linear mixed models: a practical guide for ecology and evolution. Trends in ecology & evolution 24, 3 ( 2009 ), 127-135."},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"crossref","unstructured":"Andrew P Bradley. 1997. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition 30 7 ( 1997 ) 1145-1159.  Andrew P Bradley. 1997. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition 30 7 ( 1997 ) 1145-1159.","DOI":"10.1016\/S0031-3203(96)00142-2"},{"key":"e_1_3_2_2_4_1","volume-title":"Ordinal methods for behavioral data analysis","author":"Clif Norman","unstructured":"Norman Clif . 2014. Ordinal methods for behavioral data analysis . Psychology Press . Norman Clif. 2014. Ordinal methods for behavioral data analysis. Psychology Press."},{"key":"e_1_3_2_2_5_1","volume-title":"Verification and Validation (ICST), 2011 IEEE Fourth International Conference on. IEEE, 357-366","author":"Czerwonka Jacek","year":"2011","unstructured":"Jacek Czerwonka , Rajiv Das , Nachiappan Nagappan , Alex Tarvo , and Alex Teterev . 2011 . Crane: Failure prediction, change analysis and test prioritization in practiceexperiences from windows. In Software Testing , Verification and Validation (ICST), 2011 IEEE Fourth International Conference on. IEEE, 357-366 . Jacek Czerwonka, Rajiv Das, Nachiappan Nagappan, Alex Tarvo, and Alex Teterev. 2011. Crane: Failure prediction, change analysis and test prioritization in practiceexperiences from windows. In Software Testing, Verification and Validation (ICST), 2011 IEEE Fourth International Conference on. IEEE, 357-366."},{"key":"e_1_3_2_2_6_1","volume-title":"Chaf from the Wheat: Characterizing and Determining Valid Bug Reports","author":"Fan Yuanrui","year":"2018","unstructured":"Yuanrui Fan , Xin Xia , David Lo , and Ahmed E Hassan . 2018. Chaf from the Wheat: Characterizing and Determining Valid Bug Reports . IEEE Transactions on Software Engineering ( 2018 ). Yuanrui Fan, Xin Xia, David Lo, and Ahmed E Hassan. 2018. Chaf from the Wheat: Characterizing and Determining Valid Bug Reports. IEEE Transactions on Software Engineering ( 2018 )."},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106257"},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597075"},{"key":"e_1_3_2_2_9_1","volume-title":"Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis","author":"Harrell Frank E","unstructured":"Frank E Harrell . 2001. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis . Springer . Frank E Harrell. 2001. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. Springer."},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"crossref","unstructured":"Safwat Hassan Chakkrit Tantithamthavorn Cor-Paul Bezemer and Ahmed E Hassan. 2018. Studying the dialogue between users and developers of free apps in the google play store. Empirical Software Engineering 23 3 ( 2018 ) 1275-1312.  Safwat Hassan Chakkrit Tantithamthavorn Cor-Paul Bezemer and Ahmed E Hassan. 2018. Studying the dialogue between users and developers of free apps in the google play store. Empirical Software Engineering 23 3 ( 2018 ) 1275-1312.","DOI":"10.1007\/s10664-017-9538-9"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/2337223.2337247"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2017.51"},{"key":"e_1_3_2_2_13_1","volume-title":"Revisiting Supervised and Unsupervised Models for Efort-Aware Just-in-Time Defect Prediction. Empirical Software Engineering ( 2018 )","author":"Huang Qiao","unstructured":"Qiao Huang , Xin Xia , and David Lo. 2018. Revisiting Supervised and Unsupervised Models for Efort-Aware Just-in-Time Defect Prediction. Empirical Software Engineering ( 2018 ) , In press . Qiao Huang, Xin Xia, and David Lo. 2018. Revisiting Supervised and Unsupervised Models for Efort-Aware Just-in-Time Defect Prediction. Empirical Software Engineering ( 2018 ), In press."},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2013.6693087"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Paul CD Johnson. 2014. Extension of Nakagawa & Schielzeth's R2GLMM to random slopes models. Methods in Ecology and Evolution 5 9 ( 2014 ) 944-946.  Paul CD Johnson. 2014. Extension of Nakagawa & Schielzeth's R2GLMM to random slopes models. Methods in Ecology and Evolution 5 9 ( 2014 ) 944-946.","DOI":"10.1111\/2041-210X.12225"},{"key":"e_1_3_2_2_16_1","volume-title":"Studying just-in-time defect prediction using cross-project models. Empirical Software Engineering 21, 5 ( 2016 )","author":"Kamei Yasutaka","year":"2072","unstructured":"Yasutaka Kamei , Takafumi Fukushima , Shane McIntosh , Kazuhiro Yamashita , Naoyasu Ubayashi , and Ahmed E Hassan . 2016. Studying just-in-time defect prediction using cross-project models. Empirical Software Engineering 21, 5 ( 2016 ) , 2072 -2106. Yasutaka Kamei, Takafumi Fukushima, Shane McIntosh, Kazuhiro Yamashita, Naoyasu Ubayashi, and Ahmed E Hassan. 2016. Studying just-in-time defect prediction using cross-project models. Empirical Software Engineering 21, 5 ( 2016 ), 2072-2106."},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2012.70"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.70773"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2008.35"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.5555\/2486788.2486838"},{"key":"e_1_3_2_2_21_1","volume-title":"Towards just-intime suggestions for log changes. Empirical Software Engineering 22, 4 ( 2017 )","author":"Li Heng","year":"1831","unstructured":"Heng Li , Weiyi Shang , Ying Zou , and Ahmed E Hassan . 2017. Towards just-intime suggestions for log changes. Empirical Software Engineering 22, 4 ( 2017 ) , 1831 -1865. Heng Li, Weiyi Shang, Ying Zou, and Ahmed E Hassan. 2017. Towards just-intime suggestions for log changes. Empirical Software Engineering 22, 4 ( 2017 ), 1831-1865."},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"crossref","unstructured":"Chao Liu Dan Yang Xin Xia Meng Yan and Xiaohong Zhang. 2019. A twophase transfer learning model for cross-project defect prediction. Information and Software Technology 107 ( 2019 ) 125-136.  Chao Liu Dan Yang Xin Xia Meng Yan and Xiaohong Zhang. 2019. A twophase transfer learning model for cross-project defect prediction. Information and Software Technology 107 ( 2019 ) 125-136.","DOI":"10.1016\/j.infsof.2018.11.005"},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2017.8"},{"key":"e_1_3_2_2_24_1","volume-title":"Are fix-inducing changes a moving target? a longitudinal case study of just-in-time defect prediction","author":"McIntosh Shane","year":"2017","unstructured":"Shane McIntosh and Yasutaka Kamei . 2017. Are fix-inducing changes a moving target? a longitudinal case study of just-in-time defect prediction . IEEE Transactions on Software Engineering ( 2017 ). Shane McIntosh and Yasutaka Kamei. 2017. Are fix-inducing changes a moving target? a longitudinal case study of just-in-time defect prediction. IEEE Transactions on Software Engineering ( 2017 )."},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CSMR.2010.18"},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.256941"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635892"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1287624.1287683"},{"key":"e_1_3_2_2_29_1","article-title":"Predicting risk of software changes","volume":"5","author":"Mockus Audris","year":"2000","unstructured":"Audris Mockus and David M Weiss . 2000 . Predicting risk of software changes . Bell Labs Technical Journal 5 , 2 ( 2000 ), 169-180. Audris Mockus and David M Weiss. 2000. Predicting risk of software changes. Bell Labs Technical Journal 5, 2 ( 2000 ), 169-180.","journal-title":"Bell Labs Technical Journal"},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"crossref","unstructured":"Shinichi Nakagawa and Holger Schielzeth. 2013. A general and simple method for obtaining R2 from generalized linear mixed-efects models. Methods in Ecology and Evolution 4 2 ( 2013 ) 133-142.  Shinichi Nakagawa and Holger Schielzeth. 2013. A general and simple method for obtaining R2 from generalized linear mixed-efects models. Methods in Ecology and Evolution 4 2 ( 2013 ) 133-142.","DOI":"10.1111\/j.2041-210x.2012.00261.x"},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196438"},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2001420.2001445"},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.4"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"crossref","unstructured":"Alastair J Scott and M Knott. 1974. A cluster analysis method for grouping means in the analysis of variance. Biometrics ( 1974 ) 507-512.  Alastair J Scott and M Knott. 1974. A cluster analysis method for grouping means in the analysis of variance. Biometrics ( 1974 ) 507-512.","DOI":"10.2307\/2529204"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2322358"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2393596.2393670"},{"key":"e_1_3_2_2_37_1","first-page":"1","article-title":"When do changes induce fixes?. In ACM sigsoft software engineering notes, Vol. 30","author":"\u015aliwerski Jacek","year":"2005","unstructured":"Jacek \u015aliwerski , Thomas Zimmermann , and Andreas Zeller . 2005 . When do changes induce fixes?. In ACM sigsoft software engineering notes, Vol. 30 . ACM , 1 - 5 . Jacek \u015aliwerski, Thomas Zimmermann, and Andreas Zeller. 2005. When do changes induce fixes?. In ACM sigsoft software engineering notes, Vol. 30. ACM, 1-5.","journal-title":"ACM"},{"key":"e_1_3_2_2_38_1","volume-title":"Fixed and random efects. Encyclopedia of statistics in behavioral science ( 2005 )","author":"Snijders Tom AB","unstructured":"Tom AB Snijders . 2005. Fixed and random efects. Encyclopedia of statistics in behavioral science ( 2005 ) . Tom AB Snijders. 2005. Fixed and random efects. Encyclopedia of statistics in behavioral science ( 2005 )."},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.139"},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183519.3183547"},{"key":"e_1_3_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884857"},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2584050"},{"key":"e_1_3_2_2_43_1","volume-title":"The impact of automated parameter optimization on defect prediction models","author":"Tantithamthavorn Chakkrit","year":"2018","unstructured":"Chakkrit Tantithamthavorn , Shane McIntosh , Ahmed E Hassan , and Kenichi Matsumoto . 2018. The impact of automated parameter optimization on defect prediction models . IEEE Transactions on Software Engineering ( 2018 ). Chakkrit Tantithamthavorn, Shane McIntosh, Ahmed E Hassan, and Kenichi Matsumoto. 2018. The impact of automated parameter optimization on defect prediction models. IEEE Transactions on Software Engineering ( 2018 )."},{"key":"e_1_3_2_2_44_1","volume-title":"Tom AB Snijders Roel J Bosker, and Roel J Bosker","author":"Tom AB","year":"1999","unstructured":"AB Tom , Tom AB Snijders Roel J Bosker, and Roel J Bosker . 1999 . Multilevel analysis: an introduction to basic and advanced multilevel modeling. Sage . AB Tom, Tom AB Snijders Roel J Bosker, and Roel J Bosker. 1999. Multilevel analysis: an introduction to basic and advanced multilevel modeling. Sage."},{"key":"e_1_3_2_2_45_1","unstructured":"Zhiyuan Wan Xin Xia Ahmed E Hassan David Lo Jianwei Yin and Xiaohu Yang. 2018. Perceptions Expectations and Challenges in Defect Prediction. IEEE Transactions on Software Engineering ( 2018 ).  Zhiyuan Wan Xin Xia Ahmed E Hassan David Lo Jianwei Yin and Xiaohu Yang. 2018. Perceptions Expectations and Challenges in Defect Prediction. IEEE Transactions on Software Engineering ( 2018 )."},{"key":"e_1_3_2_2_46_1","unstructured":"Wikipedia. [n.d.]. https:\/\/en.wikipedia.org\/wiki\/Alibaba_Group. ([n. d.]).  Wikipedia. [n.d.]. https:\/\/en.wikipedia.org\/wiki\/Alibaba_Group. ([n. d.])."},{"key":"e_1_3_2_2_47_1","doi-asserted-by":"crossref","unstructured":"Frank Wilcoxon. 1992. Individual comparisons by ranking methods. Breakthroughs in Statistics ( 1992 ) 196-202.  Frank Wilcoxon. 1992. Individual comparisons by ranking methods. Breakthroughs in Statistics ( 1992 ) 196-202.","DOI":"10.1007\/978-1-4612-4380-9_16"},{"key":"e_1_3_2_2_48_1","volume-title":"Ahmed E Hassan, and Zhenchang Xing.","author":"Xia Xin","year":"2017","unstructured":"Xin Xia , Lingfeng Bao , David Lo , Pavneet Singh Kochhar , Ahmed E Hassan, and Zhenchang Xing. 2017 . What do developers search for on the web? Empirical Software Engineering 22, 6 ( 2017 ), 3149-3185. Xin Xia, Lingfeng Bao, David Lo, Pavneet Singh Kochhar, Ahmed E Hassan, and Zhenchang Xing. 2017. What do developers search for on the web? Empirical Software Engineering 22, 6 ( 2017 ), 3149-3185."},{"key":"e_1_3_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2543218"},{"key":"e_1_3_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2017.48"},{"key":"e_1_3_2_2_51_1","doi-asserted-by":"crossref","unstructured":"Meng Yan Xin Xia David Lo Ahmed E Hassan and Shanping Li. 2019. Characterizing and identifying reverted commits. Empirical Software Engineering 24 4 ( 2019 ) 2171-2208.  Meng Yan Xin Xia David Lo Ahmed E Hassan and Shanping Li. 2019. Characterizing and identifying reverted commits. Empirical Software Engineering 24 4 ( 2019 ) 2171-2208.","DOI":"10.1007\/s10664-019-09688-8"},{"key":"e_1_3_2_2_52_1","article-title":"Automating change-level self-admitted technical debt determination","volume":"45","author":"Yan Meng","year":"2018","unstructured":"Meng Yan , Xin Xia , Emad Shihab , David Lo , Jianwei Yin , and Xiaohu Yang . 2018 . Automating change-level self-admitted technical debt determination . IEEE Transactions on Software Engineering 45 , 12 ( 2018 ), 1211-1229. Meng Yan, Xin Xia, Emad Shihab, David Lo, Jianwei Yin, and Xiaohu Yang. 2018. Automating change-level self-admitted technical debt determination. IEEE Transactions on Software Engineering 45, 12 ( 2018 ), 1211-1229.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_2_53_1","first-page":"206","volume-title":"TLEL: A two-layer ensemble learning approach for just-in-time defect prediction. Information and Software Technology 87 ( 2017 )","author":"Yang Xinli","year":"2017","unstructured":"Xinli Yang , David Lo , Xin Xia , and Jianling Sun . 2017 . TLEL: A two-layer ensemble learning approach for just-in-time defect prediction. Information and Software Technology 87 ( 2017 ) , 206 - 220 . Xinli Yang, David Lo, Xin Xia, and Jianling Sun. 2017. TLEL: A two-layer ensemble learning approach for just-in-time defect prediction. Information and Software Technology 87 ( 2017 ), 206-220."},{"key":"e_1_3_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/QRS.2015.14"},{"key":"e_1_3_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950353"},{"key":"e_1_3_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025121"},{"key":"e_1_3_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2597073.2597078"},{"key":"e_1_3_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2013.46"},{"key":"e_1_3_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183339"},{"key":"e_1_3_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/1595696.1595713"}],"event":{"name":"ESEC\/FSE '20: 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","location":"Virtual Event USA","acronym":"ESEC\/FSE '20","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3368089.3417048","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3368089.3417048","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:01:58Z","timestamp":1750197718000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3368089.3417048"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,8]]},"references-count":60,"alternative-id":["10.1145\/3368089.3417048","10.1145\/3368089"],"URL":"https:\/\/doi.org\/10.1145\/3368089.3417048","relation":{},"subject":[],"published":{"date-parts":[[2020,11,8]]},"assertion":[{"value":"2020-11-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}