{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T21:36:31Z","timestamp":1769722591647,"version":"3.49.0"},"reference-count":84,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,7,6]],"date-time":"2020-07-06T00:00:00Z","timestamp":1593993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2018YFB1403400"],"award-info":[{"award-number":["2018YFB1403400"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61602450"],"award-info":[{"award-number":["61602450"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2020,10,31]]},"abstract":"<jats:p>Software engineers get questions of \u201chow much testing is enough\u201d on a regular basis. Existing approaches in software testing management employ experience-, risk-, or value-based analysis to prioritize and manage testing processes. However, very few is applicable to the emerging crowdtesting paradigm to cope with extremely limited information and control over unknown, online crowdworkers. In practice, deciding when to close a crowdtesting task is largely done by experience-based guesswork and frequently results in ineffective crowdtesting. More specifically, it is found that an average of 32% testing cost was wasteful spending in current crowdtesting practice. This article intends to address this challenge by introducing automated decision support for monitoring and determining appropriate time to close crowdtesting tasks.<\/jats:p>\n          <jats:p>\n            To that end, it first investigates the necessity and feasibility of close prediction of crowdtesting tasks based on an industrial dataset. Next, it proposes a close prediction approach named iSENSE2.0, which applies incremental sampling technique to process crowdtesting reports arriving in chronological order and organizes them into fixed-sized groups as dynamic inputs. Then, a duplicate tagger analyzes the duplicate status of received crowd reports, and a CRC-based (Capture-ReCapture) close estimator generates the close decision based on the dynamic bug arrival status. In addition, a coverage-based sanity checker is designed to reinforce the stability and performance of close prediction. Finally, the evaluation of iSENSE2.0 is conducted on 56,920 reports of 306 crowdtesting tasks from one of the largest crowdtesting platforms. The results show that a median of 100% bugs can be detected with 30% saved cost. The performance of iSENSE2.0 does not demonstrate significant difference with the state-of-the-art approach\n            <jats:italic>iSENSE<\/jats:italic>\n            , while the later one relies on the duplicate tag, which is generally considered as time-consuming and tedious to obtain.\n          <\/jats:p>","DOI":"10.1145\/3394602","type":"journal-article","created":{"date-parts":[[2020,7,6]],"date-time":"2020-07-06T21:22:03Z","timestamp":1594070523000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["iSENSE2.0"],"prefix":"10.1145","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9941-6713","authenticated-orcid":false,"given":"Junjie","family":"Wang","sequence":"first","affiliation":[{"name":"Laboratory for Internet Software Technologies, State Key Laboratory of Computer Sciences, Institute of Software Chinese Academy of Sciences; University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ye","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Systems and Enterprises, Stevens Institute of Technology, Hoboken, NJ, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5040-3196","authenticated-orcid":false,"given":"Tim","family":"Menzies","sequence":"additional","affiliation":[{"name":"Department of Computer Science, North Carolina State University, Raleigh, NC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qing","family":"Wang","sequence":"additional","affiliation":[{"name":"Laboratory for Internet Software Technologies, State Key Laboratory of Computer Sciences, Institute of Software Chinese Academy of Sciences; University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,7,6]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Sahil Deva. 2018. Retrieved from https:\/\/dzone.com\/articles\/10-critical-factors-to-successfully-perform-crowds.  Sahil Deva. 2018. Retrieved from https:\/\/dzone.com\/articles\/10-critical-factors-to-successfully-perform-crowds."},{"key":"e_1_2_1_2_1","volume-title":"SERC Workshop on Model-Based System Assurance.","author":"SERC.","year":"2019","unstructured":"SERC. 2019 . SERC Workshop on Model-Based System Assurance. Retrieved from https:\/\/sercuarc.org\/event\/serc-workshop-model-based-system-assurance\/. SERC. 2019. SERC Workshop on Model-Based System Assurance. Retrieved from https:\/\/sercuarc.org\/event\/serc-workshop-model-based-system-assurance\/."},{"key":"e_1_2_1_3_1","unstructured":"Software Testing Help. 2020. Retrieved from http:\/\/www.softwaretestinghelp.com\/crowdsourced-testing-companies\/.  Software Testing Help. 2020. Retrieved from http:\/\/www.softwaretestinghelp.com\/crowdsourced-testing-companies\/."},{"key":"e_1_2_1_4_1","unstructured":"DZone. 2020. Retrieved from https:\/\/dzone.com\/articles\/top-10-benefits-of-crowd-testing-1.  DZone. 2020. Retrieved from https:\/\/dzone.com\/articles\/top-10-benefits-of-crowd-testing-1."},{"key":"e_1_2_1_5_1","unstructured":"Software Testing Help. 2020. Retrieved from https:\/\/www.softwaretestinghelp.com\/guide-to-crowdsourced-testing\/.  Software Testing Help. 2020. Retrieved from https:\/\/www.softwaretestinghelp.com\/guide-to-crowdsourced-testing\/."},{"key":"e_1_2_1_6_1","unstructured":"DZone. 2020. Retrieved from https:\/\/dzone.com\/articles\/why-google-uses-crowd-testing.  DZone. 2020. Retrieved from https:\/\/dzone.com\/articles\/why-google-uses-crowd-testing."},{"key":"e_1_2_1_7_1","unstructured":"Applause. 2020. Retrieved from https:\/\/www.applause.com\/customers\/.  Applause. 2020. Retrieved from https:\/\/www.applause.com\/customers\/."},{"key":"e_1_2_1_8_1","unstructured":"360\n    logica. 2020. Retrieved from https:\/\/www.360logica.com\/blog\/challenges-faced-in-mobile-app-testing\/.  360 logica. 2020. Retrieved from https:\/\/www.360logica.com\/blog\/challenges-faced-in-mobile-app-testing\/."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180197"},{"key":"e_1_2_1_10_1","first-page":"32","article-title":"Process improvements for crowdsourced software testing","volume":"8","author":"Alyahya Sultan","year":"2017","unstructured":"Sultan Alyahya and Dalal Alrugebh . 2017 . Process improvements for crowdsourced software testing . Int. J. Adv. Comput. Sci. Applic. 8 , 6 (2017), 32 \u2013 40 . Sultan Alyahya and Dalal Alrugebh. 2017. Process improvements for crowdsourced software testing. Int. J. Adv. Comput. Sci. Applic. 8, 6 (2017), 32\u201340.","journal-title":"Int. J. Adv. Comput. Sci. Applic."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/944919.944966"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.852741"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/65.3.625"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.2307\/2531532"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Anne Chao. 1988. Estimating animal abundance with capture frequency data. J. Wildlife Manag. (1988) 295--300.  Anne Chao. 1988. Estimating animal abundance with capture frequency data. J. Wildlife Manag. (1988) 295--300.","DOI":"10.2307\/3801237"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2351676.2351697"},{"key":"e_1_2_1_17_1","volume-title":"Ordinal Methods for Behavioral Data Analysis","author":"Cliff Norman","unstructured":"Norman Cliff . 2014. Ordinal Methods for Behavioral Data Analysis . Psychology Press . Norman Cliff. 2014. Ordinal Methods for Behavioral Data Analysis. Psychology Press."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2017.265"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.18293\/SEKE2017-102"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/143062.143090"},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the International Symposium on Software Testing and Analysis. ACM, 234--245","author":"Epitropakis Michael G.","unstructured":"Michael G. Epitropakis , Shin Yoo , Mark Harman , and Edmund K. Burke . 2015. Empirical evaluation of Pareto efficient multi-objective regression test case prioritisation . In Proceedings of the International Symposium on Software Testing and Analysis. ACM, 234--245 . Michael G. Epitropakis, Shin Yoo, Mark Harman, and Edmund K. Burke. 2015. Empirical evaluation of Pareto efficient multi-objective regression test case prioritisation. In Proceedings of the International Symposium on Software Testing and Analysis. ACM, 234--245."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786862"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970276.2970367"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218539314500041"},{"key":"e_1_2_1_25_1","first-page":"1","article-title":"Successes, challenges, and rethinking\u2014An industrial investigation on crowdsourced mobile application testing","volume":"24","author":"Gao Ruizhi","year":"2018","unstructured":"Ruizhi Gao , Yabin Wang , Yang Feng , Zhenyu Chen , and W. Eric Wong . 2018 . Successes, challenges, and rethinking\u2014An industrial investigation on crowdsourced mobile application testing . Empir. Softw. Eng. 24 , 2 (2018), 1 -- 25 . Ruizhi Gao, Yabin Wang, Yang Feng, Zhenyu Chen, and W. Eric Wong. 2018. Successes, challenges, and rethinking\u2014An industrial investigation on crowdsourced mobile application testing. Empir. Softw. Eng. 24, 2 (2018), 1--25.","journal-title":"Empir. Softw. Eng."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1049\/iet-sen.2010.0007"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897073.2897088"},{"key":"e_1_2_1_28_1","first-page":"575","article-title":"Affective crowdsourcing applied to usability testing","volume":"5","author":"Gomide V. H. M.","year":"2014","unstructured":"V. H. M. Gomide , P. A. Valle , J. O. Ferreira , J. R. G. Barbosa , A. F. da Rocha , and T. M. G. d. A. Barbosa . 2014 . Affective crowdsourcing applied to usability testing . Int. J. Comput. Sci. Inf. Technol. 5 , 1 (2014), 575 -- 579 . V. H. M. Gomide, P. A. Valle, J. O. Ferreira, J. R. G. Barbosa, A. F. da Rocha, and T. M. G. d. A. Barbosa. 2014. Affective crowdsourcing applied to usability testing. Int. J. Comput. Sci. Inf. Technol. 5, 1 (2014), 575--579.","journal-title":"Int. J. Comput. Sci. Inf. Technol."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218194015710060"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00096"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2005.06.023"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884791"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238183"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/RCIS.2014.6861072"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICMIRA.2013.105"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106164"},{"key":"e_1_2_1_37_1","volume-title":"Metrics and Models in Software Quality Engineering","author":"Kan Stephen H.","unstructured":"Stephen H. Kan . 2002. Metrics and Models in Software Quality Engineering . Addison-Wesley Longman Publishing Co., Inc. Stephen H. Kan. 2002. Metrics and Models in Software Quality Engineering. Addison-Wesley Longman Publishing Co., Inc."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2011.111"},{"key":"e_1_2_1_39_1","unstructured":"Pierre S. Laplace. 1783. Sur les naissances les mariages et les morts. Hist. I\u2019Acad. Roy. Sci. (1783) 693.  Pierre S. Laplace. 1783. Sur les naissances les mariages et les morts. Hist. I\u2019Acad. Roy. Sci. (1783) 693."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1080\/03610919608813323"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.5555\/1524219"},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER\u201918)","author":"Liu Di","unstructured":"Di Liu , Xiaofang Zhang , Yang Feng , and James A. Jones . 2018. Generating descriptions for screenshots to assist crowdsourced testing . In Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER\u201918) . IEEE, 492--496. Di Liu, Xiaofang Zhang, Yang Feng, and James A. Jones. 2018. Generating descriptions for screenshots to assist crowdsourced testing. In Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER\u201918). IEEE, 492--496."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2745802.2745816"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2372251.2372291"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.09.015"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2777831"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.256941"},{"key":"e_1_2_1_48_1","volume-title":"Proceedings of the Conference on Neural Information Processing Systems (NIPS\u201913)","author":"Mikolov T.","unstructured":"T. Mikolov , I. Sutskever , K. Chen , G. Corrado , and J. Dean . 2013. Distributed representations of words and phrases and their compositionality . In Proceedings of the Conference on Neural Information Processing Systems (NIPS\u201913) . 3111--3119. T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the Conference on Neural Information Processing Systems (NIPS\u201913). 3111--3119."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180210"},{"key":"e_1_2_1_51_1","doi-asserted-by":"crossref","unstructured":"R. Musson J. Richards D. Fisher C. Bird B. Bussone and S. Ganguly. 2013. Leveraging the crowd: How 48 000 users helped improve Lync performance. IEEE Softw. 30 4 (2013) 38--45.  R. Musson J. Richards D. Fisher C. Bird B. Bussone and S. Ganguly. 2013. Leveraging the crowd: How 48 000 users helped improve Lync performance. IEEE Softw. 30 4 (2013) 38--45.","DOI":"10.1109\/MS.2013.67"},{"key":"e_1_2_1_52_1","volume-title":"The Art of Software Testing","author":"Myers Glenford J.","unstructured":"Glenford J. Myers , Corey Sandler , and Tom Badgett . 2011. The Art of Software Testing . John Wiley 8 Sons. Glenford J. Myers, Corey Sandler, and Tom Badgett. 2011. The Art of Software Testing. John Wiley 8 Sons."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2720603"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2351676.2351687"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3339002"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2364175"},{"key":"e_1_2_1_57_1","volume-title":"Nature 72","author":"Pearson Karl","year":"1905","unstructured":"Karl Pearson . 1905 . The problem of the random walk . Nature 72 , 1867 (1905), 342. Karl Pearson. 1905. The problem of the random walk. Nature 72, 1867 (1905), 342."},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER\u201916)","author":"Rocha H.","unstructured":"H. Rocha , M. T. Valente , H. Marques-Neto , and G. C. Murphy . 2016. An empirical study on recommendations of similar bugs . In Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER\u201916) . 46--56. H. Rocha, M. T. Valente, H. Marques-Neto, and G. C. Murphy. 2016. An empirical study on recommendations of similar bugs. In Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER\u201916). 46--56."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3084226.3084250"},{"key":"e_1_2_1_60_1","volume-title":"Proceedings of the IEEE\/ACM 37th IEEE International Conference on Software Engineering","volume":"1","author":"Saha Ripon K.","unstructured":"Ripon K. Saha , Lingming Zhang , Sarfraz Khurshid , and Dewayne E. Perry . 2015. An information retrieval approach for regression test prioritization based on program changes . In Proceedings of the IEEE\/ACM 37th IEEE International Conference on Software Engineering , Vol. 1 . IEEE, 268--279. Ripon K. Saha, Lingming Zhang, Sarfraz Khurshid, and Dewayne E. Perry. 2015. An information retrieval approach for regression test prioritization based on program changes. In Proceedings of the IEEE\/ACM 37th IEEE International Conference on Software Engineering, Vol. 1. IEEE, 268--279."},{"key":"e_1_2_1_61_1","volume-title":"Introduction to Modern Information Retrieval","author":"Salton Gerard","unstructured":"Gerard Salton and Michael McGill . 1984. Introduction to Modern Information Retrieval . McGraw-Hill Book Company . Gerard Salton and Michael McGill. 1984. Introduction to Modern Information Retrieval. McGraw-Hill Book Company."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786878"},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the 2nd International Conference on Computing for Sustainable Global Development. IEEE, 1793--1797","author":"Singh Pradeep Kumar","year":"2015","unstructured":"Pradeep Kumar Singh , Dishti Agarwal , and Aakriti Gupta . 2015 . A systematic review on software defect prediction . In Proceedings of the 2nd International Conference on Computing for Sustainable Global Development. IEEE, 1793--1797 . Pradeep Kumar Singh, Dishti Agarwal, and Aakriti Gupta. 2015. A systematic review on software defect prediction. In Proceedings of the 2nd International Conference on Computing for Sustainable Global Development. IEEE, 1793--1797."},{"key":"e_1_2_1_64_1","volume-title":"Proceedings of the International Conference on Automated Software Engineering (ASE\u201911)","author":"Sun C.","unstructured":"C. Sun , D. Lo , S. Khoo , and J. Jiang . 2011. Towards more accurate retrieval of duplicate bug reports . In Proceedings of the International Conference on Automated Software Engineering (ASE\u201911) . 253--262. C. Sun, D. Lo, S. Khoo, and J. Jiang. 2011. Towards more accurate retrieval of duplicate bug reports. In Proceedings of the International Conference on Automated Software Engineering (ASE\u201911). 253--262."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.139"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9415-3"},{"key":"e_1_2_1_67_1","volume-title":"Proceedings of the International Conference on Software Testing Verification and Validation. 210--219","author":"Walia G. S.","unstructured":"G. S. Walia and J. C. Carver . 2009. Evaluating the effect of the number of naturally occurring faults on the estimates produced by capture-recapture models . In Proceedings of the International Conference on Software Testing Verification and Validation. 210--219 . G. S. Walia and J. C. Carver. 2009. Evaluating the effect of the number of naturally occurring faults on the estimates produced by capture-recapture models. In Proceedings of the International Conference on Software Testing Verification and Validation. 210--219."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/2961111.2962584"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2017.8"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2019.03.003"},{"key":"#cr-split#-e_1_2_1_71_1.1","doi-asserted-by":"crossref","unstructured":"J. Wang S. Wang J. Chen T. Menzies Q. Cui M. Xie and Q. Wang. 2019. Characterizing crowds to better optimize worker recommendation in crowdsourced testing. IEEE Trans. Softw. Eng. (2019). DOI:https:\/\/doi.org\/10.1109\/TSE.2019.2918520. 10.1109\/TSE.2019.2918520","DOI":"10.1109\/TSE.2019.2918520"},{"key":"#cr-split#-e_1_2_1_71_1.2","doi-asserted-by":"crossref","unstructured":"J. Wang S. Wang J. Chen T. Menzies Q. Cui M. Xie and Q. Wang. 2019. Characterizing crowds to better optimize worker recommendation in crowdsourced testing. IEEE Trans. Softw. Eng. (2019). DOI:https:\/\/doi.org\/10.1109\/TSE.2019.2918520.","DOI":"10.1109\/TSE.2019.2918520"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970276.2970300"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00097"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180254"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884804"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106258"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2876439"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2017.25"},{"key":"e_1_2_1_79_1","volume-title":"Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201916)","author":"Xu B.","unstructured":"B. Xu , D. Ye , Z. Xing , X. Xia , G. Chen , and S. Li . 2016. Predicting semantically linkable knowledge in developer online forums via convolutional neural network . In Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201916) . 51--62. B. Xu, D. Ye, Z. Xing, X. Xia, G. Chen, and S. Li. 2016. Predicting semantically linkable knowledge in developer online forums via convolutional neural network. In Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201916). 51--62."},{"key":"e_1_2_1_80_1","volume-title":"Proceedings of the International Symposium on Software Reliability Engineering (ISSRE\u201916)","author":"Yang X.","unstructured":"X. Yang , D. Lo , X. Xia , L. Bao , and J. Sun . 2016. Combining word embedding with information retrieval to recommend similar bug reports . In Proceedings of the International Symposium on Software Reliability Engineering (ISSRE\u201916) . 127--137. X. Yang, D. Lo, X. Xia, L. Bao, and J. Sun. 2016. Combining word embedding with information retrieval to recommend similar bug reports. In Proceedings of the International Symposium on Software Reliability Engineering (ISSRE\u201916). 127--137."},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9587-0"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180198"},{"key":"e_1_2_1_83_1","volume-title":"Research progress of crowdsourced software testing. Journal of Software 29(1)","author":"Zhang Xiaofang","year":"2018","unstructured":"Xiaofang Zhang , Yang Feng , Di Liu , Zhenyu Chen , and Baowen Xu. 2018. Research progress of crowdsourced software testing. Journal of Software 29(1) ( 2018 ), 69--88. Xiaofang Zhang, Yang Feng, Di Liu, Zhenyu Chen, and Baowen Xu. 2018. Research progress of crowdsourced software testing. Journal of Software 29(1) (2018), 69--88."},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11573-014-0721-9"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394602","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3394602","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:30Z","timestamp":1750195890000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394602"}},"subtitle":["Improving Completion-aware Crowdtesting Management with Duplicate Tagger and Sanity Checker"],"short-title":[],"issued":{"date-parts":[[2020,7,6]]},"references-count":84,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,10,31]]}},"alternative-id":["10.1145\/3394602"],"URL":"https:\/\/doi.org\/10.1145\/3394602","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,6]]},"assertion":[{"value":"2019-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-07-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}