{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T10:14:26Z","timestamp":1753870466028,"version":"3.41.2"},"reference-count":60,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2023,2,9]],"date-time":"2023-02-09T00:00:00Z","timestamp":1675900800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62172202","61772263","61772014"],"award-info":[{"award-number":["62172202","61772263","61772014"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["BK20211307"],"award-info":[{"award-number":["BK20211307"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012246","name":"Priority Academic Program Development of Jiangsu Higher Education Institutions","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100012246","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J Software Evolu Process"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In the process of mobile crowdsourced testing, a large number of test reports are generated, which often consist of short text and rich image information. One of the critical issues is how to review these reports efficiently. Researchers have recently proposed clustering, classification, and prioritization techniques to solve this problem. However, existing studies directly use related technologies to text description and segment the text content without further understanding of the text content. By deeply digging into the text information and distinguishing it according to its actual meaning, the sentences described in the text can be categorized into two types: Describing abnormal system behavior and describing reproduction steps. This paper proposes a mobile crowdsourced test report prioritization technique to improve performance. First, we use a TextCNN trained on large\u2010scale projects to distinguish the text descriptions of reports, then extract features from the text and screenshot information, respectively. Then we apply a clustering technique to gather similar reports. Finally, the inspection order is sampled from the clustering results. To validate our approach, we conduct experiments on six industrial crowdsourced projects. The results show that our method can detect all bugs faster in a limited time than existing prioritization methods, which can improve the bug reports review efficiency.<\/jats:p>","DOI":"10.1002\/smr.2541","type":"journal-article","created":{"date-parts":[[2023,2,9]],"date-time":"2023-02-09T08:59:58Z","timestamp":1675933198000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Mobile crowdsourced test report prioritization based on text and image understanding"],"prefix":"10.1002","volume":"36","author":[{"given":"Yifan","family":"Wu","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology Soochow University Suzhou China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yao","family":"Tong","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology Soochow University Suzhou China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"An","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology Soochow University Suzhou China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lei","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology Soochow University Suzhou China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8667-0456","authenticated-orcid":false,"given":"Xiaofang","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology Soochow University Suzhou China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2023,2,9]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"crossref","unstructured":"WangJ CuiQ WangS WangQ.Domain adaptation for test report classification in crowdsourced testing. In: 2017 IEEE\/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE\u2010SEIP).IEEE;2017:83\u201092.","DOI":"10.1109\/ICSE-SEIP.2017.8"},{"key":"e_1_2_10_3_1","doi-asserted-by":"crossref","unstructured":"HaoR FengY JonesJA LiY ChenZ.CTRAS: crowdsourced test report aggregation and summarization. In: 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE).IEEE;2019:900\u2010911.","DOI":"10.1109\/ICSE.2019.00096"},{"key":"e_1_2_10_4_1","doi-asserted-by":"crossref","unstructured":"FazziniM PrammerM d'AmorimM OrsoA.Automatically translating bug reports into test cases for mobile apps. In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis.ACM;2018:141\u2010152.","DOI":"10.1145\/3213846.3213869"},{"issue":"1","key":"e_1_2_10_5_1","first-page":"69","article-title":"Research progress of crowdsourced software testing","volume":"29","author":"Zhang X","year":"2018","journal-title":"J Softw"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.3017514"},{"key":"e_1_2_10_7_1","doi-asserted-by":"crossref","unstructured":"FengY JonesJA ChenZ FangC.Multi\u2010objective test report prioritization using image understanding. In: 2016 31st IEEE\/ACM International Conference on Automated Software Engineering (ASE).IEEE;2016:202\u2010213.","DOI":"10.1145\/2970276.2970367"},{"key":"e_1_2_10_8_1","unstructured":"ChenY.Convolutional neural network for sentence classification.Master's Thesis: University of Waterloo. Waterloo Ontario Canada;2015."},{"key":"e_1_2_10_9_1","doi-asserted-by":"crossref","unstructured":"LazebnikS SchmidC PonceJ.Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).IEEE;2006:2169\u20102178.","DOI":"10.1109\/CVPR.2006.68"},{"key":"e_1_2_10_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.12"},{"key":"e_1_2_10_11_1","doi-asserted-by":"crossref","unstructured":"YangY YaoX GongD.Clustering study of crowdsourced test report with multi\u2010source heterogeneous information. In: International Conference on Data Mining and Big Data.Springer;2019:135\u2010145.","DOI":"10.1007\/978-981-32-9563-6_14"},{"key":"e_1_2_10_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.05.026"},{"key":"e_1_2_10_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2850910"},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110567"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.4236\/oalib.1101229"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.07.052"},{"key":"e_1_2_10_17_1","doi-asserted-by":"crossref","unstructured":"SongP GengC LiZ.Research on text classification based on convolutional neural network. In: 2019 International Conference on Computer Network Electronic and Automation (ICCNEA).IEEE;2019:229\u2010232.","DOI":"10.1109\/ICCNEA.2019.00052"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2016.2547588"},{"key":"e_1_2_10_19_1","doi-asserted-by":"crossref","unstructured":"WangX ZhangL XieT AnvikJ SunJ.An approach to detecting duplicate bug reports using natural language and execution information. In: Proceedings of the 30th International Conference on Software Engineering.ACM;2008:461\u2010470.","DOI":"10.1145\/1368088.1368151"},{"key":"e_1_2_10_20_1","doi-asserted-by":"crossref","unstructured":"IlievaMG OrmandjievaO.Automatic transition of natural language software requirements specification into formal presentation. In: International Conference on Application of Natural Language to Information Systems.Springer;2005:392\u2010397.","DOI":"10.1007\/11428817_45"},{"key":"e_1_2_10_21_1","unstructured":"ShutovaE SunL KorhonenA.Metaphor identification using verb and noun clustering. In: Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010).Association for Computational Linguistics;2010:1002\u20101010."},{"key":"e_1_2_10_22_1","doi-asserted-by":"crossref","unstructured":"DiabM BhutadaP.Verb noun construction MWE token classification. In: Proceedings of the Workshop on Multiword Expressions: Identification Interpretation Disambiguation and Applications (MWE 2009).Association for Computational Linguistics;2009:17\u201022.","DOI":"10.3115\/1698239.1698243"},{"key":"e_1_2_10_23_1","doi-asserted-by":"crossref","unstructured":"NowakE JurieF TriggsB.Sampling strategies for bag\u2010of\u2010features image classification. In: European Conference on Computer Vision.Springer;2006:490\u2010503.","DOI":"10.1007\/11744085_38"},{"key":"e_1_2_10_24_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:VISI.0000029664.99615.94"},{"key":"e_1_2_10_25_1","doi-asserted-by":"crossref","unstructured":"ZhangT YangB.Big data dimension reduction using PCA. In: 2016 IEEE International Conference on Smart Cloud (SmartCloud).IEEE;2016:152\u2010157.","DOI":"10.1109\/SmartCloud.2016.33"},{"key":"e_1_2_10_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvcir.2019.102578"},{"key":"e_1_2_10_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/235968.233324"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0031-3203(02)00060-2"},{"key":"e_1_2_10_29_1","first-page":"226","volume-title":"KDD'96","author":"Ester M","year":"1996"},{"key":"e_1_2_10_30_1","unstructured":"McCallumA NigamK et al.A comparison of event models for naive Bayes text classification. In: AAAI\u201098 Workshop on Learning for Text Categorization.AAAI Press;1998:41\u201048."},{"key":"e_1_2_10_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00994018"},{"key":"e_1_2_10_32_1","unstructured":"JoulinA GraveE BojanowskiP MikolovT.Bag of tricks for efficient text classification.2016. arXiv e\u2010prints 2016: arXiv\u20131607."},{"key":"e_1_2_10_33_1","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/26.4.354"},{"key":"e_1_2_10_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.962562"},{"key":"e_1_2_10_35_1","doi-asserted-by":"crossref","unstructured":"LedruY PetrenkoA BorodayS.Using string distances for test case prioritisation. In: 2009 IEEE\/ACM International Conference on Automated Software Engineering.IEEE;2009:510\u2010514.","DOI":"10.1109\/ASE.2009.23"},{"key":"e_1_2_10_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-4380-9_16"},{"key":"e_1_2_10_37_1","doi-asserted-by":"publisher","DOI":"10.4300\/JGME-D-12-00156.1"},{"issue":"9","key":"e_1_2_10_38_1","first-page":"1","article-title":"How to calculate effect sizes from published research: a simplified methodology","volume":"1","author":"Thalheimer W","year":"2002","journal-title":"Work\u2010Learn Res"},{"key":"e_1_2_10_39_1","doi-asserted-by":"publisher","DOI":"10.5540\/tema.2014.015.01.0003"},{"key":"e_1_2_10_40_1","doi-asserted-by":"crossref","unstructured":"SalmanI MisirliAT JuristoN.Are students representatives of professionals in software engineering experiments?In: 2015 IEEE\/ACM 37th IEEE international conference on software engineering.IEEE;2015:666\u2010676.","DOI":"10.1109\/ICSE.2015.82"},{"key":"e_1_2_10_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.377142400"},{"key":"e_1_2_10_42_1","doi-asserted-by":"crossref","unstructured":"WangJ CuiQ WangQ WangS.Towards effectively test report classification to assist crowdsourced testing. In: Proceedings of the 10th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement.ACM;2016:1\u201010.","DOI":"10.1145\/2961111.2962584"},{"key":"e_1_2_10_43_1","doi-asserted-by":"crossref","unstructured":"ChenH HuangS LiuY LuoR XieY.An effective crowdsourced test report clustering model based on sentence embedding. In: 2021 IEEE 21st International Conference on Software Quality Reliability and Security (QRS).IEEE;2021:888\u2010899.","DOI":"10.1109\/QRS54544.2021.00098"},{"key":"e_1_2_10_44_1","doi-asserted-by":"crossref","unstructured":"MishraSK HarshavardhanK MitraS SahaS BhattacharyyaP.Bug report summarization using multi\u2010view multi\u2010objective optimization framework. In: Proceedings of the Genetic and Evolutionary Computation Conference.Association for Computing Machinery;2022:1245\u20101253.","DOI":"10.1145\/3512290.3528843"},{"key":"e_1_2_10_45_1","doi-asserted-by":"crossref","unstructured":"CaiL WangN ChenM WangJ WangJ GongJ.Reports aggregation of crowdsourcing test based on feature fusion. In: 2021 IEEE 21st International Conference on Software Quality Reliability and Security Companion (QRS\u2010C).IEEE;2021:51\u201059.","DOI":"10.1109\/QRS-C55045.2021.00018"},{"key":"e_1_2_10_46_1","doi-asserted-by":"crossref","unstructured":"RunesonP AlexanderssonM NyholmO.Detection of duplicate defect reports using natural language processing. In: 29th International Conference on Software Engineering (ICSE'07).IEEE;2007:499\u2010510.","DOI":"10.1109\/ICSE.2007.32"},{"key":"e_1_2_10_47_1","doi-asserted-by":"crossref","unstructured":"SunC LoD KhooS\u2010C JiangJ.Towards more accurate retrieval of duplicate bug reports. In: 2011 26th IEEE\/ACM International Conference on Automated Software Engineering ASE 2011 Proceedings.IEEE Computer Society;2011:253\u2010262.","DOI":"10.1109\/ASE.2011.6100061"},{"key":"e_1_2_10_48_1","doi-asserted-by":"crossref","unstructured":"NguyenAT NguyenTT NguyenTN LoD SunC.Duplicate bug report detection with a combination of information retrieval and topic modeling. In: 2012 Proceedings of the 27th IEEE\/ACM International Conference on Automated Software Engineering.IEEE;2012:70\u201079.","DOI":"10.1145\/2351676.2351687"},{"key":"e_1_2_10_49_1","doi-asserted-by":"crossref","unstructured":"LazarA RitcheyS SharifB.Improving the accuracy of duplicate bug report detection using textual similarity measures. In: Proceedings of the 11th Working Conference on Mining Software Repositories.ACM;2014:308\u2010311.","DOI":"10.1145\/2597073.2597088"},{"key":"e_1_2_10_50_1","doi-asserted-by":"crossref","unstructured":"SunC LoD WangX JiangJ KhooS\u2010C.A discriminative model approach for accurate duplicate bug report retrieval. In: Proceedings of the 32nd ACM\/IEEE International Conference on Software Engineering\u2010Volume 1.ACM;2010:45\u201054.","DOI":"10.1145\/1806799.1806811"},{"key":"e_1_2_10_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.02.022"},{"key":"e_1_2_10_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2019.05.007"},{"key":"e_1_2_10_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106344"},{"key":"e_1_2_10_54_1","doi-asserted-by":"crossref","unstructured":"XieQ WenZ ZhuJ GaoC ZhengZ.Detecting duplicate bug reports with convolutional neural networks. In: 2018 25th Asia\u2010Pacific Software Engineering Conference (APSEC).IEEE;2018:416\u2010425.","DOI":"10.1109\/APSEC.2018.00056"},{"key":"e_1_2_10_55_1","doi-asserted-by":"crossref","unstructured":"XiaoG DuX SuiY YueT.Hindbr: Heterogeneous information network based duplicate bug report prediction. In: 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE).IEEE;2020:195\u2010206.","DOI":"10.1109\/ISSRE5003.2020.00027"},{"key":"e_1_2_10_56_1","doi-asserted-by":"crossref","unstructured":"IsotaniH WashizakiH FukazawaY NomotoT OujiS SaitoS.Duplicate bug report detection by using sentence embedding and fine\u2010tuning. In: 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME).IEEE;2021:535\u2010544.","DOI":"10.1109\/ICSME52107.2021.00054"},{"key":"e_1_2_10_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2022.3193645"},{"key":"e_1_2_10_58_1","doi-asserted-by":"crossref","unstructured":"TianY LoD SunC.Drone: Predicting priority of reported bugs by multi\u2010factor analysis. In: 2013 IEEE International Conference on Software Maintenance.IEEE;2013:200\u2010209.","DOI":"10.1109\/ICSM.2013.31"},{"key":"e_1_2_10_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-014-9331-y"},{"key":"e_1_2_10_60_1","doi-asserted-by":"crossref","unstructured":"FengY ChenZ JonesJA FangC XuB.Test report prioritization to assist crowdsourced testing. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering.ACM;2015:225\u2010236.","DOI":"10.1145\/2786805.2786862"},{"key":"e_1_2_10_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2019.2959624"}],"container-title":["Journal of Software: Evolution and Process"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/smr.2541","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/smr.2541","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/smr.2541","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,13]],"date-time":"2024-10-13T21:50:40Z","timestamp":1728856240000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/smr.2541"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,9]]},"references-count":60,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["10.1002\/smr.2541"],"URL":"https:\/\/doi.org\/10.1002\/smr.2541","archive":["Portico"],"relation":{},"ISSN":["2047-7473","2047-7481"],"issn-type":[{"type":"print","value":"2047-7473"},{"type":"electronic","value":"2047-7481"}],"subject":[],"published":{"date-parts":[[2023,2,9]]},"assertion":[{"value":"2022-08-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-01-20","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2541"}}