{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T17:27:48Z","timestamp":1777397268706,"version":"3.51.4"},"reference-count":44,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2024,1,8]],"date-time":"2024-01-08T00:00:00Z","timestamp":1704672000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003392","name":"Natural Science Foundation of Fujian Province","doi-asserted-by":"publisher","award":["2022J011238"],"award-info":[{"award-number":["2022J011238"]}],"id":[{"id":"10.13039\/501100003392","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003392","name":"Natural Science Foundation of Fujian Province","doi-asserted-by":"publisher","award":["3502Z20231042"],"award-info":[{"award-number":["3502Z20231042"]}],"id":[{"id":"10.13039\/501100003392","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Xiamen Science and Technology Project","award":["2022J011238"],"award-info":[{"award-number":["2022J011238"]}]},{"name":"Xiamen Science and Technology Project","award":["3502Z20231042"],"award-info":[{"award-number":["3502Z20231042"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Collating vast test reports is a time-consuming and laborious task in crowdsourced testing. Crowdsourced test reports are usually presented in two ways, one as text and the other as images, which have symmetrical content. Researchers have proposed many text- and image-based methods for prioritizing crowdsourced test reports of mobile applications. However, crowdsourced test reports of web applications typically have clearer textual descriptions of errors but noisier screenshots of errors. This gap motivates us to propose a method for prioritizing crowdsourced test reports of web applications to detect all errors earlier. In this paper, we integrate text and image information from test reports to enhance the analysis process. First, we use the natural language processing (NLP) technique to extract textual features of error descriptions and then symmetrically extract image features of error screenshots, i.e., we use the optical character recognition (OCR) technique to obtain textual information in the screenshots and then also use the NLP technique to extract features. To validate our approach, we conduct experiments on 717 test reports. The experimental results show that our method has a higher APFD (average percentage fault detection) and shorter runtime than state-of-the-art prioritization methods.<\/jats:p>","DOI":"10.3390\/sym16010080","type":"journal-article","created":{"date-parts":[[2024,1,8]],"date-time":"2024-01-08T07:59:20Z","timestamp":1704700760000},"page":"80","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Optimizing Prioritization of Crowdsourced Test Reports of Web Applications through Image-to-Text Conversion"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9545-2429","authenticated-orcid":false,"given":"Lei","family":"Xiao","sequence":"first","affiliation":[{"name":"School of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yunhuan","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rongshang","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hanghai","family":"Shi","sequence":"additional","affiliation":[{"name":"Xiamen University of Technology Library, Xiamen University of Technology, Xiamen 361000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.jss.2016.09.015","article-title":"A survey of the use of crowdsourcing in software engineering","volume":"126","author":"Mao","year":"2017","journal-title":"J. Syst. Softw."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"106363","DOI":"10.1016\/j.infsof.2020.106363","article-title":"Crowdsourced software testing: A systematic literature review","volume":"127","author":"Alyahya","year":"2020","journal-title":"Inf. Softw. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Huang, S., Chen, H., Hui, Z., and Liu, Y. (2020, January 11\u201314). A survey of the use of test report in crowdsourced testing. Proceedings of the 2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS), Macau, China.","DOI":"10.1109\/QRS51102.2020.00062"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.infsof.2019.03.003","article-title":"Images don\u2019t lie: Duplicate crowdtesting reports detection with screenshot information","volume":"110","author":"Wang","year":"2019","journal-title":"Inf. Softw. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Feng, Y., Chen, Z., Jones, J.A., Fang, C., and Xu, B. (September, January 30). Test report prioritization to assist crowdsourced testing. Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, Bergamo, Italy.","DOI":"10.1145\/2786805.2786862"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"92692","DOI":"10.1109\/ACCESS.2021.3128726","article-title":"Crowdsourced Test Report Prioritization Based on Text Classification","volume":"10","author":"Yang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"53522","DOI":"10.1109\/ACCESS.2022.3176086","article-title":"Advanced Crowdsourced Test Report Prioritization Based on Adaptive Strategy","volume":"10","author":"Zhu","year":"2022","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Feng, Y., Jones, J.A., Chen, Z., and Fang, C. (2016, January 3\u20137). Multi-objective test report prioritization using image understanding. Proceedings of the 31st IEEE\/ACM International Conference on Automated Software Engineering, Singapore.","DOI":"10.1145\/2970276.2970367"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"106668","DOI":"10.1016\/j.infsof.2021.106668","article-title":"Crowdsourced test report prioritization considering bug severity","volume":"139","author":"Tong","year":"2021","journal-title":"Inf. Softw. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1290","DOI":"10.1109\/TSE.2020.3017514","article-title":"Clustering crowdsourced test reports of mobile applications using image understanding","volume":"48","author":"Liu","year":"2022","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1109\/32.962562","article-title":"Prioritizing test cases for regression testing","volume":"27","author":"Rothermel","year":"2001","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Cui, Q., Wang, J., Yang, G., Xie, M., Wang, Q., and Li, M. (2017, January 4\u20138). Who should be selected to perform a task in crowdsourced testing?. Proceedings of the 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC), Turin, Italy.","DOI":"10.1109\/COMPSAC.2017.265"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Cui, Q., Wang, S., Wang, J., Hu, Y., Wang, Q., and Li, M. (2017, January 5\u20137). Multi-Objective Crowd Worker Selection in Crowdsourced Testing. Proceedings of the SEKE, Pittsburgh, PA, USA.","DOI":"10.18293\/SEKE2017-102"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1259","DOI":"10.1109\/TSE.2019.2918520","article-title":"Characterizing crowds to better optimize worker recommendation in crowdsourced testing","volume":"47","author":"Wang","year":"2019","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_15","unstructured":"Wang, J., Yang, Y., Wang, S., Hu, Y., Wang, D., and Wang, Q. (July, January 27). Context-aware in-process crowdworker recommendation. Proceedings of the ACM\/IEEE 42nd International Conference on Software Engineering, Seoul, Republic of Korea."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3487571","article-title":"Context-and Fairness-Aware In-Process Crowdworker Recommendation","volume":"31","author":"Wang","year":"2022","journal-title":"ACM Trans. Softw. Eng. Methodol. (TOSEM)"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kamangar, Z.U., Kamangar, U.A., Ali, Q., Farah, I., Nizamani, S., and Ali, T.H. (2019, January 9\u201312). To enhance effectiveness of crowdsource software testing by applying personality types. Proceedings of the 2019 8th International Conference on Software and Information Engineering, Cairo, Egypt.","DOI":"10.1145\/3328833.3328838"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Xie, M., Wang, Q., Yang, G., and Li, M. (2017, January 23\u201326). Cocoon: Crowdsourced testing quality maximization under context coverage constraint. Proceedings of the 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE), Toulouse, France.","DOI":"10.1109\/ISSRE.2017.25"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3106164","article-title":"Fuzzy clustering of crowdsourced test reports for apps","volume":"18","author":"Jiang","year":"2018","journal-title":"ACM Trans. Internet Technol. (TOIT)"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chen, H., Huang, S., Liu, Y., Luo, R., and Xie, Y. (2021, January 6\u201310). An Effective Crowdsourced Test Report Clustering Model Based on Sentence Embedding. Proceedings of the 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS), Haikou, China.","DOI":"10.1109\/QRS54544.2021.00098"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yang, Y., Yao, X., and Gong, D. (2019, January 26\u201330). Clustering study of crowdsourced test report with multi-source heterogeneous information. Proceedings of the Data Mining and Big Data: 4th International Conference, DMBD 2019, Chiang Mai, Thailand.","DOI":"10.1007\/978-981-32-9563-6_14"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Cao, Z., Wang, X., Yu, S., Yun, Y., and Fang, C. (2020, January 21\u201325). STIFA: Crowdsourced mobile testing report selection based on text and image fusion analysis. Proceedings of the 35th IEEE\/ACM International Conference on Automated Software Engineering, Virtual Event.","DOI":"10.1145\/3324884.3415300"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Cooper, N., Bernal-C\u00e1rdenas, C., Chaparro, O., Moran, K., and Poshyvanyk, D. (2021, January 22\u201330). It takes two to tango: Combining visual and textual information for detecting duplicate video-based bug reports. Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE), Madrid, Spain.","DOI":"10.1109\/ICSE43902.2021.00091"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Huang, Y., Wang, J., Wang, S., Liu, Z., Hu, Y., and Wang, Q. (2020, January 5\u20139). Quest for the Golden Approach: An Experimental Evaluation of Duplicate Crowdtesting Reports Detection. Proceedings of the 14th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Bari, Italy.","DOI":"10.1145\/3382494.3410694"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yu, S., Fang, C., Cao, Z., Wang, X., Li, T., and Chen, Z. (2021, January 22\u201330). Prioritize crowdsourced test reports via deep screenshot understanding. Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE), Madrid, Spain.","DOI":"10.1109\/ICSE43902.2021.00090"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wang, J., Wang, S., Cui, Q., and Wang, Q. (2016, January 3\u20137). Local-based active classification of test report to assist crowdsourced testing. Proceedings of the 31st IEEE\/ACM International Conference on Automated Software Engineering, Singapore.","DOI":"10.1145\/2970276.2970300"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Guo, S., Chen, R., and Li, H. (2017). Using knowledge transfer and rough set to predict the severity of android test reports via text mining. Symmetry, 9.","DOI":"10.3390\/sym9080161"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, J., Cui, Q., Wang, S., and Wang, Q. (2017, January 20\u201328). Domain adaptation for test report classification in crowdsourced testing. Proceedings of the 2017 IEEE\/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP), Buenos Aires, Argentina.","DOI":"10.1109\/ICSE-SEIP.2017.8"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chen, X., Jiang, H., Li, X., He, T., and Chen, Z. (2018, January 20\u201323). Automated quality assessment for crowdsourced test reports of mobile applications. Proceedings of the 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER), Campobasso, Italy.","DOI":"10.1109\/SANER.2018.8330224"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1142\/S0218194019500256","article-title":"A unified framework for bug report assignment","volume":"29","author":"Zhao","year":"2019","journal-title":"Int. J. Softw. Eng. Knowl. Eng."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"111121","DOI":"10.1016\/j.jss.2021.111121","article-title":"Classifying crowdsourced mobile test reports with image features: An empirical study","volume":"184","author":"Li","year":"2022","journal-title":"J. Syst. Softw."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yu, S., Fang, C., Zhang, Q., Cao, Z., Yun, Y., Cao, Z., Mei, K., and Chen, Z. (2023). Mobile App Crowdsourced Test Report Consistency Detection via Deep Image-and-Text Fusion Understanding. IEEE Trans. Software Eng., 1\u201320.","DOI":"10.1109\/TSE.2023.3285787"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Liu, D., Zhang, X., Feng, Y., and Jones, J.A. (2018, January 20\u201323). Generating descriptions for screenshots to assist crowdsourced testing. Proceedings of the 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER), Campobasso, Italy.","DOI":"10.1109\/SANER.2018.8330246"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1007\/s11704-018-7308-5","article-title":"Automatic test report augmentation to assist crowdsourced testing","volume":"13","author":"Chen","year":"2019","journal-title":"Front. Comput. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Yu, S. (2019, January 11\u201315). Crowdsourced report generation via bug screenshot understanding. Proceedings of the 2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE), San Diego, CA, USA.","DOI":"10.1109\/ASE.2019.00161"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hao, R., Feng, Y., Jones, J.A., Li, Y., and Chen, Z. (2019, January 25\u201331). CTRAS: Crowdsourced test report aggregation and summarization. Proceedings of the 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE), Montreal, QC, Canada.","DOI":"10.1109\/ICSE.2019.00096"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Yu, S., Huang, W., Zhang, J., and Zheng, H. (2022, January 4\u20135). Automatic Bug Inference via Deep Image Understanding. Proceedings of the 2022 9th International Conference on Dependable Systems and Their Applications (DSA), Wulumuqi, China.","DOI":"10.1109\/DSA56465.2022.00051"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Rajaraman, A., and Ullman, J.D. (2011). Mining of Massive Datasets, Cambridge University Press. [1st ed.].","DOI":"10.1017\/CBO9781139058452"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Wang, S., Zhang, W., and Wang, Q. (2014, January 18\u201319). FixerCache: Unsupervised caching active developers for diverse bug triage. Proceedings of the 8th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement, Torino Italy.","DOI":"10.1145\/2652524.2652536"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Rocha, H., Valente, M.T., Marques-Neto, H., and Murphy, G.C. (2016, January 14\u201318). An empirical study on recommendations of similar bugs. Proceedings of the 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Osaka, Japan.","DOI":"10.1109\/SANER.2016.87"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1080\/01621459.1937.10503522","article-title":"The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance","volume":"32","author":"Friedman","year":"1937","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1214\/aoms\/1177730491","article-title":"On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other","volume":"18","author":"Mann","year":"1947","journal-title":"Ann. Math. Statist."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Salman, I., Misirli, A.T., and Juristo, N. (2015, January 16\u201324). Are students representatives of professionals in software engineering experiments?. Proceedings of the 2015 IEEE\/ACM 37th IEEE International Conference on Software Engineering, Florence, Italy.","DOI":"10.1109\/ICSE.2015.82"},{"key":"ref_44","unstructured":"Yunhuan (2023, September 12). Fusing Text and Images to Better Prioritize Crowdsourced Test Report for Web Application. V1. Available online: https:\/\/doi.org\/10.57760\/sciencedb.09699."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/1\/80\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:42:00Z","timestamp":1760103720000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/1\/80"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,8]]},"references-count":44,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["sym16010080"],"URL":"https:\/\/doi.org\/10.3390\/sym16010080","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,8]]}}}