{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T14:50:21Z","timestamp":1783954221581,"version":"3.55.0"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T00:00:00Z","timestamp":1718755200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T00:00:00Z","timestamp":1718755200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"the Key Field Research and Development Plan of Guangdong 606 Province","award":["2022B0101070001"],"award-info":[{"award-number":["2022B0101070001"]}]},{"name":"the second batch of cultivation projects of Pazhou Laboratory","award":["PZL2022KF0008"],"award-info":[{"award-number":["PZL2022KF0008"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Intell Syst"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Software Defect Detection (SDD) has always been critical to the development life cycle. A stable defect detection system can not only alleviate the workload of software testers but also enhance the overall efficiency of software development. Researchers have recently proposed various artificial intelligence-based SDD methods and achieved significant advancements. However, these methods still exhibit limitations in terms of reliability and usability. Therefore, we introduce MSDD-(IA)<jats:sup>3<\/jats:sup>, a novel framework leveraging the pre-trained CodeT5+ and (IA)<jats:sup>3<\/jats:sup>for parameter-efficient multi-classification SDD. This framework constructs a detection model based on pre-trained CodeT5+ to generate code representations while capturing defect-prone features. Considering the high overhead of pre-trained LLMs, we injects (IA)<jats:sup>3<\/jats:sup>vectors into specific layers, where only these injected parameters are updated to reduce the training cost. Furthermore, leveraging the properties of the pre-trained CodeT5+, we design a novel feature sequence that enriches the input data through the combination of source code with Natural Language (NL)-based expert metrics. Our experimental results on 64K real-world Python snippets show that MSDD-(IA)<jats:sup>3<\/jats:sup>demonstrates superior performance compared to state-of-the-art SDD methods, including PM2-CNN, in terms of F1-weighted, Recall-weighted, Precision-weighted, and Matthews Correlation Coefficient. Notably, the training parameters of MSDD-(IA)<jats:sup>3<\/jats:sup>are only 0.04% of those of the original CodeT5+. Our experimental data and code can be available at (<jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/gitee.com\/wxyzjp123\/msdd-ia3\/\">https:\/\/gitee.com\/wxyzjp123\/msdd-ia3\/<\/jats:ext-link>).<\/jats:p>","DOI":"10.1007\/s44196-024-00551-3","type":"journal-article","created":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T08:01:54Z","timestamp":1718784114000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Parameter-Efficient Multi-classification Software Defect Detection Method Based on Pre-trained LLMs"],"prefix":"10.1007","volume":"17","author":[{"given":"Xuanye","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-7422-1057","authenticated-orcid":false,"given":"Lu","family":"Lu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhanyu","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qingyan","family":"Tian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haisha","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,6,19]]},"reference":[{"key":"551_CR1","doi-asserted-by":"crossref","unstructured":"Yang, P., Zhu, L., Zhang, Y., Ma, C., Liu, L., Yu, X., Hu, W.: On the relative value of clustering techniques for unsupervised effort-aware defect prediction. Expert Systems with Applications, p. 123041 (2023)","DOI":"10.1016\/j.eswa.2023.123041"},{"key":"551_CR2","unstructured":"Zhang, D.: Applying machine learning algorithms in software development. In: Proceedings of the 2000 Monterey Workshop on Modeling Software System Structures in a Fastly Moving Scenario, pp. 275\u2013291 (2000)"},{"key":"551_CR3","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1007\/s10664-012-9218-8","volume":"19","author":"A Okutan","year":"2014","unstructured":"Okutan, A., Y\u0131ld\u0131z, O.T.: Software defect prediction using Bayesian networks. Empir. Softw. Eng. 19, 154\u2013181 (2014)","journal-title":"Empir. Softw. Eng."},{"key":"551_CR4","doi-asserted-by":"crossref","unstructured":"Zhang, X., Ben, K., Zeng, J.: Cross-entropy: A new metric for software defect prediction. In: 2018 IEEE International Conference on Software Quality, Reliability and Security (QRS). pp. 111\u2013122. IEEE (2018)","DOI":"10.1109\/QRS.2018.00025"},{"key":"551_CR5","doi-asserted-by":"crossref","unstructured":"Wang, S., Liu, T., Tan, L.: Automatically learning semantic features for defect prediction. In: Proceedings of the 38th International Conference on Software Engineering, pp. 297\u2013308 (2016)","DOI":"10.1145\/2884781.2884804"},{"key":"551_CR6","doi-asserted-by":"crossref","unstructured":"Li, J., He, P., Zhu, J., Lyu, M.R.: Software defect prediction via convolutional neural network. In: 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), pp. 318\u2013328. IEEE (2017)","DOI":"10.1109\/QRS.2017.42"},{"issue":"10","key":"551_CR7","doi-asserted-by":"publisher","first-page":"2138","DOI":"10.3390\/app9102138","volume":"9","author":"C Pan","year":"2019","unstructured":"Pan, C., Lu, M., Xu, B., Gao, H.: An improved cnn model for within-project software defect prediction. Appl. Sci. 9(10), 2138 (2019)","journal-title":"Appl. Sci."},{"issue":"4","key":"551_CR8","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1049\/iet-sen.2019.0149","volume":"14","author":"J Deng","year":"2020","unstructured":"Deng, J., Lu, L., Qiu, S.: Software defect prediction via lstm. IET Softw. 14(4), 443\u2013450 (2020)","journal-title":"IET Softw."},{"key":"551_CR9","doi-asserted-by":"publisher","first-page":"13112","DOI":"10.1109\/ACCESS.2021.3051957","volume":"9","author":"J Lin","year":"2021","unstructured":"Lin, J., Lu, L.: Semantic feature learning via dual sequences for defect prediction. IEEE Access 9, 13112\u201313124 (2021)","journal-title":"IEEE Access"},{"key":"551_CR10","doi-asserted-by":"crossref","unstructured":"Fu, M., Tantithamthavorn, C.: Linevul: A transformer-based line-level vulnerability prediction. In: Proceedings of the 19th International Conference on Mining Software Repositories, pp. 608\u2013620 (2022)","DOI":"10.1145\/3524842.3528452"},{"key":"551_CR11","doi-asserted-by":"crossref","unstructured":"Fu, M., Tantithamthavorn, C., Le, T., Nguyen, V., Phung, D.: Vulrepair: a t5-based automated software vulnerability repair. In: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 935\u2013947 (2022)","DOI":"10.1145\/3540250.3549098"},{"key":"551_CR12","doi-asserted-by":"crossref","unstructured":"Liu, J., Ai, J., Lu, M., Wang, J., Shi, H.: Semantic feature learning for software defect prediction from source code and external knowledge. J. Syst. Softw., p. 111753 (2023)","DOI":"10.1016\/j.jss.2023.111753"},{"key":"551_CR13","doi-asserted-by":"crossref","unstructured":"Wang, Y., Le, H., Gotmare, A.D., Bui, N.D., Li, J., Hoi, S.C.: Codet5+: open code large language models for code understanding and generation. arXiv preprint arXiv:2305.07922 (2023)","DOI":"10.18653\/v1\/2023.emnlp-main.68"},{"key":"551_CR14","doi-asserted-by":"crossref","unstructured":"Mamede, C., Pinconschi, E., Abreu, R., Campos, J.: Exploring transformers for multi-label classification of java vulnerabilities. In: 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS), pp. 43\u201352. IEEE (2022)","DOI":"10.1109\/QRS57517.2022.00015"},{"key":"551_CR15","first-page":"1950","volume":"35","author":"H Liu","year":"2022","unstructured":"Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang, T., Bansal, M., Raffel, C.A.: Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Adv. Neural. Inf. Process. Syst. 35, 1950\u20131965 (2022)","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"551_CR16","unstructured":"Malohtra, R., Yadav, H.S.: An improved cnn-based architecture for within-project software defect prediction. In: Soft Computing and Signal Processing: Proceedings of 3rd ICSCSP 2020, Volume 1. pp. 335\u2013349. Springer (2021)"},{"issue":"1","key":"551_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s44196-023-00400-9","volume":"17","author":"S Li","year":"2024","unstructured":"Li, S., Wang, J., Song, Y., Wang, S., Wang, Y.: A lightweight model for malicious code classification based on structural reparameterisation and large convolutional kernels. Int. J. Comput. Intell. Syst. 17(1), 1\u201318 (2024)","journal-title":"Int. J. Comput. Intell. Syst."},{"issue":"2","key":"551_CR18","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1109\/TR.2020.3047396","volume":"70","author":"H Wang","year":"2021","unstructured":"Wang, H., Zhuang, W., Zhang, X.: Software defect prediction based on gated hierarchical lstms. IEEE Trans. Reliab. 70(2), 711\u2013727 (2021)","journal-title":"IEEE Trans. Reliab."},{"key":"551_CR19","doi-asserted-by":"crossref","unstructured":"Zeng, C., Zhou, C.Y., Lv, S.K., He, P., Huang, J.: Gcn2defect: Graph convolutional networks for smotetomek-based software defect prediction. In: 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE), pp. 69\u201379. IEEE (2021)","DOI":"10.1109\/ISSRE52982.2021.00020"},{"key":"551_CR20","doi-asserted-by":"crossref","unstructured":"Tang, L., Tao, C., Guo, H., Zhang, J.: Software defect prediction via gcn based on structural and context information. In: 2022 9th International Conference on Dependable Systems and Their Applications (DSA), pp. 310\u2013319. IEEE (2022)","DOI":"10.1109\/DSA56465.2022.00049"},{"key":"551_CR21","doi-asserted-by":"publisher","first-page":"10402","DOI":"10.1109\/ACCESS.2022.3144598","volume":"10","author":"L \u0160iki\u0107","year":"2022","unstructured":"\u0160iki\u0107, L., Kurdija, A.S., Vladimir, K., \u0160ili\u0107, M.: Graph neural network for source code defect prediction. IEEE Access 10, 10402\u201310415 (2022)","journal-title":"IEEE Access"},{"key":"551_CR22","unstructured":"Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)"},{"key":"551_CR23","doi-asserted-by":"crossref","unstructured":"Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et\u00a0al.: Codebert: a pre-trained model for programming and natural languages. arXiv preprint arXiv:2002.08155 (2020)","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"551_CR24","doi-asserted-by":"crossref","unstructured":"Guo, D., Lu, S., Duan, N., Wang, Y., Zhou, M., Yin, J.: Unixcoder: unified cross-modal pre-training for code representation. arXiv preprint arXiv:2203.03850 (2022)","DOI":"10.18653\/v1\/2022.acl-long.499"},{"key":"551_CR25","doi-asserted-by":"crossref","unstructured":"Wang, Y., Wang, W., Joty, S., Hoi, S.C.: Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv preprint arXiv:2109.00859 (2021)","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"issue":"5","key":"551_CR26","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1016\/0893-6080(89)90020-8","volume":"2","author":"K Hornik","year":"1989","unstructured":"Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359\u2013366 (1989)","journal-title":"Neural Netw."},{"issue":"5","key":"551_CR27","doi-asserted-by":"publisher","first-page":"2585","DOI":"10.1007\/s10664-017-9513-5","volume":"22","author":"Y Gil","year":"2017","unstructured":"Gil, Y., Lalouche, G.: On the correlation between size and metric validity. Empir. Softw. Eng. 22(5), 2585\u20132611 (2017)","journal-title":"Empir. Softw. Eng."},{"key":"551_CR28","unstructured":"Halstead, M.H.: Elements of Software Science (Operating and programming systems series). Elsevier Science Inc. (1977)"},{"key":"551_CR29","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1109\/TSE.1976.233837","volume":"4","author":"TJ McCabe","year":"1976","unstructured":"McCabe, T.J.: A complexity measure. IEEE Trans. Softw. Eng. 4, 308\u2013320 (1976)","journal-title":"IEEE Trans. Softw. Eng."},{"key":"551_CR30","doi-asserted-by":"crossref","unstructured":"Oman, P., Hagemeister, J.: Metrics for assessing a software system\u2019s maintainability. In: Proceedings Conference on Software Maintenance 1992, pp. 337\u2013338. IEEE Computer Society (1992)","DOI":"10.1109\/ICSM.1992.242525"},{"key":"551_CR31","doi-asserted-by":"crossref","unstructured":"Wang, C., Cho, K., Gu, J.: Neural machine translation with byte-level subwords. In: Proceedings of the AAAI conference on artificial intelligence. vol.\u00a034, pp. 9154\u20139160 (2020)","DOI":"10.1609\/aaai.v34i05.6451"},{"key":"551_CR32","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)"},{"key":"551_CR33","doi-asserted-by":"crossref","unstructured":"Akimova, E.N., Bersenev, A.Y., Deikov, A.A., Kobylkin, K.S., Konygin, A.V., Mezentsev, I.P., Misilov, V.E.: Pytracebugs: a large python code dataset for supervised machine learning in software defect prediction. In: 2021 28th Asia-Pacific Software Engineering Conference (APSEC), pp. 141\u2013151. IEEE (2021)","DOI":"10.1109\/APSEC53868.2021.00022"},{"key":"551_CR34","doi-asserted-by":"crossref","unstructured":"Zhoua, Y., Lua, L., Zoub, Q., Lic, C.: Two-stage ast encoding for software defect prediction. In: 2022 34th International Conference on Software Engineering and Knowledge Engineering (SEKE), pp. 196\u2013199 (2022)","DOI":"10.18293\/SEKE2022-039"},{"key":"551_CR35","unstructured":"Sklearn evaluation metrics. [online] Available: https:\/\/scikit-learn.org\/stable\/modules\/model_evaluation.html#model-evaluation"},{"key":"551_CR36","doi-asserted-by":"crossref","unstructured":"Yi, X., Xu, H., Lu, L., Zou, Q., Yang, Z.: Software defect prediction via positional hierarchical attention network(S). In: 2023 35th International Conference on Software Engineering and Knowledge Engineering (SEKE). pp. 228\u2013231 (2023)","DOI":"10.18293\/SEKE2023-119"},{"key":"551_CR37","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2023.110941","volume":"149","author":"X Yu","year":"2023","unstructured":"Yu, X., Liu, L., Zhu, L., Keung, J.W., Wang, Z., Li, F.: A multi-objective effort-aware defect prediction approach based on nsga-ii. Appl. Soft Comput. 149, 110941 (2023)","journal-title":"Appl. Soft Comput."},{"key":"551_CR38","doi-asserted-by":"crossref","unstructured":"Zou, Q., Lu, L., Yang, Z., Xu, H.: Multi-source cross project defect prediction with joint wasserstein distance and ensemble learning. In: 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). pp. 57\u201368. IEEE (2021)","DOI":"10.1109\/ISSRE52982.2021.00019"},{"issue":"1","key":"551_CR39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s44196-023-00386-4","volume":"16","author":"Z Ai","year":"2023","unstructured":"Ai, Z., Yijia, Z., Mingyu, L.: A domain knowledge transformer model for occupation profiling. Int. J. Comput Intell. Syst. 16(1), 1\u201313 (2023)","journal-title":"Int. J. Comput Intell. Syst."},{"issue":"1","key":"551_CR40","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1007\/s44196-023-00305-7","volume":"16","author":"QT Pham Thi","year":"2023","unstructured":"Pham Thi, Q.T., Dao, Q.H., Nguyen, A.D., Dang, T.H.: Document-level chemical-induced disease semantic relation extraction using bidirectional long short-term memory on dependency graph. Int. J. Comput Intell. Syst. 16(1), 131 (2023)","journal-title":"Int. J. Comput Intell. Syst."},{"issue":"1","key":"551_CR41","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1007\/s44196-023-00395-3","volume":"17","author":"X Qin","year":"2024","unstructured":"Qin, X., Wang, C., Yuan, Y., Qi, R.: Prediction of in-class performance based on mfo-attention-lstm. Int. J. Comput Intell. Syst. 17(1), 13 (2024)","journal-title":"Int. J. Comput Intell. Syst."},{"key":"551_CR42","doi-asserted-by":"crossref","unstructured":"Zheng, W., Shen, T., Chen, X., Deng, P.: Interpretability application of the just-in-time software defect prediction model. J. Syst. Softw. 188, 111245 (2022)","DOI":"10.1016\/j.jss.2022.111245"}],"container-title":["International Journal of Computational Intelligence Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44196-024-00551-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s44196-024-00551-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44196-024-00551-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,22]],"date-time":"2024-11-22T06:15:02Z","timestamp":1732256102000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s44196-024-00551-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,19]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["551"],"URL":"https:\/\/doi.org\/10.1007\/s44196-024-00551-3","relation":{},"ISSN":["1875-6883"],"issn-type":[{"value":"1875-6883","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,19]]},"assertion":[{"value":"2 February 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 May 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 June 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"152"}}