{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T02:37:18Z","timestamp":1790995038367,"version":"4.1.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:00:00Z","timestamp":1750291200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,19]]},"abstract":"<jats:p>Deep Learning-based Vulnerability Detection (DLVD) techniques have garnered significant interest due to their ability to automatically learn vulnerability patterns from previously compromised code. Despite the notable accuracy demonstrated by pioneering tools, the broader application of DLVD methods in real-world scenarios is hindered by significant challenges. A primary issue is the \u201cone-for-all\u201d design, where a single model is trained to handle all types of vulnerabilities. This approach fails to capture the patterns of different vulnerability types, resulting in suboptimal performance, particularly for less common vulnerabilities that are often underrepresented in training datasets. To address these challenges, we propose MoEVD, which adopts the Mixture-of-Experts (MoE) framework for vulnerability detection. MoEVD decomposes vulnerability detection into two tasks: CWE type classification and CWE-specific vulnerability detection. By splitting the task, in vulnerability detection, MoEVD allows specific experts to handle distinct types of vulnerabilities instead of handling all vulnerabilities within one model. Our results show that MoEVD achieves an F1-score of 0.44, significantly outperforming all studied state-of-the-art (SOTA) baselines by at least 12.8%. MoEVD excels across almost all CWE types, improving recall over the best SOTA baseline by 9% to 77.8%. Notably, MoEVD does not sacrifice performance on long-tailed CWE types; instead, its MoE design enhances performance (F1-score) on these by at least 7.3%, addressing long-tailed issues effectively.<\/jats:p>","DOI":"10.1145\/3715736","type":"journal-article","created":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T11:15:34Z","timestamp":1750331734000},"page":"446-464","source":"Crossref","is-referenced-by-count":12,"title":["One-for-All Does Not Work! Enhancing Vulnerability Detection by Mixture-of-Experts (MoE)"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9963-6225","authenticated-orcid":false,"given":"Xu","family":"Yang","sequence":"first","affiliation":[{"name":"University of Manitoba, Winnipeg, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3823-1771","authenticated-orcid":false,"given":"Shaowei","family":"Wang","sequence":"additional","affiliation":[{"name":"University of Manitoba, Winnipeg, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5181-3146","authenticated-orcid":false,"given":"Jiayuan","family":"Zhou","sequence":"additional","affiliation":[{"name":"Huawei, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6439-0720","authenticated-orcid":false,"given":"Wenhan","family":"Zhu","sequence":"additional","affiliation":[{"name":"Huawei, Waterloo, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,19]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","unstructured":"2024. MoEVD replication pakage. Zenodo. doi:10.5281\/zenodo.11661787","DOI":"10.5281\/zenodo.11661787"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCC50000.2020.9219568"},{"issue":"2021","key":"e_1_3_1_4_2","doi-asserted-by":"crossref","first-page":"1937","DOI":"10.1007\/s10462-020-09896-5","article-title":"A comparative analysis of gradient boosting algorithms","volume":"54","author":"Bent\u00e9jac Candice","year":"2021","unstructured":"Candice Bent\u00e9jac, Anna Cs\u00f6rg\u0151, and Gonzalo Mart\u00ednez-Mu\u00f1oz. 2021. A comparative analysis of gradient boosting algorithms. Artificial Intelligence Review 54 (2021), 1937\u20131967.","journal-title":"Artificial Intelligence Review"},{"issue":"2016","key":"e_1_3_1_5_2","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1007\/s11749-016-0481-7","article-title":"A random forest guided tour","volume":"25","author":"Biau G\u00e9rard","year":"2016","unstructured":"G\u00e9rard Biau and Erwan Scornet. 2016. A random forest guided tour. Test 25 (2016), 197\u2013227.","journal-title":"Test"},{"issue":"2021","key":"e_1_3_1_6_2","first-page":"3280","article-title":"Deep learning based vulnerability detection: Are we there yet?","volume":"9","author":"Chakraborty Saikat","year":"2021","unstructured":"Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2021. Deep learning based vulnerability detection: Are we there yet? IEEE Transactions on Software Engineering 48, 9 (2021), 3280\u20133296.","journal-title":"IEEE Transactions on Software Engineering 48"},{"key":"e_1_3_1_7_2","doi-asserted-by":"crossref","unstructured":"Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 785\u2013794.","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_3_1_8_2","unstructured":"Steve Christey J Kenderdine J Mazella and B Miles. 2013. Common weakness enumeration. Mitre Corporation (2013)."},{"key":"e_1_3_1_9_2","doi-asserted-by":"crossref","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 17th 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (HLT-NAACL). 4171\u20134186.","DOI":"10.18653\/v1\/N19-1423"},{"key":"e_1_3_1_10_2","unstructured":"Nick Erickson Jonas Mueller Alexander Shirkov Hang Zhang Pedro Larroy Mu Li and Alexander Smola. 2020. Autogluon-tabular: Robust and accurate automl for structured data. arXiv preprint arXiv:2003.06505 (2020)."},{"key":"e_1_3_1_11_2","unstructured":"Nick Erickson Jonas Mueller Alexander Shirkov Hang Zhang Pedro Larroy Mu Li and Alexander Smola. 2020. AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data. arXiv preprint arXiv:2003.06505 (2020)."},{"key":"e_1_3_1_12_2","doi-asserted-by":"crossref","unstructured":"Jiahao Fan Yi Li Shaohua Wang and Tien N Nguyen. 2020. AC\/C++ code vulnerability dataset with code changes and CVE summaries. In Proceedings of the 17th International Conference on Mining Software Repositories. 508\u2013512.","DOI":"10.1145\/3379597.3387501"},{"issue":"2022","key":"e_1_3_1_13_2","first-page":"1","article-title":"Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity","volume":"120","author":"Fedus William","year":"2022","unstructured":"William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research 23, 120 (2022), 1\u201339.","journal-title":"Journal of Machine Learning Research 23"},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","unstructured":"Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng Ming Gong Linjun Shou Bing Qin Ting Liu Daxin Jiang and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of EMNLP.","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_1_15_2","unstructured":"focal loss. 2024. Focal Loss Documentation. https:\/\/focal-loss.readthedocs.io\/en\/latest\/generated\/focal_loss.binary_focal_loss.html. Accessed: 2024-06-08."},{"key":"e_1_3_1_16_2","unstructured":"Michael Fu Van Nguyen Chakkrit Kla Tantithamthavorn Trung Le and Dinh Phung. 2023. Vulexplainer: A transformerbased hierarchical distillation for explaining vulnerability types. IEEE Transactions on Software Engineering (2023)."},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","unstructured":"Michael Fu and Chakkrit Tantithamthavorn. 2022. Linevul: A transformer-based line-level vulnerability prediction. In Proceedings of the 19th International Conference on Mining Software Repositories. 608\u2013620.","DOI":"10.1145\/3524842.3528452"},{"key":"e_1_3_1_18_2","doi-asserted-by":"crossref","unstructured":"Daya Guo Shuai Lu Nan Duan Yanlin Wang Ming Zhou and Jian Yin. 2022. Unixcoder: Unified cross-modal pre-training for code representation. arXiv preprint arXiv:2203.03850 (2022).","DOI":"10.18653\/v1\/2022.acl-long.499"},{"key":"e_1_3_1_19_2","unstructured":"Daya Guo Shuo Ren Shuai Lu Zhangyin Feng Duyu Tang Shujie Liu Long Zhou Nan Duan Alexey Svyatkovskiy Shengyu Fu et al. 2020. Graphcodebert: Pre-training code representations with data flow. arXiv preprint arXiv:2009.08366 (2020)."},{"issue":"2009","key":"e_1_3_1_20_2","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from imbalanced data","volume":"9","author":"He Haibo","year":"2009","unstructured":"Haibo He and Edwardo A Garcia. 2009. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering 21, 9 (2009), 1263\u20131284.","journal-title":"IEEE Transactions on knowledge and data engineering 21"},{"key":"e_1_3_1_21_2","doi-asserted-by":"crossref","unstructured":"David Hin Andrey Kan Huaming Chen and M Ali Babar. 2022. Linevd: Statement-level vulnerability detection using graph neural networks. In Proceedings of the 19th international conference on mining software repositories. 596\u2013607.","DOI":"10.1145\/3524842.3527949"},{"key":"e_1_3_1_22_2","doi-asserted-by":"crossref","unstructured":"RA Jacobs MI Jordan SJ Nowlan and GE Hinton. 1991. aAdaptive Mixtures of Local Experts \u00b0 Neural Computation vol. 3. (1991).","DOI":"10.1162\/neco.1991.3.1.79"},{"issue":"2017","key":"e_1_3_1_23_2","article-title":"Lightgbm: A highly efficient gradient boosting decision tree","volume":"30","author":"Ke Guolin","year":"2017","unstructured":"Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems 30 (2017).","journal-title":"Advances in neural information processing systems"},{"key":"e_1_3_1_24_2","unstructured":"Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang Maxim Krikun Noam Shazeer and Zhifeng Chen. 2020. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668 (2020)."},{"key":"e_1_3_1_25_2","doi-asserted-by":"crossref","unstructured":"Yi Li Shaohua Wang and Tien N Nguyen. 2021. Vulnerability detection with fine-grained interpretations. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 292\u2013303.","DOI":"10.1145\/3468264.3468597"},{"key":"e_1_3_1_26_2","unstructured":"Zhen Li Deqing Zou Shouhuai Xu Hai Jin Yawei Zhu and Zhaoxuan Chen. 2021. Sysevr: A framework for using deep learning to detect software vulnerabilities. IEEE Transactions on Dependable and Secure Computing (2021)."},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2018.23158"},{"key":"e_1_3_1_28_2","unstructured":"Tsung-Yi Lin Priya Goyal Ross Girshick Kaiming He and Piotr Doll\u00e1r. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980\u20132988."},{"key":"e_1_3_1_29_2","unstructured":"Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)."},{"key":"e_1_3_1_30_2","unstructured":"MITRE. 2024. CWE Top 25 Most Dangerous Software Weaknesses. https:\/\/cwe.mitre.org\/top25\/"},{"key":"e_1_3_1_31_2","doi-asserted-by":"crossref","unstructured":"Van-Anh Nguyen Dai Quoc Nguyen Van Nguyen Trung Le Quan Hung Tran and Dinh Phung. 2022. Regvd: Revisiting graph neural networks for vulnerability detection. In Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering: Companion Proceedings. 178\u2013182.","DOI":"10.1145\/3510454.3516865"},{"key":"e_1_3_1_32_2","unstructured":"Aman Omer Santosh Singh Rathore and Sandeep Kumar. 2023. ME-SFP: A Mixture-of-Experts-Based Approach for Software Fault Prediction. IEEE Transactions on Reliability (2023)."},{"issue":"2019","key":"e_1_3_1_33_2","article-title":"Pytorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019).","journal-title":"Advances in neural information processing systems"},{"key":"e_1_3_1_34_2","doi-asserted-by":"crossref","unstructured":"Jeffrey Pennington Richard Socher and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532\u20131543.","DOI":"10.3115\/v1\/D14-1162"},{"issue":"2018","key":"e_1_3_1_35_2","article-title":"CatBoost: unbiased boosting with categorical features","volume":"31","author":"Prokhorenkova Liudmila","year":"2018","unstructured":"Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. CatBoost: unbiased boosting with categorical features. Advances in neural information processing systems 31 (2018).","journal-title":"Advances in neural information processing systems"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","unstructured":"Md Mahbubur Rahman Ira Ceka Chengzhi Mao Saikat Chakraborty Baishakhi Ray and Wei Le. 2024. Towards causal deep learning for vulnerability detection. In Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering. 1\u201311.","DOI":"10.1145\/3597503.3639170"},{"issue":"2014","key":"e_1_3_1_37_2","doi-asserted-by":"crossref","first-page":"993","DOI":"10.1109\/TSE.2014.2340398","article-title":"Predicting vulnerable software components via text mining","volume":"10","author":"Scandariato Riccardo","year":"2014","unstructured":"Riccardo Scandariato, James Walden, Aram Hovsepyan, and Wouter Joosen. 2014. Predicting vulnerable software components via text mining. IEEE Transactions on Software Engineering 40, 10 (2014), 993\u20131006.","journal-title":"IEEE Transactions on Software Engineering 40"},{"issue":"2008","key":"e_1_3_1_38_2","first-page":"61","article-title":"The graph neural network model","volume":"1","author":"Scarselli Franco","year":"2008","unstructured":"Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE transactions on neural networks 20, 1 (2008), 61\u201380.","journal-title":"IEEE transactions on neural networks 20"},{"issue":"2023","key":"e_1_3_1_39_2","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1007\/s11219-023-09640-6","article-title":"Implicit and explicit mixture of experts models for software defect prediction","volume":"4","author":"Mishra Aditya Shankar","year":"2023","unstructured":"Aditya Shankar Mishra and Santosh Singh Rathore. 2023. Implicit and explicit mixture of experts models for software defect prediction. Software Quality Journal 31, 4 (2023), 1331\u20131368.","journal-title":"Software Quality Journal 31"},{"key":"e_1_3_1_40_2","unstructured":"Noam Shazeer Azalia Mirhoseini Krzysztof Maziarz Andy Davis Quoc Le Geoffrey Hinton and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538 (2017)."},{"issue":"2022","key":"e_1_3_1_41_2","article-title":"Daily streamflow forecasting in mountainous catchment using XGBoost, LightGBM and CatBoost","volume":"12","author":"Szczepanek Robert","year":"2022","unstructured":"Robert Szczepanek. 2022. Daily streamflow forecasting in mountainous catchment using XGBoost, LightGBM and CatBoost. Hydrology 9, 12 (2022), 226.","journal-title":"Hydrology 9"},{"key":"e_1_3_1_42_2","unstructured":"Shengye Wan Joshua Saxe Craig Gomes Sahana Chennabasappa Avilash Rath Kun Sun and Xinda Wang. 2024. Bridging the Gap: A Study of AI-based Vulnerability Management between Industry and Academia. arXiv preprint arXiv:2405.02435 (2024)."},{"key":"e_1_3_1_43_2","unstructured":"Xin-Cheng Wen Cuiyun Gao Feng Luo Haoyu Wang Ge Li and Qing Liao. 2024. LIVABLE: exploring long-tailed classification of software vulnerability types. IEEE Transactions on Software Engineering (2024)."},{"key":"e_1_3_1_44_2","doi-asserted-by":"crossref","unstructured":"Thomas Wolf Lysandre Debut Victor Sanh Julien Chaumond Clement Delangue Anthony Moi Pierric Cistac Tim Rault R\u00e9mi Louf Morgan Funtowicz et al. 2019. Huggingface\u2019s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019).","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"issue":"2020","key":"e_1_3_1_45_2","first-page":"4","article-title":"A comprehensive survey on graph neural networks","volume":"1","author":"Wu Zonghan","year":"2020","unstructured":"Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems 32, 1 (2020), 4\u201324.","journal-title":"IEEE transactions on neural networks and learning systems 32"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2014.44"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00192"},{"issue":"2012","key":"e_1_3_1_48_2","doi-asserted-by":"crossref","first-page":"1177","DOI":"10.1109\/TNNLS.2012.2200299","article-title":"Twenty years of mixture of experts","volume":"8","author":"Yuksel Seniha Esen","year":"2012","unstructured":"Seniha Esen Yuksel, Joseph N Wilson, and Paul D Gader. 2012. Twenty years of mixture of experts. IEEE transactions on neural networks and learning systems 23, 8 (2012), 1177\u20131193.","journal-title":"IEEE transactions on neural networks and learning systems 23"},{"key":"e_1_3_1_49_2","doi-asserted-by":"crossref","unstructured":"Yakun Zhang Wensheng Dou Jiaxin Zhu Liang Xu Zhiyong Zhou Jun Wei Dan Ye and Bo Yang. 2020. Learning to detect table clones in spreadsheets. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 528\u2013540.","DOI":"10.1145\/3395363.3397384"},{"key":"e_1_3_1_50_2","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/ASE56229.2023.00157","volume-title":"2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Zhou Xin","year":"2023","unstructured":"Xin Zhou, Kisub Kim, Bowen Xu, Jiakun Liu, DongGyun Han, and David Lo. 2023. The devil is in the tails: How long-tailed code distributions impact large language models. In 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE). IEEE, 40\u201352."},{"issue":"2019","key":"e_1_3_1_51_2","article-title":"Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks","volume":"32","author":"Zhou Yaqin","year":"2019","unstructured":"Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019. Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Advances in neural information processing systems 32 (2019).","journal-title":"Advances in neural information processing systems"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3715736","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T11:04:18Z","timestamp":1787483058000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715736"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,19]]},"references-count":50,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2025,6,19]]}},"alternative-id":["10.1145\/3715736"],"URL":"https:\/\/doi.org\/10.1145\/3715736","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,19]]}}}