{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T19:58:18Z","timestamp":1784836698654,"version":"3.55.0"},"reference-count":100,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T00:00:00Z","timestamp":1700784000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Luxembourg National Research Funds (FNR) AFR","award":["17036341"],"award-info":[{"award-number":["17036341"]}]},{"name":"European Research Council (ERC) under the European Union\u2019s Horizon 2020 research and innovation program","award":["949014"],"award-info":[{"award-number":["949014"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,1,31]]},"abstract":"<jats:p>\n            Graph Neural Networks (GNNs) have achieved promising performance in a variety of practical applications. Similar to traditional DNNs, GNNs could exhibit incorrect behavior that may lead to severe consequences, and thus testing is necessary and crucial. However, labeling all the test inputs for GNNs can be costly and time-consuming, especially when dealing with large and complex graphs, which seriously affects the efficiency of GNN testing. Existing studies have focused on test prioritization for DNNs, which aims to identify and prioritize fault-revealing tests (i.e., test inputs that are more likely to be misclassified) to detect system bugs earlier in a limited time. Although some DNN prioritization approaches have been demonstrated effective, there is a significant problem when applying them to GNNs: They do not take into account the connections (edges) between GNN test inputs (nodes), which play a significant role in GNN inference. In general, DNN test inputs are independent of each other, while GNN test inputs are usually represented as a graph with complex relationships between each test. In this article, we propose GraphPrior (\n            <jats:bold>GNN<\/jats:bold>\n            -oriented Test\n            <jats:bold>Prior<\/jats:bold>\n            itization), a set of approaches to prioritize test inputs specifically for GNNs via mutation analysis. Inspired by mutation testing in traditional software engineering, in which test suites are evaluated based on the mutants they kill, GraphPrior generates mutated models for GNNs and regards test inputs that kill many mutated models as more likely to be misclassified. Then, GraphPrior leverages the mutation results in two ways, killing-based and feature-based methods. When scoring a test input, the killing-based method considers each mutated model equally important, while feature-based methods learn different importance for each mutated model through ranking models. Finally, GraphPrior ranks all the test inputs based on their scores. We conducted an extensive study based on 604 subjects to evaluate GraphPrior on both natural and adversarial test inputs. The results demonstrate that KMGP, the killing-based GraphPrior approach, outperforms the compared approaches in a majority of cases, with an average improvement of 4.76% ~49.60% in terms of APFD. Furthermore, the feature-based GraphPrior approach, RFGP, performs the best among all the GraphPrior approaches. On adversarial test inputs, RFGP outperforms the compared approaches across different adversarial attacks, with the average improvement of 2.95% ~46.69%.\n          <\/jats:p>\n          <jats:p\/>","DOI":"10.1145\/3607191","type":"journal-article","created":{"date-parts":[[2023,7,4]],"date-time":"2023-07-04T13:26:34Z","timestamp":1688477194000},"page":"1-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":23,"title":["GraphPrior: Mutation-based Test Input Prioritization for Graph Neural Networks"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4097-9543","authenticated-orcid":false,"given":"Xueqi","family":"Dang","sequence":"first","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1390-0393","authenticated-orcid":false,"given":"Yinghua","family":"Li","sequence":"additional","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1852-2547","authenticated-orcid":false,"given":"Mike","family":"Papadakis","sequence":"additional","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4052-475X","authenticated-orcid":false,"given":"Jacques","family":"Klein","sequence":"additional","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7270-9869","authenticated-orcid":false,"given":"Tegawend\u00e9 F.","family":"Bissyand\u00e9","sequence":"additional","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1045-4861","authenticated-orcid":false,"given":"Yves","family":"Le Traon","sequence":"additional","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,11,24]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1522"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.5555\/1355340"},{"key":"e_1_3_2_4_2","first-page":"695","volume-title":"International Conference on Machine Learning","author":"Bojchevski Aleksandar","year":"2019","unstructured":"Aleksandar Bojchevski and Stephan G\u00fcnnemann. 2019. Adversarial attacks on node embeddings via graph poisoning. In International Conference on Machine Learning. PMLR, 695\u2013704."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2021.04.039"},{"issue":"1","key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman Leo","year":"2001","unstructured":"Leo Breiman. 2001. Random forests. Mach. Learn. 45, 1 (2001), 5\u201332.","journal-title":"Mach. Learn."},{"key":"e_1_3_2_7_2","first-page":"63","volume-title":"IEEE International Conference on Artificial Intelligence Testing (AITest\u201919)","author":"Byun Taejoon","year":"2019","unstructured":"Taejoon Byun, Vaibhav Sharma, Abhishek Vijayakumar, Sanjai Rayadurgam, and Darren Cofer. 2019. Input prioritization for testing neural networks. In IEEE International Conference on Artificial Intelligence Testing (AITest\u201919). IEEE, 63\u201370."},{"issue":"9","key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"1616","DOI":"10.1109\/TKDE.2018.2807452","article-title":"A comprehensive survey of graph embedding: Problems, techniques, and applications","volume":"30","author":"Cai Hongyun","year":"2018","unstructured":"Hongyun Cai, Vincent W. Zheng, and Kevin Chen-Chuan Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Trans. Knowl. Data Eng. 30, 9 (2018), 1616\u20131637.","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"e_1_3_2_9_2","first-page":"39","volume-title":"IEEE Symposium on Security and Privacy (SP\u201917)","author":"Carlini Nicholas","year":"2017","unstructured":"Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy (SP\u201917). IEEE, 39\u201357."},{"key":"e_1_3_2_10_2","first-page":"597","volume-title":"39th International Conference on Software Engineering","author":"Chekam Thierry Titcheu","year":"2017","unstructured":"Thierry Titcheu Chekam, Mike Papadakis, Yves Le Traon, and Mark Harman. 2017. An empirical study on mutation, statement and branch coverage fault revelation that avoids the unreliable clean program assumption. In 39th International Conference on Software Engineering, Sebasti\u00e1n Uchitel, Alessandro Orso, and Martin P. Robillard (Eds.). IEEE\/ACM, 597\u2013608. DOI:10.1109\/ICSE.2017.61"},{"key":"e_1_3_2_11_2","first-page":"485","volume-title":"AAAI Conference on Artificial Intelligence","volume":"33","author":"Chen Cen","year":"2019","unstructured":"Cen Chen, Kenli Li, Sin G. Teo, Xiaofeng Zou, Kang Wang, Jie Wang, and Zeng Zeng. 2019. Gated residual recurrent graph neural networks for traffic prediction. In AAAI Conference on Artificial Intelligence, Vol. 33. 485\u2013492."},{"key":"e_1_3_2_12_2","first-page":"472","volume-title":"40th International Conference on Software Engineering","author":"Chen Junjie","year":"2018","unstructured":"Junjie Chen. 2018. Learning to accelerate compiler testing. In 40th International Conference on Software Engineering. 472\u2013475."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.70"},{"issue":"2","key":"e_1_3_2_14_2","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1109\/TSE.2018.2889771","article-title":"Coverage prediction for accelerating compiler testing","volume":"47","author":"Chen Junjie","year":"2018","unstructured":"Junjie Chen, Guancheng Wang, Dan Hao, Yingfei Xiong, Hongyu Zhang, Lu Zhang, and Bing Xie. 2018. Coverage prediction for accelerating compiler testing. IEEE Trans. Softw. Eng. 47, 2 (2018), 261\u2013278.","journal-title":"IEEE Trans. Softw. Eng."},{"issue":"4","key":"e_1_3_2_15_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3394112","article-title":"Practical accuracy estimation for efficient deep neural network testing","volume":"29","author":"Chen Junjie","year":"2020","unstructured":"Junjie Chen, Zhuo Wu, Zan Wang, Hanmo You, Lingming Zhang, and Ming Yan. 2020. Practical accuracy estimation for efficient deep neural network testing. ACM Trans. Softw. Eng. Method. 29, 4 (2020), 1\u201335.","journal-title":"ACM Trans. Softw. Eng. Method."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_3_2_17_2","first-page":"1115","volume-title":"International Conference on Machine Learning","author":"Dai Hanjun","year":"2018","unstructured":"Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. 2018. Adversarial attack on graph structured data. In International Conference on Machine Learning. PMLR, 1115\u20131124."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/C-M.1978.218136"},{"key":"e_1_3_2_19_2","first-page":"655","volume-title":"38th International Conference on Software Engineering","author":"Devroey Xavier","year":"2016","unstructured":"Xavier Devroey, Gilles Perrouin, Mike Papadakis, Axel Legay, Pierre-Yves Schobbens, and Patrick Heymans. 2016. Featured model-based mutation analysis. In 38th International Conference on Software Engineering, Laura K. Dillon, Willem Visser, and Laurie A. Williams (Eds.). ACM, 655\u2013666. DOI:10.1145\/2884781.2884821"},{"key":"e_1_3_2_20_2","first-page":"302","volume-title":"IEEE 6th International Conference on Software Testing, Verification and Validation","author":"Nardo Daniel Di","year":"2013","unstructured":"Daniel Di Nardo, Nadia Alshahwan, Lionel Briand, and Yvan Labiche. 2013. Coverage-based test case prioritisation: An industrial case study. In IEEE 6th International Conference on Software Testing, Verification and Validation. IEEE, 302\u2013311."},{"issue":"9","key":"e_1_3_2_21_2","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1109\/TSE.2006.92","article-title":"On the use of mutation faults in empirical assessments of test case prioritization techniques","volume":"32","author":"Do Hyunsook","year":"2006","unstructured":"Hyunsook Do and Gregg Rothermel. 2006. On the use of mutation faults in empirical assessments of test case prioritization techniques. IEEE Trans. Softw. Eng. 32, 9 (2006), 733\u2013752.","journal-title":"IEEE Trans. Softw. Eng."},{"key":"e_1_3_2_22_2","article-title":"Topology adaptive graph convolutional networks","author":"Du Jian","year":"2017","unstructured":"Jian Du, Shanghang Zhang, Guanhang Wu, Jos\u00e9 M. F. Moura, and Soummya Kar. 2017. Topology adaptive graph convolutional networks. arXiv preprint arXiv:1710.10370 (2017).","journal-title":"arXiv preprint arXiv:1710.10370"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.988497"},{"key":"e_1_3_2_24_2","first-page":"235","volume-title":"22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering","author":"Elbaum Sebastian","year":"2014","unstructured":"Sebastian Elbaum, Gregg Rothermel, and John Penix. 2014. Techniques for improving regression testing in continuous integration development environments. In 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 235\u2013245."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2009.07.001"},{"key":"e_1_3_2_26_2","first-page":"417","volume-title":"World Wide Web Conference","author":"Fan Wenqi","year":"2019","unstructured":"Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. In World Wide Web Conference. 417\u2013426."},{"key":"e_1_3_2_27_2","first-page":"177","volume-title":"29th ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Feng Yang","year":"2020","unstructured":"Yang Feng, Qingkai Shi, Xinyu Gao, Jun Wan, Chunrong Fang, and Zhenyu Chen. 2020. DeepGini: Prioritizing massive tests to enhance the robustness of deep neural networks. In 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 177\u2013188."},{"issue":"6","key":"e_1_3_2_28_2","article-title":"Utilizing graph machine learning within drug discovery and development","volume":"22","author":"Gaudelet Thomas","year":"2021","unstructured":"Thomas Gaudelet, Ben Day, Arian R. Jamasb, Jyothish Soman, Cristian Regep, Gertrude Liu, Jeremy B. R. Hayter, Richard Vickers, Charles Roberts, Jian Tang, David Roblin, Tom L. Blundell, Michael M. Bronstein, and Jake P. Taylor-King. 2021. Utilizing graph machine learning within drug discovery and development. Brief. Bioinform. 22, 6 (2021).","journal-title":"Brief. Bioinform."},{"key":"e_1_3_2_29_2","first-page":"7637","article-title":"Robustness of graph neural networks at scale","volume":"34","author":"Geisler Simon","year":"2021","unstructured":"Simon Geisler, Tobias Schmidt, Hakan \u015eirin, Daniel Z\u00fcgner, Aleksandar Bojchevski, and Stephan G\u00fcnnemann. 2021. Robustness of graph neural networks at scale. Adv. Neural Inf. Process. Syst. 34 (2021), 7637\u20137649.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_30_2","first-page":"1263","volume-title":"International Conference on Machine Learning","author":"Gilmer Justin","year":"2017","unstructured":"Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural message passing for quantum chemistry. In International Conference on Machine Learning. PMLR, 1263\u20131272."},{"key":"e_1_3_2_31_2","article-title":"Inductive representation learning on large graphs","volume":"30","author":"Hamilton Will","year":"2017","unstructured":"Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. Adv. Neural Inf. Process. Syst. 30 (2017).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401063"},{"issue":"1","key":"e_1_3_2_33_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2430536.2430540","article-title":"Achieving scalable model-based testing through test case diversity","volume":"22","author":"Hemmati Hadi","year":"2013","unstructured":"Hadi Hemmati, Andrea Arcuri, and Lionel Briand. 2013. Achieving scalable model-based testing through test case diversity. ACM Trans. Softw. Eng. Methodol. 22, 1 (2013), 1\u201342.","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"e_1_3_2_34_2","first-page":"523","volume-title":"IEEE\/ACM 38th International Conference on Software Engineering (ICSE\u201916)","author":"Henard Christopher","year":"2016","unstructured":"Christopher Henard, Mike Papadakis, Mark Harman, Yue Jia, and Yves Le Traon. 2016. Comparing white-box and black-box test prioritization. In IEEE\/ACM 38th International Conference on Software Engineering (ICSE\u201916). IEEE, 523\u2013534."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2327020"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TGRS.2020.3015157"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678672"},{"key":"e_1_3_2_38_2","first-page":"1158","volume-title":"34th IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201919)","author":"Hu Qiang","year":"2019","unstructured":"Qiang Hu, Lei Ma, Xiaofei Xie, Bing Yu, Yang Liu, and Jianjun Zhao. 2019. DeepMutation++: A mutation testing framework for deep learning systems. In 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201919). IEEE, 1158\u20131161."},{"key":"e_1_3_2_39_2","first-page":"22118","article-title":"Open graph benchmark: Datasets for machine learning on graphs","volume":"33","author":"Hu Weihua","year":"2020","unstructured":"Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. Adv. Neural Inf. Process. Syst. 33 (2020), 22118\u201322133.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_40_2","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1145\/3460319.3464825","volume-title":"30th ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Humbatova Nargiz","year":"2021","unstructured":"Nargiz Humbatova, Gunel Jahangirova, and Paolo Tonella. 2021. DeepCrime: Mutation testing of deep learning systems based on real faults. In 30th ACM SIGSOFT International Symposium on Software Testing and Analysis. 67\u201378."},{"issue":"1","key":"e_1_3_2_41_2","first-page":"1","article-title":"Prediction of protein\u2013protein interaction using graph neural networks","volume":"12","author":"Jha Kanchan","year":"2022","unstructured":"Kanchan Jha, Sriparna Saha, and Hiteshi Singh. 2022. Prediction of protein\u2013protein interaction using graph neural networks. Scient. Rep. 12, 1 (2022), 1\u201312.","journal-title":"Scient. Rep."},{"key":"e_1_3_2_42_2","doi-asserted-by":"crossref","first-page":"117921","DOI":"10.1016\/j.eswa.2022.117921","article-title":"Graph neural network for traffic forecasting: A survey","author":"Jiang Weiwei","year":"2022","unstructured":"Weiwei Jiang and Jiayun Luo. 2022. Graph neural network for traffic forecasting: A survey. Expert Syst. Applic. 207 (2022), 117921.","journal-title":"Expert Syst. Applic."},{"key":"e_1_3_2_43_2","article-title":"LightGBM: A highly efficient gradient boosting decision tree","volume":"30","author":"Ke Guolin","year":"2017","unstructured":"Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: A highly efficient gradient boosting decision tree. Adv. Neural Inf. Process. Syst. 30 (2017).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_44_2","article-title":"Examples are not enough, learn to criticize! Criticism for interpretability","volume":"29","author":"Kim Been","year":"2016","unstructured":"Been Kim, Rajiv Khanna, and Oluwasanmi O. Koyejo. 2016. Examples are not enough, learn to criticize! Criticism for interpretability. Adv. Neural Inf. Process. Syst. 29 (2016).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00108"},{"key":"e_1_3_2_46_2","article-title":"Semi-supervised classification with graph convolutional networks","author":"Kipf Thomas N.","year":"2016","unstructured":"Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).","journal-title":"arXiv preprint arXiv:1609.02907"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-011-0093-0"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052643"},{"key":"e_1_3_2_49_2","article-title":"DeepRobust: A PyTorch library for adversarial attacks and defenses","author":"Li Yaxin","year":"2020","unstructured":"Yaxin Li, Wei Jin, Han Xu, and Jiliang Tang. 2020. DeepRobust: A PyTorch library for adversarial attacks and defenses. arXiv preprint arXiv:2005.06149 (2020).","journal-title":"arXiv preprint arXiv:2005.06149"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.38"},{"key":"e_1_3_2_51_2","first-page":"499","volume-title":"27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"Li Zenan","year":"2019","unstructured":"Zenan Li, Xiaoxing Ma, Chang Xu, Chun Cao, Jingwei Xu, and Jian L\u00fc. 2019. Boosting operational DNN testing efficiency through conditioning. In 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 499\u2013509."},{"key":"e_1_3_2_52_2","first-page":"1","volume-title":"Advances in Computers","author":"Lou Yiling","year":"2019","unstructured":"Yiling Lou, Junjie Chen, Lingming Zhang, and Dan Hao. 2019. A survey on regression test-case prioritization. In Advances in Computers. Vol. 113. Elsevier, 1\u201346."},{"key":"e_1_3_2_53_2","first-page":"46","volume-title":"IEEE 26th International Symposium on Software Reliability Engineering (ISSRE\u201915)","author":"Lou Yiling","year":"2015","unstructured":"Yiling Lou, Dan Hao, and Lu Zhang. 2015. Mutation-based test-case prioritization in software evolution. In IEEE 26th International Symposium on Software Reliability Engineering (ISSRE\u201915). IEEE, 46\u201357."},{"key":"e_1_3_2_54_2","first-page":"4756","article-title":"Towards more practical adversarial attacks on graph neural networks","volume":"33","author":"Ma Jiaqi","year":"2020","unstructured":"Jiaqi Ma, Shuangrui Ding, and Qiaozhu Mei. 2020. Towards more practical adversarial attacks on graph neural networks. Adv. Neural Inf. Process. Syst. 33 (2020), 4756\u20134766.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_55_2","first-page":"614","volume-title":"IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER\u201919)","author":"Ma Lei","year":"2019","unstructured":"Lei Ma, Felix Juefei-Xu, Minhui Xue, Bo Li, Li Li, Yang Liu, and Jianjun Zhao. 2019. DeepCT: Tomographic combinatorial testing for deep learning systems. In IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER\u201919). IEEE, 614\u2013618."},{"key":"e_1_3_2_56_2","first-page":"120","volume-title":"33rd ACM\/IEEE International Conference on Automated Software Engineering","author":"Ma Lei","year":"2018","unstructured":"Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang. 2018. DeepGauge: Multi-granularity testing criteria for deep learning systems. In 33rd ACM\/IEEE International Conference on Automated Software Engineering. 120\u2013131."},{"key":"e_1_3_2_57_2","first-page":"100","volume-title":"IEEE 29th International Symposium on Software Reliability Engineering (ISSRE\u201918)","author":"Ma Lei","year":"2018","unstructured":"Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, et\u00a0al. 2018. DeepMutation: Mutation testing of deep learning systems. In IEEE 29th International Symposium on Software Reliability Engineering (ISSRE\u201918). IEEE, 100\u2013111."},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3417330"},{"key":"e_1_3_2_59_2","article-title":"Attacking graph convolutional networks via rewiring","author":"Ma Yao","year":"2019","unstructured":"Yao Ma, Suhang Wang, Tyler Derr, Lingfei Wu, and Jiliang Tang. 2019. Attacking graph convolutional networks via rewiring. arXiv preprint arXiv:1906.03750 (2019).","journal-title":"arXiv preprint arXiv:1906.03750"},{"key":"e_1_3_2_60_2","article-title":"Towards deep learning models resistant to adversarial attacks","author":"Madry Aleksander","year":"2017","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017).","journal-title":"arXiv preprint arXiv:1706.06083"},{"issue":"7540","key":"e_1_3_2_61_2","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih Volodymyr","year":"2015","unstructured":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529\u2013533.","journal-title":"Nature"},{"key":"e_1_3_2_62_2","first-page":"1","article-title":"Influence of data splitting on performance of machine learning models in prediction of shear strength of soil","volume":"2021","author":"Nguyen Quang Hung","year":"2021","unstructured":"Quang Hung Nguyen, Hai-Bang Ly, Lanh Si Ho, Nadhir Al-Ansari, Hiep Van Le, Van Quan Tran, Indra Prakash, and Binh Thai Pham. 2021. Influence of data splitting on performance of machine learning models in prediction of shear strength of soil. Math. Prob. Eng. 2021 (2021), 1\u201315.","journal-title":"Math. Prob. Eng."},{"key":"e_1_3_2_63_2","first-page":"127","volume-title":"European Conference on Artificial Neural Networks","author":"Pancino Niccol\u00f2","year":"2020","unstructured":"Niccol\u00f2 Pancino, Alberto Rossi, Giorgio Ciano, Giorgia Giacomini, Simone Bonechi, Paolo Andreini, Franco Scarselli, Monica Bianchini, and Pietro Bongini. 2020. Graph neural networks for the prediction of protein-protein interfaces. In European Conference on Artificial Neural Networks. 127\u2013132."},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2014.11"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1016\/bs.adcom.2018.03.015"},{"key":"e_1_3_2_66_2","article-title":"PyTorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32 (2019).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_67_2","first-page":"1","volume-title":"26th Symposium on Operating Systems Principles","author":"Pei Kexin","year":"2017","unstructured":"Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. 2017. DeepXplore: Automated whitebox testing of deep learning systems. In 26th Symposium on Operating Systems Principles. 1\u201318."},{"issue":"3","key":"e_1_3_2_68_2","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1002\/j.2168-9830.2004.tb00809.x","article-title":"Does active learning work? A review of the research","volume":"93","author":"Prince Michael","year":"2004","unstructured":"Michael Prince. 2004. Does active learning work? A review of the research. J. Eng. Educ. 93, 3 (2004), 223\u2013231.","journal-title":"J. Eng. Educ."},{"issue":"9","key":"e_1_3_2_69_2","first-page":"1","article-title":"A survey of deep active learning","volume":"54","author":"Ren Pengzhen","year":"2021","unstructured":"Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B. Gupta, Xiaojiang Chen, and Xin Wang. 2021. A survey of deep active learning. ACM Comput. Surv. 54, 9 (2021), 1\u201340.","journal-title":"ACM Comput. Surv."},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.962562"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411866"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2008.2005605"},{"key":"e_1_3_2_73_2","first-page":"108","volume-title":"IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C\u201918)","author":"Shen Weijun","year":"2018","unstructured":"Weijun Shen, Jun Wan, and Zhenyu Chen. 2018. MuNN: Mutation analysis of neural networks. In IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C\u201918). IEEE, 108\u2013115."},{"key":"e_1_3_2_74_2","article-title":"GraphAF: A flow-based autoregressive model for molecular graph generation","author":"Shi Chence","year":"2020","unstructured":"Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. 2020. GraphAF: A flow-based autoregressive model for molecular graph generation. arXiv preprint arXiv:2001.09382 (2020).","journal-title":"arXiv preprint arXiv:2001.09382"},{"issue":"1","key":"e_1_3_2_75_2","doi-asserted-by":"crossref","first-page":"e1695","DOI":"10.1002\/stvr.1695","article-title":"Empirical evaluation of mutation-based test case prioritization techniques","volume":"29","author":"Shin Donghwan","year":"2019","unstructured":"Donghwan Shin, Shin Yoo, Mike Papadakis, and Doo-Hwan Bae. 2019. Empirical evaluation of mutation-based test case prioritization techniques. Softw. Test., Verif. Reliab. 29, 1-2 (2019), e1695.","journal-title":"Softw. Test., Verif. Reliab."},{"key":"e_1_3_2_76_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).","journal-title":"arXiv preprint arXiv:1409.1556"},{"key":"e_1_3_2_77_2","first-page":"273","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Sun Chen","year":"2019","unstructured":"Chen Sun, Abhinav Shrivastava, Carl Vondrick, Rahul Sukthankar, Kevin Murphy, and Cordelia Schmid. 2019. Relational action forecasting. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 273\u2013283."},{"key":"e_1_3_2_78_2","article-title":"Adversarial attack and defense on graph data: A survey","author":"Sun Lichao","year":"2018","unstructured":"Lichao Sun, Yingtong Dou, Carl Yang, Ji Wang, Philip S. Yu, Lifang He, and Bo Li. 2018. Adversarial attack and defense on graph data: A survey. arXiv preprint arXiv:1812.10528 (2018).","journal-title":"arXiv preprint arXiv:1812.10528"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2017.2761740"},{"key":"e_1_3_2_80_2","article-title":"Graph attention networks","author":"Veli\u010dkovi\u0107 Petar","year":"2017","unstructured":"Petar Veli\u010dkovi\u0107, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).","journal-title":"arXiv preprint arXiv:1710.10903"},{"key":"e_1_3_2_81_2","first-page":"112","volume-title":"International Joint Conference on Neural Networks (IJCNN\u201914)","author":"Wang Dan","year":"2014","unstructured":"Dan Wang and Yi Shang. 2014. A new active labeling method for deep learning. In International Joint Conference on Neural Networks (IJCNN\u201914). IEEE, 112\u2013119."},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00046"},{"key":"e_1_3_2_83_2","first-page":"139","volume-title":"31st ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Weiss Michael","year":"2022","unstructured":"Michael Weiss and Paolo Tonella. 2022. Simple techniques work surprisingly well for neural network test prioritization and active learning (replicability study). In 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. 139\u2013150."},{"key":"e_1_3_2_84_2","unstructured":"Raymond E. Wright. 1995. Logistic regression. In Reading and Understanding Multivariate Statistics L. G. Grimm and P. R. Yarnold (Eds.). American Psychological Association 217\u2013244."},{"key":"e_1_3_2_85_2","article-title":"SocialGCN: An efficient graph convolutional network based model for social recommendation","author":"Wu Le","year":"2018","unstructured":"Le Wu, Peijie Sun, Richang Hong, Yanjie Fu, Xiting Wang, and Meng Wang. 2018. SocialGCN: An efficient graph convolutional network based model for social recommendation. arXiv preprint arXiv:1811.02815 (2018).","journal-title":"arXiv preprint arXiv:1811.02815"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/3535101"},{"key":"e_1_3_2_87_2","article-title":"Topology attack and defense for graph neural networks: An optimization perspective","author":"Xu Kaidi","year":"2019","unstructured":"Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Mingyi Hong, and Xue Lin. 2019. Topology attack and defense for graph neural networks: An optimization perspective. arXiv preprint arXiv:1906.04214 (2019).","journal-title":"arXiv preprint arXiv:1906.04214"},{"key":"e_1_3_2_88_2","article-title":"How powerful are graph neural networks?","author":"Xu Keyulu","year":"2018","unstructured":"Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).","journal-title":"arXiv preprint arXiv:1810.00826"},{"key":"e_1_3_2_89_2","first-page":"40","volume-title":"International Conference on Machine Learning","author":"Yang Zhilin","year":"2016","unstructured":"Zhilin Yang, William Cohen, and Ruslan Salakhudinov. 2016. Revisiting semi-supervised learning with graph embeddings. In International Conference on Machine Learning. PMLR, 40\u201348."},{"key":"e_1_3_2_90_2","first-page":"7370","volume-title":"AAAI Conference on Artificial Intelligence","volume":"33","author":"Yao Liang","year":"2019","unstructured":"Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. Graph convolutional networks for text classification. In AAAI Conference on Artificial Intelligence, Vol. 33. 7370\u20137377."},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2019.105020"},{"key":"e_1_3_2_92_2","doi-asserted-by":"crossref","first-page":"974","DOI":"10.1145\/3219819.3219890","volume-title":"24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Ying Rex","year":"2018","unstructured":"Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 974\u2013983."},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1002\/stv.430"},{"key":"e_1_3_2_94_2","article-title":"Enhance social recommendation with adversarial graph convolutional networks","author":"Yu Junliang","year":"2020","unstructured":"Junliang Yu, Hongzhi Yin, Jundong Li, Min Gao, Zi Huang, and Lizhen Cui. 2020. Enhance social recommendation with adversarial graph convolutional networks. IEEE Trans. Knowl. Data Eng. 34, 8 (2020).","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"e_1_3_2_95_2","article-title":"A noise-sensitivity-analysis-based test prioritization technique for deep neural networks","author":"Zhang Long","year":"2019","unstructured":"Long Zhang, Xuechao Sun, Yong Li, and Zhenyu Zhang. 2019. A noise-sensitivity-analysis-based test prioritization technique for deep neural networks. arXiv preprint arXiv:1901.00054 (2019).","journal-title":"arXiv preprint arXiv:1901.00054"},{"issue":"5","key":"e_1_3_2_96_2","doi-asserted-by":"crossref","first-page":"3015","DOI":"10.1109\/TNSE.2021.3126830","article-title":"Graph neural network-driven traffic forecasting for the connected internet of vehicles","volume":"9","author":"Zhang Qin","year":"2021","unstructured":"Qin Zhang, Keping Yu, Zhiwei Guo, Sahil Garg, Joel J. P. C. Rodrigues, Mohammad Mehedi Hassan, and Mohsen Guizani. 2021. Graph neural network-driven traffic forecasting for the connected internet of vehicles. IEEE Trans. Netw. Sci. Eng. 9, 5 (2021), 3015\u20133027.","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"e_1_3_2_97_2","first-page":"1","volume-title":"IEEE Symposium Series on Computational Intelligence (SSCI\u201916)","author":"Zhao Dongbin","year":"2016","unstructured":"Dongbin Zhao, Haitao Wang, Kun Shao, and Yuanheng Zhu. 2016. Deep reinforcement learning with experience replay based on SARSA. In IEEE Symposium Series on Computational Intelligence (SSCI\u201916). IEEE, 1\u20136."},{"issue":"18","key":"e_1_3_2_98_2","doi-asserted-by":"crossref","first-page":"6135","DOI":"10.3390\/molecules27186135","article-title":"Graph neural network for protein\u2013protein interaction prediction: A comparative study","volume":"27","author":"Zhou Hang","year":"2022","unstructured":"Hang Zhou, Weikun Wang, Jiayun Jin, Zengwei Zheng, and Binbin Zhou. 2022. Graph neural network for protein\u2013protein interaction prediction: A comparative study. Molecules 27, 18 (2022), 6135.","journal-title":"Molecules"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2021.01.001"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220078"},{"key":"e_1_3_2_101_2","article-title":"Adversarial attacks on graph neural networks via meta learning","author":"Z\u00fcgner Daniel","year":"2019","unstructured":"Daniel Z\u00fcgner and Stephan G\u00fcnnemann. 2019. Adversarial attacks on graph neural networks via meta learning. arXiv preprint arXiv:1902.08412 (2019).","journal-title":"arXiv preprint arXiv:1902.08412"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607191","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3607191","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:34Z","timestamp":1750178254000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607191"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,24]]},"references-count":100,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1,31]]}},"alternative-id":["10.1145\/3607191"],"URL":"https:\/\/doi.org\/10.1145\/3607191","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,24]]},"assertion":[{"value":"2023-01-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-13","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}