{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T21:18:20Z","timestamp":1772831900886,"version":"3.50.1"},"reference-count":74,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,1,20]],"date-time":"2025-01-20T00:00:00Z","timestamp":1737331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2021YFB3601400"],"award-info":[{"award-number":["2021YFB3601400"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62232016 and 62072442"],"award-info":[{"award-number":["62232016 and 62072442"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004739","name":"Youth Innovation Promotion Association Chinese Academy of Sciences","doi-asserted-by":"crossref","award":["ISCAS-JCZD-202304"],"award-info":[{"award-number":["ISCAS-JCZD-202304"]}],"id":[{"id":"10.13039\/501100004739","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Major Program of ISCAS","award":["ISCAS-ZD-202302"],"award-info":[{"award-number":["ISCAS-ZD-202302"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>\n            The competitive game between agents exists in many critical applications, such as military unmanned aerial vehicles. It is urgent to test these agents to reduce the significant losses caused by their failures. Existing studies mainly are to construct a testing agent that competes with the target agent to induce its failures. These approaches usually focus on a single task, requiring much more time for multi-task testing. However, if the previously tested tasks (source tasks) and the task to be tested (target task) share similar agents or task objectives, the transferable knowledge in source tasks can potentially increase the effectiveness of testing in the target task. We propose Demo2Test for conducting transfer testing of agents in the competitive environment, i.e., leveraging the demonstrations of failure scenarios from the source task to boost the testing effectiveness in the target task. It trains a testing agent with demonstrations and incorporates the action perturbation at key states to balance the number of revealed failures and their diversity. We conduct experiments in the simulated robotics competitive environments of MuJoCo. The results indicate that Demo2Test outperforms the best-performing baseline with improvements ranging from\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(22.38\\%\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            to\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(87.98\\%\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            , and\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(12.69\\%\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            to\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(60.98\\%\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            , in terms of the number and diversity of discovered failure scenarios, respectively.\n          <\/jats:p>","DOI":"10.1145\/3696001","type":"journal-article","created":{"date-parts":[[2024,9,13]],"date-time":"2024-09-13T13:56:18Z","timestamp":1726235778000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Demo2Test: Transfer Testing of Agent in Competitive Environment with Failure Demonstrations"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9191-0696","authenticated-orcid":false,"given":"Jianming","family":"Chen","sequence":"first","affiliation":[{"name":"Institute of Software, Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2854-4889","authenticated-orcid":false,"given":"Yawen","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Software, Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9941-6713","authenticated-orcid":false,"given":"Junjie","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Software, Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1288-6502","authenticated-orcid":false,"given":"Xiaofei","family":"Xie","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8380-050X","authenticated-orcid":false,"given":"Dandan","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Software, Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2618-5694","authenticated-orcid":false,"given":"Qing","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Software, Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-6016-7360","authenticated-orcid":false,"given":"Fanjiang","family":"Xu","sequence":"additional","affiliation":[{"name":"Institute of Software, Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,1,20]]},"reference":[{"issue":"6","key":"e_1_3_2_2_2","first-page":"4999","article-title":"Transfer Deep Learning Approach for Detecting Coronavirus Disease in X-Ray Images","volume":"11","author":"Al-Smadi Mohammed","year":"2021","unstructured":"Mohammed Al-Smadi, Mahmoud Hammad, Qanita Bani Baker, Saja Khaled Tawalbeh, and A. Sa\u2019ad. 2021. Transfer Deep Learning Approach for Detecting Coronavirus Disease in X-Ray Images. International Journal of Electrical and Computer Engineering 11, 6 (2021), 4999.","journal-title":"International Journal of Electrical and Computer Engineering"},{"issue":"2","key":"e_1_3_2_3_2","first-page":"61 pages.","article-title":"Testing, Validation, and Verification of Robotic and Autonomous Systems: A Systematic Review","volume":"32","author":"Araujo Hugo","year":"2023","unstructured":"Hugo Araujo, Mohammad Reza Mousavi, and Mahsa Varshosaz. 2023. Testing, Validation, and Verification of Robotic and Autonomous Systems: A Systematic Review. ACM Transactions on Software Engineering and Methodology 32, 2, Article 51 (Mar. 2023), 61 pages.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_4_2","unstructured":"Trapit Bansal Jakub Pachocki Szymon Sidor Ilya Sutskever and Igor Mordatch. 2018. Emergent Complexity via Multi-Agent Competition. arXiv:1710.03748. Retrieved from http:\/\/arxiv.org\/abs\/1710.03748"},{"key":"e_1_3_2_5_2","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1007\/978-3-319-62416-7_19","volume-title":"Proceedings of the Machine Learning and Data Mining in Pattern Recognition - 13th International Conference (MLDM \u201917)","volume":"10358","author":"Behzadan Vahid","year":"2017","unstructured":"Vahid Behzadan and Arslan Munir. 2017. Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks. In Proceedings of the Machine Learning and Data Mining in Pattern Recognition - 13th International Conference (MLDM \u201917), Lecture Notes in Computer Science, Vol. 10358, Springer, 262\u2013275."},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1109\/ICST57152.2023.00045","volume-title":"Proceedings of the 2023 IEEE Conference on Software Testing, Verification and Validation (ICST \u201923)","author":"Berglund Lukas","year":"2023","unstructured":"Lukas Berglund, Tim Grube, Gregory Gay, Francisco Gomes de Oliveira Neto, and Dimitrios Platis. 2023. Test Maintenance for Machine Learning Systems: A Case Study in the Automotive Industry. In Proceedings of the 2023 IEEE Conference on Software Testing, Verification and Validation (ICST \u201923), 410\u2013421."},{"issue":"4","key":"e_1_3_2_7_2","first-page":"46 pages","article-title":"Testing the Plasticity of Reinforcement Learning-Based Systems","volume":"31","author":"Biagiola Matteo","year":"2022","unstructured":"Matteo Biagiola and Paolo Tonella. 2022. Testing the Plasticity of Reinforcement Learning-Based Systems. ACM Transactions on Software Engineering and Methodology 31, 4, Article 80 (Jul. 2022), 46 pages.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"issue":"3","key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3631970","article-title":"Testing of Deep Reinforcement Learning Agents with Surrogate Models","volume":"33","author":"Biagiola Matteo","year":"2023","unstructured":"Matteo Biagiola and Paolo Tonella. 2023. Testing of Deep Reinforcement Learning Agents with Surrogate Models. ACM Transactions on Software Engineering and Methodology 33, 3 (Nov. 2023), 1\u201333.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"issue":"2","key":"e_1_3_2_9_2","first-page":"30 pages","article-title":"Single and Multi-Objective Test Cases Prioritization for Self-Driving Cars in Virtual Environments","volume":"32","author":"Birchler Christian","year":"2023","unstructured":"Christian Birchler, Sajad Khatiri, Pouria Derakhshanfar, Sebastiano Panichella, and Annibale Panichella. 2023. Single and Multi-Objective Test Cases Prioritization for Self-Driving Cars in Virtual Environments. ACM Transactions on Software Engineering and Methodology 32, 2, Article 28 (Apr. 2023), 30 pages.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_10_2","first-page":"131","volume-title":"Proceedings of the 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW \u201923)","author":"Bisht Rohini","year":"2023","unstructured":"Rohini Bisht, Selomie Kindu Ejigu, Gregory Gay, and Predrag Filipovikj. 2023. Identifying Redundancies and Gaps across Testing Levels during Verification of Automotive Software. In Proceedings of the 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW \u201923), 131\u2013139."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.5555\/3545946.3598774"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.106853"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598072"},{"issue":"1","key":"e_1_3_2_14_2","first-page":"32","article-title":"Ethical Adversaries:","volume":"23","author":"Delobelle Pieter","year":"2021","unstructured":"Pieter Delobelle, Paul Temple, Gilles Perrouin, Benoit Fr\u00e9nay, Patrick Heymans, and Bettina Berendt. 2021. Ethical Adversaries: Towards Mitigating Unfairness with Adversarial Machine Learning. SIGKDD Explorations Newsletter 23, 1 (May 2021), 32\u201341.","journal-title":"Towards Mitigating Unfairness with Adversarial Machine Learning"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.5555\/2343576.2343638"},{"key":"e_1_3_2_16_2","doi-asserted-by":"crossref","first-page":"4314","DOI":"10.1109\/ICRA40945.2020.9197145","volume-title":"Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA \u201920)","author":"Ding Wenhao","year":"2020","unstructured":"Wenhao Ding, Mengdi Xu, and Ding Zhao. 2020. CMTS: A Conditional Multiple Trajectory Synthesizer for Generating Safety-Critical Driving Scenarios. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA \u201920). IEEE, 4314\u20134321."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338954"},{"key":"e_1_3_2_18_2","first-page":"103","volume-title":"Proceedings of the 2021 IEEE International Conference on Artificial Intelligence Testing (AITest \u201921)","author":"Ebadi Hamid","year":"2021","unstructured":"Hamid Ebadi, Mahshid Helali Moghadam, Markus Borg, Gregory Gay, Afonso Fontes, and Kasper Socha. 2021. Efficient and Effective Generation of Test Cases for Pedestrian Detection - Search-Based Software Testing of Baidu Apollo in SVL. In Proceedings of the 2021 IEEE International Conference on Artificial Intelligence Testing (AITest \u201921), 103\u2013110."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534392"},{"issue":"1","key":"e_1_3_2_20_2","first-page":"32 pages","article-title":"Automated and Efficient Test-Generation for Grid-Based Multiagent Systems: Comparing Random Input Filtering versus Constraint Solving","volume":"33","author":"Entekhabi Sina","year":"2023","unstructured":"Sina Entekhabi, Wojciech Mostowski, and Mohammad Reza Mousavi. 2023. Automated and Efficient Test-Generation for Grid-Based Multiagent Systems: Comparing Random Input Filtering versus Constraint Solving. ACM Transactions on Software Engineering and Methodology 33, 1, Article 12 (Nov. 2023), 32 pages.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_21_2","first-page":"1","volume-title":"Proceedings of the 2022 10th International Conference on Intelligent Computing and Wireless Optical Communications (ICWOC \u201922)","author":"Fan Chunlong","year":"2022","unstructured":"Chunlong Fan and Yingyu Hao. 2022. Precise Key Frames Adversarial Attack against Deep Reinforcement Learning. In Proceedings of the 2022 10th International Conference on Intelligent Computing and Wireless Optical Communications (ICWOC \u201922), 1\u20136."},{"key":"e_1_3_2_22_2","first-page":"720","volume-title":"Probabilistic Policy Reuse in a Reinforcement Learning Agent (AAMAS \u201906)","author":"Fern\u00e1ndez Fernando","year":"2006","unstructured":"Fernando Fern\u00e1ndez and Manuela Veloso. 2006. Probabilistic Policy Reuse in a Reinforcement Learning Agent (AAMAS \u201906). ACM, 720\u2013727."},{"key":"e_1_3_2_23_2","unstructured":"Chrisantha Fernando Dylan Banarse Charles Blundell Yori Zwols David Ha Andrei A. Rusu Alexander Pritzel and Daan Wierstra. 2017. Pathnet: Evolution Channels Gradient Descent in Super Neural Networks. arXiv:1701.08734. Retrieved from http:\/\/arxiv.org\/abs\/1701.08734"},{"issue":"4","key":"e_1_3_2_24_2","doi-asserted-by":"crossref","first-page":"e1845","DOI":"10.1002\/stvr.1845","article-title":"The Integration of Machine Learning into Automated Test Generation: A Systematic Mapping Study","volume":"33","author":"Fontes Afonso","year":"2023","unstructured":"Afonso Fontes and Gregory Gay. 2023. The Integration of Machine Learning into Automated Test Generation: A Systematic Mapping Study. Software Testing, Verification and Reliability 33, 4 (2023), e1845.","journal-title":"Software Testing, Verification and Reliability"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330566"},{"key":"e_1_3_2_26_2","first-page":"1","volume-title":"Proceedings of the 2nd Workshop on New Frontiers in Adversarial Machine Learning","author":"Ghofrani Fatemeh","year":"2023","unstructured":"Fatemeh Ghofrani, Mehdi Yaghouti, and Pooyan Jamshidi. 2023. Rethinking Robust Contrastive Learning from the Adversarial Perspective. In Proceedings of the 2nd Workshop on New Frontiers in Adversarial Machine Learning, 1\u201314."},{"issue":"3","key":"e_1_3_2_27_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3635709","article-title":"Causality-Driven Testing of Autonomous Driving Systems","volume":"33","author":"Giamattei Luca","year":"2023","unstructured":"Luca Giamattei, Antonio Guerriero, Roberto Pietrantuono, and Stefano Russo. 2023. Causality-Driven Testing of Autonomous Driving Systems. ACM Transactions on Software Engineering and Methodology 33, 3 (Dec. 2023), 1\u201335.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_28_2","first-page":"1","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920)","author":"Gleave Adam","year":"2020","unstructured":"Adam Gleave, Michael Dennis, Cody Wild, Neel Kant, Sergey Levine, and Stuart Russell. 2020. Adversarial Policies: Attacking Deep Reinforcement Learning. In Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920), 1\u201316."},{"key":"e_1_3_2_29_2","first-page":"3910","volume-title":"Proceedings of the 38th International Conference on Machine Learning (ICML \u201921)","volume":"139","author":"Guo Wenbo","year":"2021","unstructured":"Wenbo Guo, Xian Wu, Sui Huang, and Xinyu Xing. 2021. Adversarial Policy Learning in Two-player Competitive Games. In Proceedings of the 38th International Conference on Machine Learning (ICML \u201921), Vol. 139, PMLR, 3910\u20133919."},{"key":"e_1_3_2_30_2","first-page":"1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Gupta Abhishek","year":"2017","unstructured":"Abhishek Gupta, Coline Devin, YuXuan Liu, Pieter Abbeel, and Sergey Levine. 2017. Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning. In Proceedings of the International Conference on Learning Representations, 1\u201314."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00155"},{"key":"e_1_3_2_32_2","first-page":"4565","article-title":"Generative Adversarial Imitation Learning","volume":"29","author":"Ho Jonathan","year":"2016","unstructured":"Jonathan Ho and Stefano Ermon. 2016. Generative Adversarial Imitation Learning. In Advances in Neural Information Processing Systems (NIPS \u201916), Vol. 29, 4565\u20134573.","journal-title":"Advances in Neural Information Processing Systems (NIPS \u201916)"},{"key":"e_1_3_2_33_2","volume-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR \u201917)","author":"Huang Sandy H.","year":"2017","unstructured":"Sandy H. Huang, Nicolas Papernot, Ian J. Goodfellow, Yan Duan, and Pieter Abbeel. 2017. Adversarial Attacks on Neural Network Policies. In Proceedings of the 5th International Conference on Learning Representations (ICLR \u201917), Workshop Track Proceedings, 1\u20137."},{"key":"e_1_3_2_34_2","first-page":"43","volume-title":"Proceedings of the 2019 USENIX Conference on Operational Machine Learning (OpML \u201919)","author":"Iqbal Md Shahriar","year":"2019","unstructured":"Md Shahriar Iqbal, Lars Kotthoff, and Pooyan Jamshidi. 2019. Transfer Learning for Performance Modeling of Deep Neural Network Systems. In Proceedings of the 2019 USENIX Conference on Operational Machine Learning (OpML \u201919), 43\u201346."},{"key":"e_1_3_2_35_2","first-page":"7976","article-title":"Champion-Level Drone Racing Using Deep Reinforcement Learning","volume":"620","author":"Kaufmann Elia","year":"2023","unstructured":"Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias M\u00fcller, Vladlen Koltun, and Davide Scaramuzza. 2023. Champion-Level Drone Racing Using Deep Reinforcement Learning. Nature 620, 7976 (2023), 982\u2013987.","journal-title":"Nature"},{"issue":"3","key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3631969","article-title":"Safety of Perception Systems for Automated Driving: A Case Study on Apollo","volume":"33","author":"Kochanthara Sangeeth","year":"2023","unstructured":"Sangeeth Kochanthara, Tajinder Singh, Alexandru Forrai, and Loek Cleophas. 2023. Safety of Perception Systems for Automated Driving: A Case Study on Apollo. ACM Transactions on Software Engineering and Methodology 33, 3 (Nov. 2023), 1\u201328.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_37_2","unstructured":"Jernej Kos and Dawn Song. 2017. Delving into Adversarial Attacks on Deep Policies. arXiv:1705.06452. Retrieved from https:\/\/arxiv.org\/abs\/1705.06452"},{"key":"e_1_3_2_38_2","first-page":"25","volume-title":"Proceedings of the 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE \u201920)","author":"Li Guanpeng","year":"2020","unstructured":"Guanpeng Li, Yiran Li, Saurabh Jha, Timothy Tsai, Michael Sullivan, Siva Kumar Sastry Hari, Zbigniew Kalbarczyk, and Ravishankar Iyer. 2020. Av-fuzzer: Finding Safety Violations in Autonomous Driving Systems. In Proceedings of the 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE \u201920), 25\u201336."},{"key":"e_1_3_2_39_2","first-page":"365","volume-title":"Proceedings of the Reinforcement Learning for Sequential Decision and Optimal Control","author":"Li Shengbo Eben","year":"2023","unstructured":"Shengbo Eben Li. 2023. Deep Reinforcement Learning. In Proceedings of the Reinforcement Learning for Sequential Decision and Optimal Control. Springer, 365\u2013402."},{"key":"e_1_3_2_40_2","first-page":"243","volume-title":"Proceedings of the 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923)","author":"Li Zhuo","year":"2023","unstructured":"Zhuo Li, Xiongfei Wu, Derui Zhu, Mingfei Cheng, Siyuan Chen, Fuyuan Zhang, Xiaofei Xie, Lei Ma, and Jianjun Zhao. 2023. Generative Model-Based Testing on Decision-Making Policies. In Proceedings of the 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923), 243\u2013254."},{"key":"e_1_3_2_41_2","first-page":"1","volume-title":"Proceedings of the 11th International Conference on Learning Representations (ICLR \u201923)","author":"Li Zhuo","year":"2023","unstructured":"Zhuo Li, Derui Zhu, Yujing Hu, Xiaofei Xie, Lei Ma, YAN ZHENG, Yan Song, Yingfeng Chen, and Jianjun Zhao. 2023. Neural Episodic Control with State Abstraction. In Proceedings of the 11th International Conference on Learning Representations (ICLR \u201923), 1\u201318."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2017\/525"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549111"},{"key":"e_1_3_2_44_2","first-page":"14543","article-title":"Policy Poisoning in Batch Reinforcement Learning and Control","volume":"32","author":"Ma Yuzhe","year":"2019","unstructured":"Yuzhe Ma, Xuezhou Zhang, Wen Sun, and Jerry Zhu. 2019. Policy Poisoning in Batch Reinforcement Learning and Control. In Advances in Neural Information Processing Systems (NIPS), Vol. 32, 14543\u201314553.","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"e_1_3_2_45_2","first-page":"1","article-title":"Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective","author":"Meng Mark Huasong","year":"2022","unstructured":"Mark Huasong Meng, Guangdong Bai, Sin Gee Teo, Zhe Hou, Yan Xiao, Yun Lin, and Jin Song Dong. 2022. Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective. IEEE Transactions on Dependable and Secure Computing (2022), 1\u20131.","journal-title":"IEEE Transactions on Dependable and Secure Computing"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8463162"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3640335"},{"key":"e_1_3_2_48_2","first-page":"278","article-title":"Policy Invariance under Reward Transformations: Theory and Application to Reward Shaping","volume":"99","author":"Ng Andrew Y.","year":"1999","unstructured":"Andrew Y. Ng, Daishi Harada, and Stuart Russell. 1999. Policy Invariance under Reward Transformations: Theory and Application to Reward Shaping. In ICML, Vol. 99, 278\u2013287.","journal-title":"ICML"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534388"},{"key":"e_1_3_2_50_2","unstructured":"Emilio Parisotto Jimmy Lei Ba and Ruslan Salakhutdinov. 2015. Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning. arXiv:1511.06342. Retrieved from https:\/\/arxiv.org\/abs\/1511.06342"},{"key":"e_1_3_2_51_2","first-page":"8093","volume-title":"Proceedings of the 37th International Conference on Machine Learning","volume":"119","author":"Rice Leslie","year":"2020","unstructured":"Leslie Rice, Eric Wong, and Zico Kolter. 2020. Overfitting in Adversarially Robust Deep Learning. In Proceedings of the 37th International Conference on Machine Learning, Vol. 119, PMLR, 8093\u20138104."},{"key":"e_1_3_2_52_2","unstructured":"Andrei A. Rusu Sergio Gomez Colmenarejo Caglar Gulcehre Guillaume Desjardins James Kirkpatrick Razvan Pascanu Volodymyr Mnih Koray Kavukcuoglu and Raia Hadsell. 2015. Policy distillation. arXiv:1511.06295. Retrieved from https:\/\/arxiv.org\/abs\/1511.06295"},{"key":"e_1_3_2_53_2","unstructured":"John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347. Retrieved from https:\/\/arxiv.org\/abs\/1707.06347"},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/AST52587.2021.00010","volume-title":"Proceedings of the 2021 IEEE\/ACM International Conference on Automation of Software Test (AST \u201921)","author":"Sedaghatbaf Ali","year":"2021","unstructured":"Ali Sedaghatbaf, Mahshid Helali Moghadam, and Mehrdad Saadatmand. 2021. Automated Performance Testing Based on Active Deep Learning. In Proceedings of the 2021 IEEE\/ACM International Conference on Automation of Software Test (AST \u201921). IEEE, 11\u201319."},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3640334"},{"issue":"5","key":"e_1_3_2_56_2","first-page":"62","article-title":"A Survey on Automated Driving System Testing: Landscapes and Trends","volume":"32","author":"Tang Shuncheng","year":"2023","unstructured":"Shuncheng Tang, Zhenya Zhang, Yi Zhang, Jixiang Zhou, Yan Guo, Shuang Liu, Shengjian Guo, Yan-Fu Li, Lei Ma, Yinxing Xue, and Yang Liu. 2023. A Survey on Automated Driving System Testing: Landscapes and Trends. ACM Transactions on Software Engineering and Methodology 32, 5, Article 124 (Jul. 2023), 62 pages.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_57_2","first-page":"6:1","volume-title":"Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924)ACM","author":"Tappler Martin","year":"2024","unstructured":"Martin Tappler, Andrea Pferscher, Bernhard K. Aichernig, and Bettina K\u00f6nighofer. 2024. Learning and Repair of Deep Reinforcement Learning Policies from Fuzz-Testing Data. In Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924). ACM, 6:1\u20136:13."},{"key":"e_1_3_2_58_2","first-page":"277","volume-title":"Proceedings of the 23rd International Systems and Software Product Line Conference","author":"Temple Paul","year":"2019","unstructured":"Paul Temple, Mathieu Acher, Gilles Perrouin, Battista Biggio, Jean-Marc Jezequel, and Fabio Roli. 2019. Towards Quality Assurance of Software Product Lines with Adversarial Configurations. In Proceedings of the 23rd International Systems and Software Product Line Conference, Vol. A, 277\u2013288."},{"key":"e_1_3_2_59_2","first-page":"1","volume-title":"Proceedings of the 1st Workshop on Deep Learning < = > Testing","author":"Temple Paul","year":"2019","unstructured":"Paul Temple, Gilles Perrouin, Beno\u00eet Fr\u00e9nay, and Pierre-Yves Schobbens. 2019. Customizing Adversarial Machine Learning to Test Deep Learning Techniques. In Proceedings of the 1st Workshop on Deep Learning < = > Testing, 1\u20132."},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_61_2","first-page":"1072","volume-title":"Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE \u201921)","author":"Velez Miguel","year":"2021","unstructured":"Miguel Velez, Pooyan Jamshidi, Norbert Siegmund, Sven Apel, and Christian K\u00e4stner. 2021. White-Box Analysis over Machine Learning: Modeling Performance of Configurable Systems. In Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE \u201921), 1072\u20131084."},{"key":"e_1_3_2_62_2","unstructured":"Michael Wan Tanmay Gangwani and Jian Peng. 2020. Mutual Information Based Knowledge Transfer Under State-Action Dimension Mismatch. arXiv:2006.07041. Retrieved from https:\/\/arxiv.org\/abs\/2006.07041"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616266"},{"key":"e_1_3_2_64_2","first-page":"6586","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Wang Yisen","year":"2019","unstructured":"Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. 2019. On the Convergence and Robustness of Adversarial Training. In Proceedings of the International Conference on Machine Learning, 6586\u20136595."},{"key":"e_1_3_2_65_2","first-page":"792","volume-title":"Proceedings of the 20th International Conference on Machine Learning (ICML \u201903)","author":"Wiewiora Eric","year":"2003","unstructured":"Eric Wiewiora, Garrison W. Cottrell, and Charles Elkan. 2003. Principled Methods for Advising Reinforcement Learning Agents. In Proceedings of the 20th International Conference on Machine Learning (ICML \u201903), 792\u2013799."},{"key":"e_1_3_2_66_2","first-page":"1883","volume-title":"Proceedings of the 30th USENIX Security Symposium, USENIX Security 2021","author":"Wu Xian","year":"2021","unstructured":"Xian Wu, Wenbo Guo, Hua Wei, and Xinyu Xing. 2021. Adversarial Policy Training against Deep Reinforcement Learning. In Proceedings of the 30th USENIX Security Symposium, USENIX Security 2021. USENIX Association, 1883\u20131900."},{"issue":"3","key":"e_1_3_2_67_2","first-page":"27","article-title":"NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks","volume":"31","author":"Xie Xiaofei","year":"2022","unstructured":"Xiaofei Xie, Tianlin Li, Jian Wang, Lei Ma, Qing Guo, Felix Juefei-Xu, and Yang Liu. 2022. NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks. ACM Transactions on Software Engineering and Methodology 31, 3, Article 47 (Apr. 2022), 27 pages.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_68_2","first-page":"1","volume-title":"Proceedings of the 2020 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA \u201920)","author":"Zak Yuval","year":"2020","unstructured":"Yuval Zak, Yisrael Parmet, and Tal Oron-Gilad. 2020. An Artificial Intelligence Algorithm to Automate Situation Management for Operators of Unmanned Aerial Vehicles. In Proceedings of the 2020 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA \u201920), 1\u20139."},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616370"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2017.8206049"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238187"},{"key":"e_1_3_2_72_2","article-title":"Finding Critical Scenarios for Automated Driving Systems:","author":"Zhang Xinhai","year":"2021","unstructured":"Xinhai Zhang, Jianbo Tao, Kaige Tan, Martin T\u00f6rngren, Jos\u00e9 Manuel Gaspar S\u00e1nchez, Muhammad Rusyadi Ramli, Xin Tao, Magnus Gyllenhammar, Franz Wotawa, Naveen Mohan, Mihai Nica, and Hermann Felbinger. 2021. Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review. arXiv: 2110.08664. Retrieved from https:\/\/arxiv.org\/abs\/2110.08664","journal-title":"A Systematic Literature Review"},{"issue":"3","key":"e_1_3_2_73_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3631977","article-title":"Attack as Detection: Using Adversarial Attack Methods to Detect Abnormal Examples","volume":"33","author":"Zhao Zhe","year":"2023","unstructured":"Zhe Zhao, Guangke Chen, Tong Liu, Taishan Li, Fu Song, Jingyi Wang, and Jun Sun. 2023. Attack as Detection: Using Adversarial Attack Methods to Detect Abnormal Examples. ACM Transactions on Software Engineering and Methodology 33, 3 (Nov. 2023), 1\u201345.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_74_2","first-page":"772","volume-title":"Proceedings of the 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201919)","author":"Zheng Yan","year":"2019","unstructured":"Yan Zheng, Changjie Fan, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, and Yingfeng Chen. 2019. Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning. In Proceedings of the 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201919). IEEE, 772\u2013784."},{"key":"e_1_3_2_75_2","first-page":"11","article-title":"Transfer Learning in Deep Reinforcement Learning: A Survey","volume":"45","author":"Zhu Zhuangdi","year":"2023","unstructured":"Zhuangdi Zhu, Kaixiang Lin, Anil K. Jain, and Jiayu Zhou. 2023. Transfer Learning in Deep Reinforcement Learning: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 11 (2023), 13344\u201313362.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696001","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3696001","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:04:30Z","timestamp":1750291470000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696001"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,20]]},"references-count":74,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3696001"],"URL":"https:\/\/doi.org\/10.1145\/3696001","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,20]]},"assertion":[{"value":"2024-04-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-30","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}