{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T02:19:01Z","timestamp":1781835541561,"version":"3.54.5"},"reference-count":90,"publisher":"Association for Computing Machinery (ACM)","issue":"8","license":[{"start":{"date-parts":[[2024,11,30]],"date-time":"2024-11-30T00:00:00Z","timestamp":1732924800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"crossref","award":["RGPIN-2019-06956"],"award-info":[{"award-number":["RGPIN-2019-06956"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100007631","name":"Canadian Institute for Advanced Research","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100007631","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,11,30]]},"abstract":"<jats:p>Evolutionary search (ES)-based techniques are commonly used for testing autonomous robotic systems. However, these approaches often rely on computationally expensive simulator-based models for test scenario evaluation. To improve the computational efficiency of the search-based testing, we propose augmenting the ES with a reinforcement learning (RL) agent trained using surrogate rewards derived from domain knowledge. In our approach, known as RIGAA (Reinforcement learning Informed Genetic Algorithm for Autonomous systems testing), we first train an RL agent to learn useful constraints of the problem and then use it to produce a certain part of the initial population of the search algorithm. By incorporating an RL agent into the search process, we aim to guide the algorithm towards promising regions of the search space from the start, enabling more efficient exploration of the solution space. We evaluate RIGAA on two case studies: maze generation for an autonomous \u201cAnt\u201d robot and road topology generation for an autonomous vehicle lane-keeping assist system. In both case studies, RIGAA reveals more failures of a high level of diversity than the compared baselines. RIGAA also outperforms the state-of-the-art tools for vehicle lane-keeping assist system testing, such as AmbieGen, CRAG, WOGAN, and Frenetic in terms of the number of revealed failures in a two-hour budget.<\/jats:p>","DOI":"10.1145\/3680468","type":"journal-article","created":{"date-parts":[[2024,7,27]],"date-time":"2024-07-27T14:52:37Z","timestamp":1722091957000},"page":"1-45","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Reinforcement Learning Informed Evolutionary Search for Autonomous Systems Testing"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2983-8312","authenticated-orcid":false,"given":"Dmytro","family":"Humeniuk","sequence":"first","affiliation":[{"name":"Polytechnique Montr\u00e9al, Montreal, QC, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5704-4173","authenticated-orcid":false,"given":"Foutse","family":"Khomh","sequence":"additional","affiliation":[{"name":"Polytechnique Montr\u00e9al, Montreal, QC, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7892-9202","authenticated-orcid":false,"given":"Giuliano","family":"Antoniol","sequence":"additional","affiliation":[{"name":"Polytechnique Montr\u00e9al, Montreal, QC, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,12,3]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"1016","volume-title":"Proceedings of the 2018 IEEE\/ACM 40th International Conference on Software Engineering (ICSE)","author":"Abdessalem Raja Ben","year":"2018","unstructured":"Raja Ben Abdessalem, Shiva Nejati, Lionel C. Briand, and Thomas Stifter. 2018. Testing vision-based control systems using learnable evolutionary algorithms. In Proceedings of the 2018 IEEE\/ACM 40th International Conference on Software Engineering (ICSE). IEEE, 1016\u20131026."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238192"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-19835-9_21"},{"key":"e_1_3_2_5_2","first-page":"240","volume-title":"Proceedings of the 21st IEEE International Conference on Software Maintenance (ICSM \u201905)","author":"Antoniol Giuliano","year":"2005","unstructured":"Giuliano Antoniol, Massimiliano Di Penta, and Mark Harman. 2005. Search-based techniques applied to optimization of project planning for a massive maintenance project. In Proceedings of the 21st IEEE International Conference on Software Maintenance (ICSM \u201905). IEEE, 240\u2013249."},{"key":"e_1_3_2_6_2","first-page":"41","volume-title":"Proceedings of the IEEE\/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","author":"Arcaini Paolo","year":"2023","unstructured":"Paolo Arcaini and Ahmet Cetinkaya. 2023. Crag at the sbft 2023 tool competition-cyber-physical systems track. In Proceedings of the IEEE\/ACM International Workshop on Search-Based and Fuzz Testing (SBFT), IEEE, 41\u201342."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1486"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-89694-4_7"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1007\/978-3-642-40793-2_4","volume-title":"Proceedings of the 32nd International Conference on Computer Safety, Reliability, and Security (SAFECOMP \u201913)","author":"Arnold James","year":"2013","unstructured":"James Arnold and Rob Alexander. 2013. Testing autonomous robot control software using procedural content generation. In Proceedings of the 32nd International Conference on Computer Safety, Reliability, and Security (SAFECOMP \u201913). Springer, 33\u201344."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3532182"},{"issue":"1","key":"e_1_3_2_11_2","first-page":"B1","article-title":"Handbook of evolutionary computation","volume":"97","author":"B\u00e4ck Thomas","year":"1997","unstructured":"Thomas B\u00e4ck, David B. Fogel, and Zbigniew Michalewicz. 1997. Handbook of evolutionary computation. Release 97, 1 (1997), B1.","journal-title":"Release"},{"key":"e_1_3_2_12_2","unstructured":"BeamNG GmbH. 2022-06-15. BeamNG.tech. Retrieved from https:\/\/www.beamng.tech\/"},{"key":"e_1_3_2_13_2","unstructured":"Christopher Berner Greg Brockman Brooke Chan Vicki Cheung Przemys\u0142aw Debiak Christy Dennison David Farhi Quirin Fischer Shariq Hashme Chris Hesse Rafal J\u00f3zefowicz Scott Gray Catherine Olsson Jakub Pachocki Michael Petrov Henrique P. d.O. Pinto Jonathan Raiman Tim Salimans Jeremy Schlatter Jonas Schneider Szymon Sidor Ilya Sutskever Jie Tang Filip Wolski and Susan Zhang. 2019. Dota 2 with large scale deep reinforcement learning. arXiv:1912.06680. Retrieved from https:\/\/arxiv.org\/abs\/1912.06680"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2006.08.008"},{"key":"e_1_3_2_15_2","first-page":"45","volume-title":"Proceedings of the IEEE\/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","author":"Biagiola Matteo","year":"2023","unstructured":"Matteo Biagiola, Stefan Klikovits, Jarkko Peltom\u00e4ki, and Vincenzo Riccio. 2023. Sbft tool competition 2023-cyber-physical systems track. In Proceedings of the IEEE\/ACM International Workshop on Search-Based and Fuzz Testing (SBFT). IEEE, 45\u201348."},{"key":"e_1_3_2_16_2","first-page":"164","volume-title":"Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","author":"Birchler Christian","year":"2022","unstructured":"Christian Birchler, Nicolas Ganz, Sajad Khatiri, Alessio Gambi, and Sebastiano Panichella. 2022. Cost-effective simulation-based test selection in self-driving cars software with SDC-Scissor. In Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 164\u2013168."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2990567"},{"key":"e_1_3_2_18_2","volume-title":"Parallel Surrogate-Based Algorithms for Solving Expensive Optimization Problems","author":"Briffoteaux Guillaume","year":"2022","unstructured":"Guillaume Briffoteaux. 2022. Parallel Surrogate-Based Algorithms for Solving Expensive Optimization Problems. Ph.D. Dissertation. Universit\u00e9 de Lille; Universit\u00e9 de Mons (UMONS)."},{"key":"e_1_3_2_19_2","unstructured":"Yuri Burda Harrison Edwards Amos Storkey and Oleg Klimov. 2018. Exploration by random network distillation. arXiv:1810.12894. Retrieved from https:\/\/arxiv.org\/abs\/1810.12894"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/70.481750"},{"key":"e_1_3_2_21_2","first-page":"36","volume-title":"Proceedings of the IEEE\/ACM 14th International Workshop on Search-Based Software Testing (SBST)","author":"Castellano Ezequiel","year":"2021","unstructured":"Ezequiel Castellano, Ahmet Cetinkaya, C\u00e9dric Ho Thanh, Stefan Klikovits, Xiaoyi Zhang, and Paolo Arcaini. 2021. Frenetic at the SBST 2021 tool competition. In Proceedings of the IEEE\/ACM 14th International Workshop on Search-Based Software Testing (SBST). IEEE, 36\u201337."},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2021.3091477"},{"key":"e_1_3_2_23_2","article-title":"Reinforcement learning-based genetic algorithm in optimizing multidimensional data discretization scheme","volume":"2020","author":"Chen Qiong","year":"2020","unstructured":"Qiong Chen, Mengxing Huang, Qiannan Xu, Hao Wang, and Jinghui Wang. 2020. Reinforcement learning-based genetic algorithm in optimizing multidimensional data discretization scheme. Mathematical Problems in Engineering 2020 (2020).","journal-title":"Mathematical Problems in Engineering"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2019.05.013"},{"key":"e_1_3_2_25_2","first-page":"320","volume-title":"Proceedings of the 9th Asian Computing Science Conference on Advances in Computer Science-ASIAN 2004. Higher-Level Decision Making","author":"Chen Tsong Yueh","year":"2005","unstructured":"Tsong Yueh Chen, Hing Leung, and Ieng Kei Mak. 2005. Adaptive random testing. In Proceedings of the 9th Asian Computing Science Conference on Advances in Computer Science-ASIAN 2004. Higher-Level Decision Making. Springer, 320\u2013329."},{"issue":"15","key":"e_1_3_2_26_2","first-page":"1","article-title":"Jaccard\/Tanimoto similarity test and estimation methods for biological presence-absence data","volume":"20","author":"Chung Neo Christopher","year":"2019","unstructured":"Neo Christopher Chung, Ba\u017bej Miasojedow, Micha Startek, and Anna Gambin. 2019. Jaccard\/Tanimoto similarity test and estimation methods for biological presence-absence data. BMC Bioinformatics 20, 15 (2019), 1\u201311.","journal-title":"BMC Bioinformatics"},{"key":"e_1_3_2_27_2","volume-title":"Evolutionary Algorithms for Solving Multi-Objective Problems","author":"Coello Carlos A. Coello","year":"2007","unstructured":"Carlos A. Coello Coello, Gary B. Lamont, and David A. Van Veldhuizen. 2007. Evolutionary Algorithms for Solving Multi-Objective Problems, Vol. 5, Springer."},{"key":"e_1_3_2_28_2","volume-title":"Implementation of the Pure Pursuit Path Tracking Algorithm","author":"Coulter R. Craig","year":"1992","unstructured":"R. Craig Coulter. 1992. Implementation of the Pure Pursuit Path Tracking Algorithm. Technical Report. Carnegie-Mellon UNIV Pittsburgh PA Robotics INST."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2013.2281535"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/4235.996017"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-44874-8"},{"key":"e_1_3_2_32_2","first-page":"1407","volume-title":"International Conference on Machine Learning","author":"Espeholt Lasse","year":"2018","unstructured":"Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, Shane Legg, and Koray Kavukcuoglu. 2018. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures. In International Conference on Machine Learning. PMLR, 1407\u20131416."},{"key":"e_1_3_2_33_2","first-page":"1157","volume-title":"Proceedings of the IEEE International Conference on Evolutionary Computation","author":"Fonseca Carlos M","year":"2006","unstructured":"Carlos M Fonseca, Lu\u00eds Paquete, and Manuel L\u00f3pez-Ib\u00e1nez. 2006. An improved dimension-sweep algorithm for the hypervolume indicator. In Proceedings of the IEEE International Conference on Evolutionary Computation. IEEE, 1157\u20131163."},{"key":"e_1_3_2_34_2","unstructured":"Farma Foundation. 2020. Ant Gym. Retrieved from https:\/\/gymnasium.farama.org\/environments\/mujoco\/ant\/"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2012.92"},{"key":"e_1_3_2_36_2","unstructured":"Justin Fu Aviral Kumar Ofir Nachum George Tucker and Sergey Levine. 2020. D4RL: Datasets for deep data-driven reinforcement learning. arXiv:2004.07219. Retrieved from https:\/\/arxiv.org\/abs\/2004.07219"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3526072.3527538"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330566"},{"key":"e_1_3_2_39_2","first-page":"95","volume-title":"Proceedings of the IEEE International Conference on Artificial Intelligence Testing (AITest)","author":"Gambi Alessio","year":"2022","unstructured":"Alessio Gambi, Vuong Nguyen, Jasim Ahmed, and Gordon Fraser. 2022b. Generating critical driving scenarios from accident sketches. In Proceedings of the IEEE International Conference on Artificial Intelligence Testing (AITest). IEEE, 95\u2013102."},{"key":"e_1_3_2_40_2","first-page":"8806","volume-title":"Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)","author":"Gog Ionel","year":"2021","unstructured":"Ionel Gog, Sukrit Kalra, Peter Schafhalter, Matthew A. Wright, Joseph E. Gonzalez, and Ion Stoica. 2021. Pylot: A modular platform for exploring latency-accuracy tradeoffs in autonomous vehicles. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). IEEE, 8806\u20138813."},{"key":"e_1_3_2_41_2","first-page":"69","article-title":"A comparative analysis of selection schemes used in genetic algorithms","volume":"1","author":"Goldberg David E.","year":"1991","unstructured":"David E. Goldberg and Kalyanmoy Deb. 1991. A comparative analysis of selection schemes used in genetic algorithms. In Foundations of Genetic Algorithms, Vol. 1, Elsevier, 69\u201393.","journal-title":"Foundations of Genetic Algorithms"},{"key":"e_1_3_2_42_2","first-page":"811","volume-title":"Proceedings of the 44th International Conference on Software Engineering","author":"Ul Haq Fitash","year":"2022","unstructured":"Fitash Ul Haq, Donghwan Shin, and Lionel Briand. 2022. Efficient online testing for DNN-enabled systems using surrogate-assisted and many-objective optimization. In Proceedings of the 44th International Conference on Software Engineering, 811\u2013822."},{"key":"e_1_3_2_43_2","first-page":"1814","volume-title":"Proceedings of the IEEE\/ACM 45th International Conference on Software Engineering (ICSE)","author":"Ul Haq Fitash","year":"2023","unstructured":"Fitash Ul Haq, Donghwan Shin, and Lionel C. Briand. 2023. Many-objective reinforcement learning for online testing of dnn-enabled systems. In Proceedings of the IEEE\/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1814\u20131826."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(01)00189-6"},{"key":"e_1_3_2_45_2","first-page":"1","volume-title":"LASER Summer School on Software Engineering","author":"Harman Mark","year":"2008","unstructured":"Mark Harman, Phil McMinn, Jerffeson Teixeira De Souza, and Shin Yoo. 2008. Search based software engineering: Techniques, taxonomy, tutorial. In LASER Summer School on Software Engineering. Springer, 1\u201359."},{"key":"e_1_3_2_46_2","unstructured":"Nicolas Heess Dhruva Tb Srinivasan Sriram Jay Lemmon Josh Merel Greg Wayne Yuval Tassa Tom Erez Ziyu Wang S. M. Eslami Martin Riedmiller and David Silver. 2017. Emergence of locomotion behaviours in rich environments. arXiv:1707.02286. Retrieved from https:\/\/arxiv.org\/abs\/1707.02286"},{"key":"e_1_3_2_47_2","unstructured":"Ashley Hill Antonin Raffin Maximilian Ernestus Adam Gleave Anssi Kanervisto Rene Traore Prafulla Dhariwal Christopher Hesse Oleg Klimov Alex Nichol Matthias Plappert Alec Radford John Schulman Szymon Sidor and Yuhuai Wu. 2018. Stable Baselines. Retrieved from https:\/\/github.com\/hill-a\/stable-baselines"},{"key":"e_1_3_2_48_2","volume-title":"Proceedings of the International Workshop on Design and Implementation of Formal Tools and Systems","author":"Hoxha Bardh","year":"2014","unstructured":"Bardh Hoxha, Hoang Bach, Houssam Abbas, Adel Dokhanchi, Yoshihiro Kobayashi, and Georgios Fainekos. 2014. Towards formal specification visualization for testing and monitoring of cyber-physical systems. In Proceedings of the International Workshop on Design and Implementation of Formal Tools and Systems."},{"issue":"274","key":"e_1_3_2_49_2","first-page":"1","article-title":"CleanRL: High-quality single-file implementations of deep reinforcement learning algorithms","volume":"23","author":"Huang Shengyi","year":"2022","unstructured":"Shengyi Huang, Rousslan Fernand Julien Dossa, Chang Ye, Jeff Braga, Dipam Chakraborty, Kinal Mehta, and Jo\u00e3o G. M. Ara\u00fajo. 2022. CleanRL: High-quality single-file implementations of deep reinforcement learning algorithms. Journal of Machine Learning Research 23, 274 (2022), 1\u201318. Retrieved from http:\/\/jmlr.org\/papers\/v23\/21-1342.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.106936"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2023.102990"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.8242223"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/34.232073"},{"key":"e_1_3_2_54_2","first-page":"1009","article-title":"Actor-critic algorithms","volume":"12","author":"Konda Vijay","year":"1999","unstructured":"Vijay Konda and John Tsitsiklis. 1999. Actor-critic algorithms. Advances in Neural Information Processing Systems 12 (1999), 1009\u20131014.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.5555\/183459"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.12190"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/2001576.2001606"},{"key":"e_1_3_2_58_2","first-page":"707","volume-title":"Soviet Physics Doklady","author":"Levenshtein Vladimir I.","year":"1966","unstructured":"Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet Physics Doklady, Vol. 10, Soviet Union, 707\u2013710."},{"key":"e_1_3_2_59_2","first-page":"387","volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation (CEC)","author":"Lopez-Herrejon Roberto E","year":"2014","unstructured":"Roberto E Lopez-Herrejon, Javier Ferrer, Francisco Chicano, Alexander Egyed, and Enrique Alba. 2014. Comparative analysis of classical multi-objective evolutionary algorithms and seeding strategies for pairwise testing of software product lines. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC). IEEE, 387\u2013396."},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.11144\/Javeriana.upsy10-2.cdcp"},{"key":"e_1_3_2_61_2","series-title":"Proceedings of Machine Learning Research","first-page":"561","volume-title":"Proceedings of the 2nd Conference on Robot Learning","volume":"87","author":"Mahmood A. Rupam","year":"2018","unstructured":"A. Rupam Mahmood, Dmytro Korenkevych, Gautham Vasan, William Ma, and James Bergstra. 2018. Benchmarking reinforcement learning algorithms on real-world robots. In Proceedings of the 2nd Conference on Robot Learning. Aude Billard, Anca Dragan, Jan Peters, and Jun Morimoto (Eds.), Proceedings of Machine Learning Research, Vol. 87, PMLR, 561\u2013591. Retrieved from https:\/\/proceedings.mlr.press\/v87\/mahmood18a.html"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730491"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931054"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.2000.10485979"},{"key":"e_1_3_2_65_2","doi-asserted-by":"crossref","first-page":"372","DOI":"10.1145\/3377811.3380370","volume-title":"Proceedings of the ACM\/IEEE 42nd International Conference on Software Engineering","author":"Menghi Claudio","year":"2020","unstructured":"Claudio Menghi, Shiva Nejati, Lionel Briand, and Yago Isasi Parache. 2020. Approximation-refinement testing of compute-intensive cyber-physical models: An approach based on system identification. In Proceedings of the ACM\/IEEE 42nd International Conference on Software Engineering, 372\u2013384."},{"key":"e_1_3_2_66_2","first-page":"1928","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Mnih Volodymyr","year":"2016","unstructured":"Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In Proceedings of the International Conference on Machine Learning. PMLR, 1928\u20131937."},{"key":"e_1_3_2_67_2","unstructured":"Volodymyr Mnih Koray Kavukcuoglu David Silver Alex Graves Ioannis Antonoglou Daan Wierstra and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv:1312.5602. Retrieved from https:\/\/arxiv.org\/abs\/1312.5602"},{"key":"e_1_3_2_68_2","doi-asserted-by":"crossref","unstructured":"Mahshid Helali Moghadam Markus Borg Mehrdad Saadatmand Seyed Jalaleddin Mousavirad Markus Bohlin and Bj\u00f6rn Lisper. 2022. Machine learning testing in an ADAS case study using simulation-integrated bio-inspired search-based testing. arXiv:2203.12026. Retrieved from https:\/\/arxiv.org\/abs\/2203.12026","DOI":"10.1002\/smr.2591"},{"key":"e_1_3_2_69_2","first-page":"20","volume-title":"Proceedings of the IEEE\/ACM 14th International Workshop on Search-Based Software Testing (SBST)","author":"Panichella Sebastiano","year":"2021","unstructured":"Sebastiano Panichella, Alessio Gambi, Fiorella Zampetti, and Vincenzo Riccio. 2021. Sbst tool competition 2021. In Proceedings of the IEEE\/ACM 14th International Workshop on Search-Based Software Testing (SBST). IEEE, 20\u201327."},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.5555\/2955491.2955607"},{"key":"e_1_3_2_71_2","first-page":"183","volume-title":"Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering","volume":"220","author":"Plummer Andrew R.","year":"2006","unstructured":"Andrew R. Plummer. 2006. Model-in-the-loop testing. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering 220, 3 (2006), 183\u2013199."},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3449726.3463203"},{"issue":"268","key":"e_1_3_2_73_2","first-page":"1","article-title":"Stable-Baselines3: Reliable reinforcement learning implementations","volume":"22","author":"Raffin Antonin","year":"2021","unstructured":"Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. 2021. Stable-Baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research 22, 268 (2021), 1\u20138. Retrieved from http:\/\/jmlr.org\/papers\/v22\/20-1364.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_74_2","volume-title":"Vehicle Dynamics and Control","author":"Rajamani Rajesh","year":"2011","unstructured":"Rajesh Rajamani. 2011. Vehicle Dynamics and Control. Springer Science & Business Media."},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409730"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409730"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1601"},{"key":"e_1_3_2_78_2","first-page":"2548","volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation","author":"Rosales-P\u00e9rez Alejandro","year":"2013","unstructured":"Alejandro Rosales-P\u00e9rez, Carlos A. Coello Coello, Jesus A. Gonzalez, Carlos A. Reyes-Garcia, and Hugo Jair Escalante. 2013. A hybrid surrogate-based approach for evolutionary multi-objective optimization. In Proceedings of the IEEE Congress on Evolutionary Computation. IEEE, 2548\u20132555."},{"key":"e_1_3_2_79_2","volume-title":"Artificial Intelligence: A Modern Approach","author":"Russell Stuart J.","year":"2016","unstructured":"Stuart J. Russell and Peter Norvig. 2016. Artificial Intelligence: A Modern Approach. Pearson."},{"key":"e_1_3_2_80_2","first-page":"74","volume-title":"Proceedings of the 6th International Conference on Signal-Image Technology and Internet Based Systems","author":"Sakurai Yoshitaka","year":"2010","unstructured":"Yoshitaka Sakurai, Kouhei Takada, Takashi Kawabe, and Setsuo Tsuruta. 2010. A method to control parameters of evolutionary algorithms by using reinforcement learning. In Proceedings of the 6th International Conference on Signal-Image Technology and Internet Based Systems. IEEE, 74\u201379."},{"key":"e_1_3_2_81_2","unstructured":"John Schulman Philipp Moritz Sergey Levine Michael Jordan and Pieter Abbeel. 2015. High-dimensional continuous control using generalized advantage estimation. arXiv:1506.02438. Retrieved from https:\/\/arxiv.org\/abs\/1506.02438"},{"key":"e_1_3_2_82_2","unstructured":"John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv:1707.06347. Retrieved from https:\/\/arxiv.org\/abs\/1707.06347"},{"key":"e_1_3_2_83_2","article-title":"Practical bayesian optimization of machine learning algorithms","volume":"25","author":"Snoek Jasper","year":"2012","unstructured":"Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. 2012. Practical bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems 25 (2012).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_84_2","first-page":"153","volume-title":"Proceedings of the 12th European Dependable Computing Conference (EDCC)","author":"Sotiropoulos Thierry","year":"2016","unstructured":"Thierry Sotiropoulos, Guiochet Guiochet, Ingrand Ingrand, and Weaselynck Waeselynck. 2016. Virtual worlds for testing robot navigation: A study on the difficulty level. In Proceedings of the 12th European Dependable Computing Conference (EDCC). IEEE, 153\u2013160."},{"key":"e_1_3_2_85_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press."},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386109"},{"key":"e_1_3_2_87_2","first-page":"1555","volume-title":"Proceedings of the IEEE Intelligent Vehicles Symposium (IV)","author":"Tuncali Cumhur Erkan","year":"2018","unstructured":"Cumhur Erkan Tuncali, Georgios Fainekos, Hisahiro Ito, and James Kapinski. 2018. Simulation-based adversarial test generation for autonomous vehicles with machine learning components. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV). IEEE, 1555\u20131562."},{"key":"e_1_3_2_88_2","first-page":"113","volume-title":"Proceedings of the 35th Uncertainty in Artificial Intelligence Conference","author":"Wang Yuhui","year":"2020","unstructured":"Yuhui Wang, Hao He, and Xiaoyang Tan. 2020. Truly proximal policy optimization. In Proceedings of the 35th Uncertainty in Artificial Intelligence Conference. PMLR, 113\u2013122."},{"key":"e_1_3_2_89_2","first-page":"43","volume-title":"Proceedings of the IEEE\/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","author":"Winsten Jesper","year":"2023","unstructured":"Jesper Winsten and Ivan Porres. 2023. WOGAN at the SBFT 2023 tool competition-cyber-physical systems track. In Proceedings of the IEEE\/ACM International Workshop on Search-Based and Fuzz Testing (SBFT). IEEE, 43\u201344."},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464811"},{"key":"e_1_3_2_91_2","volume-title":"IEEE Transactions on Software Engineering, 49","author":"Zolfagharian Amirhossein","year":"2023","unstructured":"Amirhossein Zolfagharian, Manel Abdellatif, Lionel C. Briand, Mojtaba Bagherzadeh, and S. Ramesh. 2023. A search-based testing approach for deep reinforcement learning agents. IEEE Transactions on Software Engineering, 49 (2023)."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3680468","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3680468","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:58:26Z","timestamp":1750294706000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3680468"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,30]]},"references-count":90,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2024,11,30]]}},"alternative-id":["10.1145\/3680468"],"URL":"https:\/\/doi.org\/10.1145\/3680468","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,30]]},"assertion":[{"value":"2023-08-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-26","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}