{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T21:41:53Z","timestamp":1779313313606,"version":"3.51.4"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2025,4,4]],"date-time":"2025-04-04T00:00:00Z","timestamp":1743724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100004359","name":"Swedish Research Council","doi-asserted-by":"crossref","award":["SyTeC VR 2016-06204"],"award-info":[{"award-number":["SyTeC VR 2016-06204"]}],"id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001858","name":"Swedish Governmental Agency for Innovation Systems","doi-asserted-by":"crossref","award":["TESTRON 2015-04893"],"award-info":[{"award-number":["TESTRON 2015-04893"]}],"id":[{"id":"10.13039\/501100001858","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Wallenberg AI, Autonomous Systems, and Software program (WASP) funded by the Knut and Alice Wallenberg Foundation"},{"DOI":"10.13039\/501100004359","name":"Swedish Research Council","doi-asserted-by":"crossref","award":["2022-06725"],"award-info":[{"award-number":["2022-06725"]}],"id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2025,5,31]]},"abstract":"<jats:p>Cyber-physical systems (CPSs) are often complex and safety-critical, making it both challenging and crucial to ensure that the system\u2019s specifications are met. Simulation-based falsification is a practical testing technique for increasing confidence in a CPS\u2019s correctness, as it only requires that the system be simulated. Reducing the number of computationally intensive simulations needed for falsification is a key concern. In this study, we investigate Bayesian optimization (BO), a sample-efficient approach that learns a surrogate model to capture the relationship between input signal parameterization and specification evaluation. We propose two enhancements to the basic BO for improving falsification: (1) leveraging local surrogate models, and (2) utilizing the user\u2019s prior knowledge. Additionally, we address the formulation of acquisition functions for falsification by proposing and evaluating various alternatives. Our benchmark evaluation demonstrates significant improvements when using local surrogate models in BO for falsifying challenging benchmark examples. Incorporating prior knowledge is found to be especially beneficial when the simulation budget is constrained. For some benchmark problems, the choice of acquisition function noticeably impacts the number of simulations required for successful falsification.<\/jats:p>","DOI":"10.1145\/3711922","type":"journal-article","created":{"date-parts":[[2025,2,27]],"date-time":"2025-02-27T14:44:51Z","timestamp":1740667491000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Falsification of Cyber-physical Systems Using Bayesian Optimization"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2131-2276","authenticated-orcid":false,"given":"Zahra","family":"Ramezani","sequence":"first","affiliation":[{"name":"Electrical Engineering, Chalmers University of Technology, Goteborg, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3660-1428","authenticated-orcid":false,"given":"Kenan","family":"\u0160ehi\u0107","sequence":"additional","affiliation":[{"name":"Lund University, Sweden, Sweden and University of Sarajevo, Sarajevo, Bosnia and Herzegovina"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4601-2264","authenticated-orcid":false,"given":"Luigi","family":"Nardi","sequence":"additional","affiliation":[{"name":"Lund University, Sweden, Sweden and Stanford University, Stanford, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6105-2726","authenticated-orcid":false,"given":"Knut","family":"\u00c5kesson","sequence":"additional","affiliation":[{"name":"Electrical Engineering, Chalmers University of Technology, Goteborg, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,4,4]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46982-9_27"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.5555\/2774947"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-68913-5"},{"key":"e_1_3_3_5_2","first-page":"1","article-title":"Bayesian optimization with safety constraints: Safe and automatic parameter tuning in robotics","author":"Berkenkamp Felix","year":"2021","unstructured":"Felix Berkenkamp, Andreas Krause, and Angela P. Schoellig. 2021. Bayesian optimization with safety constraints: Safe and automatic parameter tuning in robotics. Machine Learning (2021), 1\u201335.","journal-title":"Machine Learning"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ifacol.2018.06.333"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30494-4_3"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3126521"},{"key":"e_1_3_3_9_2","volume-title":"ARCH@ADHS","author":"Dokhanchi Adel","year":"2018","unstructured":"Adel Dokhanchi, Shakiba Yaghoubi, Bardh Hoxha, Georgios Fainekos, Gidon Ernst, Zhenya Zhang, Paolo Arcaini, Ichiro Hasuo, and Sean Sedwards. 2018. ARCH-COMP18 category report: Results on the falsification benchmarks. In ARCH@ADHS."},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.5555\/2830865.2830869"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14295-6_17"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15297-9_9"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.strusafe.2011.01.002"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.29007\/f4vs"},{"key":"e_1_3_3_15_2","first-page":"5496","article-title":"Scalable global optimization via local Bayesian optimization","volume":"32","author":"Eriksson David","year":"2019","unstructured":"David Eriksson, Michael Pearce, Jacob Gardner, Ryan D. Turner, and Matthias Poloczek. 2019. Scalable global optimization via local Bayesian optimization. Advances in Neural Information Processing Systems 32 (2019), 5496\u20135507.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_16_2","first-page":"129","volume-title":"6th International Workshop on Applied Verification of Continuous and Hybrid Systems (ARCH19","author":"Ernst Gidon","year":"2019","unstructured":"Gidon Ernst, Paolo Arcaini, Alexandre Donz\u00e9, Georgios Fainekos, Logan Mathesen, Giulia Pedrielli, Shakiba Yaghoubi, Yoriyuki Yamagata, and Zhenya Zhang. 2019. ARCH-COMP 2019 category report: Falsification. In 6th International Workshop on Applied Verification of Continuous and Hybrid Systems (ARCH19)Vol. 61. EasyChair, 129\u2013140."},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tcs.2009.06.021"},{"key":"e_1_3_3_18_2","article-title":"A tutorial on Bayesian optimization","author":"Frazier P. I.","year":"2018","unstructured":"P. I. Frazier. 2018. A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811 (2018).","journal-title":"arXiv preprint arXiv:1807.02811"},{"key":"e_1_3_3_19_2","first-page":"1","volume-title":"5th International Workshop on Applied Verification of Continuous and Hybrid Systems","author":"Frehse Goran","year":"2018","unstructured":"Goran Frehse, Alessandro Abate, Dieky Adzkiya, Lei Bu, Mirco Giacobbe, Muhammad Syifa\u2019Ul Mufid, and Enea Zaffanella. 2018. ARCH-COMP18 category report: Hybrid systems with piecewise constant dynamics. In 5th International Workshop on Applied Verification of Continuous and Hybrid Systems, Vol. 54. EasyChair, 1\u201313."},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-32494-1_4"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/321062.321069"},{"key":"e_1_3_3_22_2","volume-title":"ARCH@CPSWeek","author":"Hoxha Bardh","year":"2014","unstructured":"Bardh Hoxha, Houssam Abbas, and Georgios Fainekos. 2014. Benchmarks for temporal logic requirements for automotive systems. In ARCH@CPSWeek."},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-46430-1_16"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/1377612.1377613"},{"key":"e_1_3_3_25_2","article-title":"\\(\\pi\\) BO: Augmenting acquisition functions with user beliefs for Bayesian optimization","author":"Hvarfner Carl","year":"2022","unstructured":"Carl Hvarfner, Danny Stoll, Artur Souza, Marius Lindauer, Frank Hutter, and Luigi Nardi. 2022. \\(\\pi\\) BO: Augmenting acquisition functions with user beliefs for Bayesian optimization. arXiv preprint arXiv:2204.11051 (2022).","journal-title":"arXiv preprint arXiv:2204.11051"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2562059.2562140"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1115\/1.3653121"},{"key":"e_1_3_3_28_2","first-page":"1546","volume-title":"Advances in Neural Information Processing Systems 33 (NeurIPS\u201920)","author":"Letham B.","year":"2020","unstructured":"B. Letham, R. Calandra, A. Rai, and E. Bakshy. 2020. Re-examining linear embeddings for high-dimensional Bayesian optimization. In Advances in Neural Information Processing Systems 33 (NeurIPS\u201920), Vol. 33. 1546\u20131558."},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30206-3_12"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/CASE49439.2021.9551474"},{"key":"e_1_3_3_31_2","article-title":"Design NARMA-L2 Neural Controller in Simulink","year":"2020","unstructured":"MathWorks. 2020. Design NARMA-L2 Neural Controller in Simulink. https:\/\/au.mathworks.com\/help\/deeplearning\/ug\/design-narma-l2-neural-controller-in-simulink.htmlOnline: accessed 1 March 2021.","journal-title":"https:\/\/au.mathworks.com\/help\/deeplearning\/ug\/design-narma-l2-neural-controller-in-simulink.html"},{"key":"e_1_3_3_32_2","article-title":"Skill-based multi-objective reinforcement learning of industrial robot tasks with planning and knowledge integration","author":"Mayr Matthias","year":"2022","unstructured":"Matthias Mayr, Faseeh Ahmad, Konstantinos Chatzilygeroudis, Luigi Nardi, and Volker Krueger. 2022. Skill-based multi-objective reinforcement learning of industrial robot tasks with planning and knowledge integration. arXiv preprint arXiv:2203.10033 (2022).","journal-title":"arXiv preprint arXiv:2203.10033"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/MASCOTS.2019.00045"},{"key":"e_1_3_3_34_2","first-page":"4752","volume-title":"36th International Conference on Machine Learning","author":"Nayebi A.","year":"2019","unstructured":"A. Nayebi, A. Munteanu, and M. Poloczek. 2019. A framework for Bayesian optimization in embedded subspaces. In 36th International Conference on Machine Learning. 4752\u20134761."},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/7.4.308"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2021.3110740"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.29007\/q4k7"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ifacol.2021.04.040"},{"key":"e_1_3_3_39_2","volume-title":"Gaussian Processes for Machine Learning","author":"Rasmussen C. E.","year":"2006","unstructured":"C. E. Rasmussen and C. K. I. Williams. 2006. Gaussian Processes for Machine Learning. MIT Press. 248 pages."},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF01100688"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1061\/AJRUA6.0000870"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.29007\/tf1p"},{"key":"e_1_3_3_43_2","article-title":"LassoBench: A high-dimensional hyperparameter optimization benchmark suite for lasso","author":"\u0160ehi\u0107 Kenan","year":"2021","unstructured":"Kenan \u0160ehi\u0107, Alexandre Gramfort, Joseph Salmon, and Luigi Nardi. 2021. LassoBench: A high-dimensional hyperparameter optimization benchmark suite for lasso. arXiv preprint arXiv:2111.02790 (2021).","journal-title":"arXiv preprint arXiv:2111.02790"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2015.2494218"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-66845-1_1"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-86523-8_17"},{"key":"e_1_3_3_47_2","series-title":"ICML\u201910","first-page":"1015","volume-title":"Proceedings of the 27th International Conference on International Conference on Machine Learning","author":"Srinivas Niranjan","year":"2010","unstructured":"Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. 2010. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th International Conference on International Conference on Machine Learning (Haifa, Israel) (ICML\u201910). Omnipress, Madison, WI, USA, 1015\u20131022."},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.2307\/2332286"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.4806"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3302504.3311814"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3302504.3311814"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780198505143.003.0023"}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3711922","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3711922","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:10Z","timestamp":1750295890000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3711922"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,4]]},"references-count":51,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,5,31]]}},"alternative-id":["10.1145\/3711922"],"URL":"https:\/\/doi.org\/10.1145\/3711922","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,4]]},"assertion":[{"value":"2022-09-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-23","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}