{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T21:43:01Z","timestamp":1788212581741,"version":"build-2803163510"},"reference-count":85,"publisher":"Association for Computing Machinery (ACM)","issue":"ISSTA","license":[{"start":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T00:00:00Z","timestamp":1750550400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,22]]},"abstract":"<jats:p>Autonomous surface vehicles (ASVs) need to complete missions without posing risks to other maritime traffic. Safe traffic in open sea encounters is controlled by the International Regulations for Preventing Collisions at Sea (COLREGS) formulated by the International Maritime Organization (IMO). Designed with human operators in mind, the COLREGS are intentionally underspecified, which may result in ambiguous requirements for correct behaviour for ASVs. Hence the systematic testing of such ambiguous situations is particularly important. This paper investigates to what extent existing test scenario generation approaches deemed effective in the automotive domain can be adapted to test COLREGS-compliance in a maritime context with multi-vessel encounters. In a series of experiments involving synthetic and real-world test scenarios, their performance is evaluated with respect to relevance, diversity, completeness, scalability and speed. Our results indicate that (1) test scenarios derived from historic maritime traffic are insufficient for testing multi-ship encounters. Moreover, (2) existing test scenario generation techniques provide sufficient scalability and speed, but they are very limited in terms of diversity and completeness when the number of vessels increases.<\/jats:p>","DOI":"10.1145\/3728919","type":"journal-article","created":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T10:52:56Z","timestamp":1750589576000},"page":"985-1008","source":"Crossref","is-referenced-by-count":4,"title":["Assessing Scene Generation Techniques for Testing COLREGS-Compliance of Autonomous Surface Vehicles"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-7174-4209","authenticated-orcid":false,"given":"Dominik","family":"Frey","sequence":"first","affiliation":[{"name":"Link\u00f6ping University, Link\u00f6ping, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3009-4314","authenticated-orcid":false,"given":"Ulf","family":"Karg\u00e9n","sequence":"additional","affiliation":[{"name":"Link\u00f6ping University, Link\u00f6ping, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8790-252X","authenticated-orcid":false,"given":"D\u00e1niel","family":"Varr\u00f3","sequence":"additional","affiliation":[{"name":"Link\u00f6ping University, Link\u00f6ping, Sweden"},{"name":"McGill University, Montreal, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,22]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238192"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.11648\/j.ijse.20200402.12"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/tse.2023.3331254"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ivs.2018.8500632"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/tits.2021.3095547"},{"key":"e_1_3_1_7_2","first-page":"63","volume-title":"Proceedings of the 31st IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201916) (Singapore, Singapore)","author":"Abdessalem Raja Ben","year":"2016","unstructured":"Raja Ben Abdessalem, Shiva Nejati, Lionel C. Briand, and Thomas Stifter. 2016. Testing advanced driver assistance systems using multi-objective search and neural networks. In Proceedings of the 31st IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201916) (Singapore, Singapore). Association for Computing Machinery, New York, NY, USA, 63\u201374."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/2970276.2970311"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180160"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","unstructured":"G\u00e1bor Bergmann Zolt\u00e1n Ujhelyi Istv\u00e1n R\u00e1th and D\u00e1niel Varr\u00f3. 2011. A Graph Query Language for EMF Models. Springer Berlin Heidelberg 167\u2013182. doi:10.1007\/978-3-642-21732-6_12","DOI":"10.1007\/978-3-642-21732-6_12"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2015.07.002"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2990567"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2022.111309"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/icst46399.2020.00045"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","unstructured":"A.N. Cockcroft and J.N.F. Lameijer. 2012. Steering and sailing rules. Elsevier 11\u2013104. doi:10.1016\/b978-0-08-097170-4.00002-7","DOI":"10.1016\/b978-0-08-097170-4.00002-7"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/c2010-0-68322-2"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/s0045-7825(01)00323-1"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.4324\/9780203771587"},{"issue":"3","key":"e_1_3_1_19_2","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1017\/S0373463300039783","article-title":"Marine traffic behaviour in restricted waters","volume":"36","author":"Coldwell TG","year":"1983","unstructured":"TG Coldwell. 1983. Marine traffic behaviour in restricted waters. The Journal of Navigation 36, 3 (1983), 430\u2013444.","journal-title":"The Journal of Navigation"},{"key":"e_1_3_1_20_2","unstructured":"UCG Commandant. 1999. International regulations for prevention of collisions at sea 1972 (72 COLREGS). US Department of Transportation US Coast Guard Commandant Instruction M 16672 (1999)."},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TFR.2024.3457273"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/4235.996017"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/1276958.1277190"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1017\/s0373463300021408"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jocs.2022.101937"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","unstructured":"A. E. Eiben and S. K. Smit. 2011. Evolutionary Algorithm Parameters and Methods to Tune Them. Springer Berlin Heidelberg 15\u201336. doi:10.1007\/978-3-642-21434-9_2","DOI":"10.1007\/978-3-642-21434-9_2"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1177\/027836499801700706"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3314221.3314633"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ase.2019.00013"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","unstructured":"Denis Gopan Frank DiMaio Nurit Dor Thomas Reps and Mooly Sagiv. 2004. Numeric Domains with Summarized Dimensions. Springer Berlin Heidelberg 512\u2013529. doi:10.1007\/978-3-540-24730-2_38","DOI":"10.1007\/978-3-540-24730-2_38"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781410612915"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11334-024-00567-0"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1080\/23311916.2018.1502242"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00773-024-01008-0"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510188"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2017.05.029"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/itsc.2015.327"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/bf02764347"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jai.2022.100003"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2018.01.001"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.3390\/jmse9121458"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3640310.3674098"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/iv48863.2021.9575685"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/itsc55140.2022.9921925"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.12716\/1001.07.02"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729694"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.23919\/fusion49465.2021.9627027"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/JOE.2013.2254214"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730491"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3639478.3640045"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/tse.2020.3025732"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/ivs.2019.8814099"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","unstructured":"Seyedali Mirjalili. 2018. Genetic Algorithm. Springer International Publishing 43\u201355. doi:10.1007\/978-3-319-93025-1_4","DOI":"10.1007\/978-3-319-93025-1_4"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.mechatronics.2011.09.012"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","unstructured":"Matthew O\u2019Kelly Aman Sinha Hongseok Namkoong John Duchi and Russ Tedrake. 2018. Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation. doi:10.48550\/ARXIV.1811.00145","DOI":"10.48550\/ARXIV.1811.00145"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/icst.2015.7102604"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/tse.2017.2663435"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.3390\/s21216959"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1080\/14786440009463897"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ssci.2020.104799"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2020.108423"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/seaa51224.2020.00090"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.entcs.2006.01.022"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409730"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.5555\/645512.657233"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3365438.3410962"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10009-019-00530-6"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1186\/2190-8532-1-4"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/oceanse.2019.8867549"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2017.09.020"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1177\/1748006x211069277"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2015.164"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/2311\/1\/012026"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00773-020-00787-6"},{"key":"e_1_3_1_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2021.3070634"},{"key":"e_1_3_1_76_2","doi-asserted-by":"publisher","unstructured":"Eric Vin Shun Kashiwa Matthew Rhea Daniel J. Fremont Edward Kim Tommaso Dreossi Shromona Ghosh Xiangyu Yue Alberto L. Sangiovanni-Vincentelli and Sanjit A. Seshia. 2023. 3D Environment Modeling for Falsification and Beyond with Scenic 3.0. Springer Nature Switzerland 253\u2013265. doi:10.1007\/978-3-031-37706-8_13","DOI":"10.1007\/978-3-031-37706-8_13"},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2023.116436"},{"key":"e_1_3_1_78_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0265681"},{"key":"e_1_3_1_79_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-018-9765-y"},{"key":"e_1_3_1_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/oceans.2016.7761413"},{"key":"e_1_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.7307\/ptt.v26i6.1386"},{"key":"e_1_3_1_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/cvpr52733.2024.01464"},{"key":"e_1_3_1_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/itsc.2018.8569800"},{"key":"e_1_3_1_84_2","doi-asserted-by":"publisher","unstructured":"Xiaoxue Zhang Jun Ma Zilong Cheng Masayoshi Tomizuka and Tong Heng Lee. 2022. Velocity Obstacle Based Risk-Bounded Motion Planning for Stochastic Multi-Agent Systems. doi:10.48550\/ARXIV.2202.09748","DOI":"10.48550\/ARXIV.2202.09748"},{"key":"e_1_3_1_85_2","doi-asserted-by":"publisher","DOI":"10.3390\/app12188926"},{"key":"e_1_3_1_86_2","doi-asserted-by":"publisher","DOI":"10.3390\/jmse10111588"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728919","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728919","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T13:54:23Z","timestamp":1787493263000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3728919"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,22]]},"references-count":85,"journal-issue":{"issue":"ISSTA","published-print":{"date-parts":[[2025,6,22]]}},"alternative-id":["10.1145\/3728919"],"URL":"https:\/\/doi.org\/10.1145\/3728919","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,22]]}}}