{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T10:42:13Z","timestamp":1756464133960,"version":"3.41.2"},"reference-count":27,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T00:00:00Z","timestamp":1722556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["101017258"],"award-info":[{"award-number":["101017258"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Few mobile robot developers already test their software on simulated robots in virtual environments or sceneries. However, the majority still shy away from simulation-based test campaigns because it remains challenging to specify and execute suitable testing <jats:italic>scenarios<\/jats:italic>, that is, models of the environment <jats:italic>and<\/jats:italic> the robots\u2019 tasks. Through developer interviews, we identified that managing the enormous variability of testing scenarios is a major barrier to the application of simulation-based testing in robotics. Furthermore, traditional CAD or 3D-modelling tools such as SolidWorks, 3ds Max, or Blender are not suitable for specifying sceneries that vary significantly and serve different testing objectives. For some testing campaigns, it is required that the scenery replicates the dynamic (e.g., opening doors) and static features of real-world environments, whereas for others, simplified scenery is sufficient. Similarly, the task and mission specifications used for simulation-based testing range from simple point-to-point navigation tasks to more elaborate tasks that require advanced deliberation and decision-making. We propose the concept of <jats:italic>composable and executable scenarios<\/jats:italic> and associated tooling to support developers in specifying, reusing, and executing scenarios for the simulation-based testing of robotic systems. Our approach differs from traditional approaches in that it offers a means of creating scenarios that allow the addition of new semantics (e.g., dynamic elements such as doors or varying task specifications) to existing models without altering them. Thus, we can systematically construct richer scenarios that remain manageable. We evaluated our approach in a small simulation-based testing campaign, with scenarios defined around the navigation stack of a mobile robot. The scenarios gradually increased in complexity, composing new features into the scenery of previous scenarios. Our evaluation demonstrated how our approach can facilitate the reuse of models and revealed the presence of errors in the configuration of the publicly available navigation stack of our SUT, which had gone unnoticed despite its frequent use.<\/jats:p>","DOI":"10.3389\/frobt.2024.1363281","type":"journal-article","created":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T04:10:59Z","timestamp":1722571859000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Composable and executable scenarios for simulation-based testing of mobile robots"],"prefix":"10.3389","volume":"11","author":[{"given":"Argentina","family":"Ortega","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samuel","family":"Parra","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sven","family":"Schneider","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nico","family":"Hochgeschwender","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,8,2]]},"reference":[{"key":"B1","first-page":"90","article-title":"A study on challenges of testing robotic systems","author":"Afzal","year":"2020"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2104.08625","article-title":"GzScenic: automatic scene generation for gazebo simulator","author":"Afzal","year":""},{"key":"B3","first-page":"263","article-title":"Simulation for robotics test automation: developer perspectives","author":"Afzal","year":""},{"volume-title":"ASAM OpenSCENARIO standard","year":"2022","key":"B4"},{"key":"B5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/978-3-319-92862-3_1","article-title":"Building information modeling: why? What? How?","volume-title":"Building information modeling: Technology foundations and industry practice","author":"Borrmann","year":"2018"},{"key":"B6","doi-asserted-by":"publisher","first-page":"753","DOI":"10.1007\/s00371-015-1102-2","article-title":"Procedural floor plan generation from building sketches","volume":"31","author":"Camozzato","year":"2015","journal-title":"Vis. Comput."},{"key":"B7","first-page":"12625","article-title":"Intelligent home 3D: automatic 3D-house design from linguistic descriptions only","volume-title":"Proc. Of the IEEE\/CVF conf","author":"Chen","year":"2020"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.knosys.2016.10.023","article-title":"TextX: a Python tool for domain-specific languages implementation","volume":"115","author":"Dejanovi\u0107","year":"2017","journal-title":"Knowledge-Based Syst."},{"key":"B9","first-page":"63","article-title":"Scenic: a Language for scenario specification and scene generation","author":"Fremont","year":"2019"},{"key":"B10","doi-asserted-by":"publisher","first-page":"1305","DOI":"10.1109\/IV47402.2020.9304678","article-title":"Re-Using concrete test scenarios generally is a bad idea","author":"Hauer","year":"2020","journal-title":"IEEE Intell. Veh. Symp."},{"key":"B11","first-page":"11654","article-title":"Connecting semantic building information models and robotics: an application to 2d lidar-based localization","author":"Hendrikx","year":"2021"},{"key":"B12","first-page":"10","article-title":"Experiences from conducting semi-structured interviews in empirical software engineering research","author":"Hove","year":"2005"},{"key":"B13","article-title":"Feature-oriented domain analysis (FODA) feasibility study","author":"Kang","year":"1990","journal-title":"Tech. Rep. CMU\/SEI-90-TR-021"},{"key":"B14","doi-asserted-by":"publisher","first-page":"2149","DOI":"10.1109\/IROS.2004.1389727","article-title":"Design and use paradigms for gazebo, an open-source multi-robot simulator","volume":"3","author":"Koenig","year":"2004","journal-title":"IEEE\/RSJ Intl. Conf. Intell. Robots Syst. (IROS)"},{"key":"B15","first-page":"87","article-title":"Component-based modeling of complete buildings","volume":"2011","author":"Leblanc","year":"2011","journal-title":"Graph. Interface"},{"key":"B16","first-page":"13","article-title":"A constrained growth method for procedural floor plan generation","volume-title":"Proc. Of the int. Conf. Intell. Games simul","author":"Lopes","year":"2010"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1211.5842","article-title":"A novel algorithm for real-time procedural generation of building floor plans","author":"Mirahmadi","year":"2012"},{"key":"B18","first-page":"3839","article-title":"Semantic mapping extension for OpenStreetMap applied to indoor robot navigation","author":"Naik","year":"2019"},{"key":"B19","first-page":"165","article-title":"Testing service robots in the field: an experience report","author":"Ortega","year":"2022"},{"key":"B20","first-page":"6690","article-title":"Generative layout modeling using constraint graphs","author":"Para","year":"2021"},{"key":"B21","first-page":"5537","article-title":"A thousand worlds: scenery specification and generation for simulation-based testing of mobile robot navigation stacks","volume-title":"IEEE\/RSJ Intl. Conf. On intell. Robots and syst. (IROS)","author":"Parra","year":"2023"},{"key":"B22","first-page":"754","article-title":"A survey of autonomous driving scenarios and scenario databases","author":"Ren","year":"2022"},{"key":"B23","first-page":"1","article-title":"The coding manual for qualitative researchers","author":"Salda\u00f1a","year":"2021","journal-title":"Coding Man. Qual. Res."},{"key":"B24","first-page":"9104","article-title":"Domain-specific languages for kinematic chains and their solver algorithms: lessons learned for composable models","author":"Schneider","year":"2023"},{"key":"B25","first-page":"150","article-title":"Can robot navigation bugs Be found in simulation? An exploratory study","author":"Sotiropoulos","year":"2017"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3579642","article-title":"A survey on automated driving system testing: Landscapes and trends","volume":"32","author":"Tang","year":"2023","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"B27","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1109\/ICST.2018.00040","article-title":"Crashing simulated planes is cheap: can simulation detect robotics bugs early?","author":"Timperley","year":"2018","journal-title":"IEEE Intl. Conf. Softw. Test. Verification Validation (ICST)"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1363281\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T04:11:08Z","timestamp":1722571868000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1363281\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,2]]},"references-count":27,"alternative-id":["10.3389\/frobt.2024.1363281"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2024.1363281","relation":{},"ISSN":["2296-9144"],"issn-type":[{"type":"electronic","value":"2296-9144"}],"subject":[],"published":{"date-parts":[[2024,8,2]]},"article-number":"1363281"}}