{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T11:34:37Z","timestamp":1791459277966,"version":"4.3.4"},"reference-count":101,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T00:00:00Z","timestamp":1711152000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T00:00:00Z","timestamp":1711152000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100008047","name":"Carnegie Mellon University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100008047","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2024,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>As robotic systems such as autonomous cars and delivery drones assume greater roles and responsibilities within society, the likelihood and impact of catastrophic software failure within those systems is increased. To aid researchers in the development of new methods to measure and assure the safety and quality of robotics software, we systematically curated a dataset of 221 bugs across 7 popular and diverse software systems implemented via the Robot Operating System (ROS). We produce historically accurate recreations of each of the 221 defective software versions in the form of Docker images, and use a grounded theory approach to examine and categorize their corresponding faults, failures, and fixes. Finally, we reflect on the implications of our findings and outline future research directions for the community.<\/jats:p>","DOI":"10.1007\/s10664-024-10440-0","type":"journal-article","created":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T11:03:04Z","timestamp":1711191784000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["ROBUST: 221 bugs in the Robot Operating System"],"prefix":"10.1007","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9785-324X","authenticated-orcid":false,"given":"Christopher S.","family":"Timperley","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gijs","family":"van der Hoorn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andr\u00e9","family":"Santos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Harshavardhan","family":"Deshpande","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrzej","family":"W\u0105sowski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,3,23]]},"reference":[{"key":"10440_CR1","doi-asserted-by":"crossref","unstructured":"Abal I, Brabrand C, W\u0105sowski A (2014) 42 Variability Bugs in the Linux Kernel: A Qualitative Analysis. In: International conference on automated software engineering, ASE \u201914, pp 421\u2013432","DOI":"10.1145\/2642937.2642990"},{"key":"10440_CR2","doi-asserted-by":"crossref","unstructured":"Abal I, Melo J, St\u0103nciulescu c, Brabrand C, Ribeiro M, W\u0105sowski A (2018) Variability Bugs in Highly Configurable Systems: A Qualitative Analysis. ACM Trans Softw Eng Methodol 26(3)","DOI":"10.1145\/3149119"},{"key":"10440_CR3","doi-asserted-by":"crossref","unstructured":"Afzal A, Le Goues C, Hilton M, Timperley CS (2020) A study on challenges of testing robotic systems. In: International conference on software testing, validation and verification, ICST \u201920, pp 96\u2013107","DOI":"10.1109\/ICST46399.2020.00020"},{"key":"10440_CR4","unstructured":"Afzal A, Le Goues C, Timperley CS (2021a) GzScenic: Automatic Scene Generation for Gazebo Simulator. 2104.08625"},{"key":"10440_CR5","doi-asserted-by":"publisher","unstructured":"Afzal A, Le Goues C, Timperley CS (2021) Mithra: Anomaly detection as an oracle for cyberphysical systems. IEEE Trans Softw Eng 1\u20131. https:\/\/doi.org\/10.1109\/TSE.2021.3120680","DOI":"10.1109\/TSE.2021.3120680"},{"key":"10440_CR6","doi-asserted-by":"crossref","unstructured":"Alami A, Dittrich Y, Wasowski A (2018) Influencers of quality assurance in an open source community. In: International workshop on cooperative and human aspects of software engineering, CHASE \u201918, pp 61\u201368","DOI":"10.1145\/3195836.3195853"},{"key":"10440_CR7","doi-asserted-by":"crossref","unstructured":"Aliabadi MR, Kamath AA, Gascon-Samson J, Pattabiraman K (2017) ARTINALI: dynamic invariant detection for cyber-physical system security. In: Joint meeting on European software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE \u201917, pp 349\u2013361","DOI":"10.1145\/3106237.3106282"},{"key":"10440_CR8","unstructured":"ArduPilot (2023) ArduPilot. https:\/\/ardupilot.org"},{"issue":"1","key":"10440_CR9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13174-016-0052-x","volume":"8","author":"SA Asadollah","year":"2017","unstructured":"Asadollah SA, Sundmark D, Eldh S, Hansson H (2017) Concurrency bugs in open source software: a case study. J Internet Serv Appl 8(1):1\u201315","journal-title":"J Internet Serv Appl"},{"key":"10440_CR10","unstructured":"Atwood J (2012) New programming jargon. https:\/\/blog.codinghorror.com\/new-programming-jargon"},{"key":"10440_CR11","unstructured":"Baidu (2021) Apollo. https:\/\/github.com\/ApolloAuto\/apollo"},{"key":"10440_CR12","doi-asserted-by":"crossref","unstructured":"Bardaro G, Semprebon A, Matteucci M (2018) A Use Case in Model-Based Robot Development Using AADL and ROS. In: International workshop on robotics software engineering, RoSE \u201918, pp 9\u201316","DOI":"10.1145\/3196558.3196560"},{"key":"10440_CR13","doi-asserted-by":"crossref","unstructured":"Benton S, Ghanbari A, Zhang L (2019) Defexts: A curated dataset of reproducible real-world bugs for modern jvm languages. In: International conference on software engineering: companion proceedings, ICSE \u201919, pp 47\u201350","DOI":"10.1109\/ICSE-Companion.2019.00035"},{"key":"10440_CR14","doi-asserted-by":"crossref","unstructured":"Bhattacharya P, Ulanova L, Neamtiu I, Koduru SC (2013) An empirical analysis of bug reports and bug fixing in open source Android apps. In: European conference on software maintenance and reengineering, IEEE, pp 133\u2013143","DOI":"10.1109\/CSMR.2013.23"},{"key":"10440_CR15","doi-asserted-by":"crossref","unstructured":"B\u00f6hme M, Roychoudhury A (2014) Corebench: Studying complexity of regression errors. In: International symposium on software testing and analysis, ISSTA \u201914, pp 105\u2013115","DOI":"10.1145\/2610384.2628058"},{"key":"10440_CR16","doi-asserted-by":"crossref","unstructured":"B\u00f6hme M, Soremekun EO, Chattopadhyay S, Ugherughe E, Zeller A (2017) Where is the bug and how is it fixed? an experiment with practitioners. In: Joint meeting of the european software engineering conference and the symposium on the foundations of software engineering, ESEC\/FSE \u201917, pp 1\u201311","DOI":"10.1145\/3106237.3106255"},{"key":"10440_CR17","doi-asserted-by":"crossref","unstructured":"Bozhinoski D, Aguado E, Oviedo MG, Hernandez C, Sanz R, W\u0105sowski A (2021) A Modeling Tool for Reconfigurable Skills in ROS. In: International workshop on robotics software engineering, RoSE \u201921, pp 25\u201328","DOI":"10.1109\/RoSE52553.2021.00011"},{"key":"10440_CR18","unstructured":"Charette RN (2014) Nissan recalls nearly 1 million cars for air bag software fix. IEEE Spectrum. https:\/\/spectrum.ieee.org\/riskfactor\/transportation\/safety\/nissan-recalls-nearly-1-million-cars-for-airbag-software-fix"},{"key":"10440_CR19","doi-asserted-by":"crossref","unstructured":"Chou A, Yang J, Chelf B, Hallem S, Engler D (2001) An empirical study of operating systems errors. In: Symposium on operating systems principles, pp 73\u201388","DOI":"10.1145\/502059.502042"},{"key":"10440_CR20","unstructured":"Coghlan N, Stufft D (2013) PEP 440 \u2013 Version Identification and Dependency Specification. https:\/\/www.python.org\/dev\/peps\/pep-0440. Accessed 16 June 2021"},{"key":"10440_CR21","doi-asserted-by":"crossref","unstructured":"Di Franco A, Guo H, Rubio-Gonz\u00e1lez C (2017) A comprehensive study of real-world numerical bug characteristics. In: International conference on automated software engineering, ASE \u201917, pp 509\u2013519","DOI":"10.1109\/ASE.2017.8115662"},{"issue":"4","key":"10440_CR22","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1007\/s10664-005-3861-2","volume":"10","author":"H Do","year":"2005","unstructured":"Do H, Elbaum S, Rothermel G (2005) Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact. Empir Softw Eng 10(4):405\u2013435","journal-title":"Empir Softw Eng"},{"key":"10440_CR23","unstructured":"Dronecode Foundation (2023) PX4. https:\/\/px4.io"},{"issue":"2","key":"10440_CR24","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1145\/857076.857078","volume":"35","author":"PT Eugster","year":"2003","unstructured":"Eugster PT, Felber PA, Guerraoui R, Kermarrec AM (2003) The many faces of publish\/subscribe. ACM Comput Surv 35(2):114\u2013131","journal-title":"ACM Comput Surv"},{"key":"10440_CR25","unstructured":"Facebook (2022) Infer. https:\/\/fbinfer.com. Accessed 22 July 2022"},{"key":"10440_CR26","doi-asserted-by":"crossref","unstructured":"Fischer-Nielsen A, Fu Z, Su T, W\u0105sowski A (2020) The forgotten case of the dependency bugs: on the example of the Robot Operating System. In: International conference on software engineering, software engineering in practice, ICSE-SEIP \u201920, pp 21\u201330","DOI":"10.1145\/3377813.3381364"},{"key":"10440_CR27","doi-asserted-by":"crossref","unstructured":"Fonseca P, Li C, Singhal V, Rodrigues R (2010) A study of the internal and external effects of concurrency bugs. In: International conference on dependable systems & networks, DSN \u201910, pp 221\u2013230","DOI":"10.1109\/DSN.2010.5544315"},{"key":"10440_CR28","unstructured":"Foote T (2021) Parameter api design in ros. http:\/\/design.ros2.org\/articles\/ros_parameters.html. Accessed 26 May 2021"},{"key":"10440_CR29","doi-asserted-by":"crossref","unstructured":"Fremont DJ, Dreossi T, Ghosh S, Yue X, Sangiovanni-Vincentelli AL, Seshia SA (2019) Scenic: a language for scenario specification and scene generation. In: Conference on programming language design and implementation, PLDI \u201919, pp 63\u201378","DOI":"10.1145\/3314221.3314633"},{"key":"10440_CR30","doi-asserted-by":"crossref","unstructured":"Garcia J, Feng Y, Shen J, Almanee S, Xia Y, Chen QA (2020) A comprehensive study of autonomous vehicle bugs. In: International conference on software engineering, pp 385\u2013396","DOI":"10.1145\/3377811.3380397"},{"key":"10440_CR31","doi-asserted-by":"crossref","unstructured":"Grottke M, Nikora AP, Trivedi KS (2010) An empirical investigation of fault types in space mission system software. In: International conference on dependable systems & networks, DSN \u201910, pp 447\u2013456","DOI":"10.1109\/DSN.2010.5544284"},{"key":"10440_CR32","doi-asserted-by":"crossref","unstructured":"Gyimesi P, Vancsics B, Stocco A, Mazinanian D, Besz\u00e9des A, Ferenc R, Mesbah A (2019) BugsJS: a benchmark of JavaScript bugs. In: International conference on software testing, validation and verification, ICST \u201919, pp 90\u2013101","DOI":"10.1109\/ICST.2019.00019"},{"key":"10440_CR33","doi-asserted-by":"crossref","unstructured":"Han X, Yu T (2016) An empirical study on performance bugs for highly configurable software systems. In: International symposium on empirical software engineering and measurement, ESEM \u201916, pp 1\u201310","DOI":"10.1145\/2961111.2962602"},{"key":"10440_CR34","doi-asserted-by":"crossref","unstructured":"He Z, Chen Y, Huang E, Wang Q, Pei Y, Yuan H (2019) A system identification based oracle for control-cps software fault localization. In: International conference on software engineering, ICSE \u201919, pp 116\u2013127","DOI":"10.1109\/ICSE.2019.00029"},{"key":"10440_CR35","doi-asserted-by":"publisher","unstructured":"Henning A (2016) Physical continuous integration \u2014 CI on real robots! In: ROSCon Seoul 2016, Open Robotics. https:\/\/doi.org\/10.36288\/ROSCon2016-900758. https:\/\/doi.org\/10.36288\/ROSCon2016-900758","DOI":"10.36288\/ROSCon2016-900758"},{"key":"10440_CR36","doi-asserted-by":"crossref","unstructured":"Hu Y, Ahmed UZ, Mechtaev S, Leong B, Roychoudhury A (2019) Re-factoring based Program Repair applied to Programming Assignments. In: International conference on automated software engineering, ASE \u201919, pp 388\u2013398","DOI":"10.1109\/ASE.2019.00044"},{"key":"10440_CR37","doi-asserted-by":"crossref","unstructured":"Humbatova N, Jahangirova G, Bavota G, Riccio V, Stocco A, Tonella P (2020) Taxonomy of real faults in deep learning systems. In: International conference on software engineering, ICSE \u201920, pp 1110\u20131121","DOI":"10.1145\/3377811.3380395"},{"key":"10440_CR38","doi-asserted-by":"crossref","unstructured":"Inoue J, Yamagata Y, Chen Y, Poskitt CM, Sun J (2017) Anomaly detection for a water treatment system using unsupervised machine learning. In: International conference on data mining workshops, ICDMW \u201917, pp 1058\u20131065","DOI":"10.1109\/ICDMW.2017.149"},{"key":"10440_CR39","doi-asserted-by":"crossref","unstructured":"Islam MJ, Nguyen G, Pan R, Rajan H (2019) A comprehensive study on deep learning bug characteristics. In: Joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE \u201919, pp 510\u2013520","DOI":"10.1145\/3338906.3338955"},{"issue":"6","key":"10440_CR40","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1145\/2345156.2254075","volume":"47","author":"G Jin","year":"2012","unstructured":"Jin G, Song L, Shi X, Scherpelz J, Lu S (2012) Understanding and detecting real-world performance bugs. ACM SIGPLAN Notices 47(6):77\u201388","journal-title":"ACM SIGPLAN Notices"},{"key":"10440_CR41","doi-asserted-by":"crossref","unstructured":"Just R, Jalali D, Ernst MD (2014) Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In: International symposium on software testing and analysis, ISSTA \u201914, pp 437\u2013440","DOI":"10.1145\/2610384.2628055"},{"key":"10440_CR42","doi-asserted-by":"crossref","unstructured":"Kate S, Ore JP, Zhang X, Elbaum S, Xu Z (2018) Phys: probabilistic physical unit assignment and inconsistency detection. In: Joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE \u201918, pp 563\u2013573","DOI":"10.1145\/3236024.3236035"},{"key":"10440_CR43","doi-asserted-by":"crossref","unstructured":"Kate S, Chinn M, Choi H, Zhang X, Elbaum S (2021) PHYSFRAME: Type Checking Physical Frames of Reference for Robotic Systems. In: Joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE \u201921, pp 45\u201356","DOI":"10.1145\/3468264.3468608"},{"key":"10440_CR44","unstructured":"Kim T, Kim CH, Rhee J, Fei F, Tu Z, Walkup G, Zhang X, Deng X, Xu D (2019) RVFUZZER: Finding input validation bugs in robotic vehicles through control-guided testing. In: USENIX Security Symposium, pp 425-442"},{"key":"10440_CR45","doi-asserted-by":"crossref","unstructured":"Kl\u00fcck F, Li Y, Nica M, Tao J, Wotawa F (2018) Using ontologies for test suites generation for automated and autonomous driving functions. In: International Symposium on Software Reliability Engineering Workshops, ISSREW \u201918, pp 118\u2013123","DOI":"10.1109\/ISSREW.2018.00-20"},{"key":"10440_CR46","doi-asserted-by":"crossref","unstructured":"Kolak S, Afzal A, Le Goues C, Hilton MC, Timperley CS (2020) It takes a village to build a robot: An empirical study of the ros ecosystem. Int Conf Softw Maint Evol 430\u2013440","DOI":"10.1109\/ICSME46990.2020.00048"},{"key":"10440_CR47","doi-asserted-by":"crossref","unstructured":"Larsen H, Hoorn Gvd, W\u0105sowski A (2021) Reactive Programming of Robots with RxROS. In: Robot Operating System (ROS), Springer, pp 55\u201383","DOI":"10.1007\/978-3-030-75472-3_2"},{"issue":"12","key":"10440_CR48","doi-asserted-by":"publisher","first-page":"1236","DOI":"10.1109\/TSE.2015.2454513","volume":"41","author":"C Le Goues","year":"2015","unstructured":"Le Goues C, Holtschulte N, Smith EK, Brun Y, Devanbu P, Forrest S, Weimer W (2015) The ManyBugs and IntroClass benchmarks for automated repair of C programs. Trans Softw Eng 41(12):1236\u20131256","journal-title":"Trans Softw Eng"},{"key":"10440_CR49","doi-asserted-by":"crossref","unstructured":"Lin D, Koppel J, Chen A, Solar-Lezama A (2017) QuixBugs: A multi-lingual program repair benchmark set based on the Quixey Challenge. In: International conference on systems, programming, languages, and applications: software for humanity, pp 55\u201356","DOI":"10.1145\/3135932.3135941"},{"key":"10440_CR50","unstructured":"Linneberg MS, Korsgaard S (2019) Coding qualitative data: A synthesis guiding the novice. Qual Res J"},{"key":"10440_CR51","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-90800-7","volume-title":"Software languages: Syntax, semantics, and metaprogramming","author":"R L\u00e4mmel","year":"2018","unstructured":"L\u00e4mmel R (2018) Software languages: Syntax, semantics, and metaprogramming. Springer"},{"key":"10440_CR52","doi-asserted-by":"crossref","unstructured":"Madeiral F, Urli S, Maia M, Monperrus M (2019) Bears: An Extensible Java Bug Benchmark for Automatic Program Repair Studies. In: International Conference on Software Analysis, Evolution and Reengineering, SANER \u201919, pp 468\u2013478","DOI":"10.1109\/SANER.2019.8667991"},{"key":"10440_CR53","unstructured":"Majumdar R, Mathur A, Pirron M, Stegner L, Zufferey D (2019) Paracosm: A language and tool for testing autonomous driving systems. arXiv:1902.01084"},{"key":"10440_CR54","doi-asserted-by":"crossref","unstructured":"Martinez J, Ruiz A, Garzo A, Keller T, Radermacher A, Tonetta S (2021) Modelling the Component-based Architecture and Safety Contracts of ArmAssist in Papyrus for Robotics. In: International Workshop on Robotics Software Engineering, RoSE \u201921, pp 13\u201318","DOI":"10.1109\/RoSE52553.2021.00009"},{"key":"10440_CR55","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1145\/2692956.2663188","volume":"34","author":"ND Matsakis","year":"2014","unstructured":"Matsakis ND, Klock FS II (2014) The Rust language. ACM SIGAda Ada Letters 34:103\u2013104","journal-title":"ACM SIGAda Ada Letters"},{"key":"10440_CR56","unstructured":"McCausland P (2019) Self-driving uber car that hit and killed woman did not recognize that pedestrians jaywalk. NBC News. https:\/\/www.nbcnews.com\/tech\/tech-news\/self-driving-uber-car-hit-killed-woman-did-not-recognize-n1079281"},{"key":"10440_CR57","doi-asserted-by":"crossref","unstructured":"O\u2019Kane S (2020) Boeing finds another software problem on the 737 max. The Verge. https:\/\/www.theverge.com\/2020\/2\/6\/21126364\/boeing-737-max-software-glitch-flaw-problem","DOI":"10.1016\/j.eng.2019.11.002"},{"key":"10440_CR58","doi-asserted-by":"crossref","unstructured":"Ore JP, Detweiler C, Elbaum S (2017) Phriky-units: a lightweight, annotationfree physical unit inconsistency detection tool. In: International Symposium on Software Testing and Analysis, ISSTA \u201917, pp 352\u2013355","DOI":"10.1145\/3092703.3098219"},{"key":"10440_CR59","doi-asserted-by":"crossref","unstructured":"Purandare R, Darsie J, Elbaum S, Dwyer MB (2012) Extracting conditional component dependence for distributed robotic systems. In: International Conference on Intelligent Robots and Systems, RSJ \u201912, pp 1533\u20131540","DOI":"10.1109\/IROS.2012.6385719"},{"key":"10440_CR60","unstructured":"Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) Ros: an open-source Robot Operating System. In: ICRA Workshop on Open Source Software"},{"key":"10440_CR61","unstructured":"ROS2 (2021) rclcpp: C++ ros client library api: Node class reference. https:\/\/docs.ros2.org\/latest\/api\/rclcpp\/classrclcpp_1_1Node.html#a12d535bced9f26b65c0a450e6f40aff8. Accessed 26 May 2021"},{"key":"10440_CR62","unstructured":"ROSIn (2022) ROS-Industrial: Current Members. https:\/\/web.archive.org\/web\/20220723013322\/https:\/\/rosindustrial.org\/ric\/current-members. Accessed 28 July 2022"},{"key":"10440_CR63","doi-asserted-by":"crossref","unstructured":"Saha RK, Lyu Y, Lam W, Yoshida H, Prasad MR (2018) Bugs.jar: a largescale, diverse dataset of real-world java bugs. In: International Conference on Mining Software Repositories, MSR \u201918, pp 10\u201313","DOI":"10.1145\/3196398.3196473"},{"key":"10440_CR64","volume-title":"The Coding Manual for Qualitative Researchers","author":"J Salda\u00f1a","year":"2015","unstructured":"Salda\u00f1a J (2015) The Coding Manual for Qualitative Researchers. Sage, Thousand Oaks, CA"},{"key":"10440_CR65","doi-asserted-by":"crossref","unstructured":"Santos A, Cunha A, Macedo N, Louren\u00e7o C (2016) A framework for quality assessment of ROS repositories. In: International conference on intelligent robots and systems, IROS \u201916, pp 4491\u20134496","DOI":"10.1109\/IROS.2016.7759661"},{"key":"10440_CR66","doi-asserted-by":"crossref","unstructured":"Santos A, Cunha A, Macedo N (2019) Static-time extraction and analysis of the ROS computation graph. In: International Conference on Robotic Computing, IRC \u201919, pp 62\u201369","DOI":"10.1109\/IRC.2019.00018"},{"key":"10440_CR67","doi-asserted-by":"crossref","unstructured":"Santos A, Cunha A, Macedo N (2021) The high-assurance ROS framework. In: International Workshop on Robotics Software Engineering, RoSE \u201921, p In press","DOI":"10.1109\/RoSE52553.2021.00013"},{"key":"10440_CR68","doi-asserted-by":"publisher","unstructured":"Santos A, Cunha A, Macedo N (2022) Schema-guided testing of message-oriented systems. In: International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE), SCITEPRESS, pp 26\u201337. https:\/\/doi.org\/10.5220\/0010976100003176","DOI":"10.5220\/0010976100003176"},{"key":"10440_CR69","doi-asserted-by":"crossref","unstructured":"Seaman CB, Shull F, Regardie M, Elbert D, Feldmann RL, Guo Y, Godfrey S (2008) Defect categorization: Making use of a decade of widely varying historical data. In: International Symposium on Empirical Software Engineering and Measurement, ESEM \u201908, pp 149\u2013157","DOI":"10.1145\/1414004.1414030"},{"key":"10440_CR70","doi-asserted-by":"crossref","unstructured":"Serebryany K, Iskhodzhanov T (2009) Threadsanitizer: data race detection in practice. In: Workshop on Binary Instrumentation and Applications, pp 62\u201371","DOI":"10.1145\/1791194.1791203"},{"key":"10440_CR71","unstructured":"Serebryany K, Bruening D, Potapenko A, Vyukov D (2012) AddressSanitizer: A fast address sanity checker. In: USENIX Annual Technical Conference, pp 309\u2013318"},{"key":"10440_CR72","doi-asserted-by":"publisher","first-page":"63","DOI":"10.3233\/EFI-2004-22201","volume":"22","author":"A Shenton","year":"2004","unstructured":"Shenton A (2004) Strategies for ensuring trustworthiness in qualitative research projects. Educ Inf 22:63\u201375. https:\/\/doi.org\/10.3233\/EFI-2004-22201","journal-title":"Educ Inf"},{"key":"10440_CR73","unstructured":"Sikolia D, Biros DP, Mason M, Weiser M (2013) Trustworthiness of grounded theory methodology research in information systems. MWAIS 2013 Proceedings, vol 16. https:\/\/aisel.aisnet.org\/mwais2013\/16"},{"key":"10440_CR74","doi-asserted-by":"crossref","unstructured":"Steinbauer G (2012) A survey about faults of robots used in RoboCup. In: Robot Soccer World Cup, Springer, pp 344\u2013355","DOI":"10.1007\/978-3-642-39250-4_31"},{"key":"10440_CR75","doi-asserted-by":"crossref","unstructured":"Stepanov E, Serebryany K (2015) MemorySanitizer: fast detector of uninitialized memory use in C++. In: International Symposium on Code Generation and Optimization, pp 46\u201355","DOI":"10.1109\/CGO.2015.7054186"},{"key":"10440_CR76","unstructured":"Tan SH, Yi J, Yulis, Mechtaev S, Roychoudhury A (2017) Codeflaws: A Programming Competition Benchmark for Evaluating Automated Program Repair Tools. In: International Conference on Software Engineering, ICSE \u201917, pp 180\u2013182"},{"key":"10440_CR77","doi-asserted-by":"crossref","unstructured":"Tan SH, Dong Z, Gao X, Roychoudhury A (2018) Repairing crashes in android apps. In: International Conference on Software Engineering, ICSE \u201918, pp 187\u2013198","DOI":"10.1145\/3180155.3180243"},{"key":"10440_CR78","unstructured":"The Autoware Foundation (2021) Autoware.ai. https:\/\/github.com\/Autoware-AI\/autoware.ai"},{"key":"10440_CR79","unstructured":"The Robotics Back-End (2021) rclcpp Params Tutorial - Get and Set ROS2 Params with Cpp. https:\/\/roboticsbackend.com\/rclcpp-params-tutorial-get-set-ros2-params-with-cpp\/#rclcpp_parameter_callback. Accessed 26 May 2021"},{"key":"10440_CR80","doi-asserted-by":"crossref","unstructured":"Thung F, Wang S, Lo D, Jiang L (2012) An empirical study of bugs in machine learning systems. In: International Symposium on Software Reliability Engineering, ISSRE \u201912, pp 271\u2013280","DOI":"10.1109\/ISSRE.2012.22"},{"key":"10440_CR81","doi-asserted-by":"publisher","unstructured":"Timperley C, Wasowski A (2019) 188 ros bugs later: Where do we go from here? In: ROSCon Macau 2019, Open Robotics. https:\/\/doi.org\/10.36288\/ROSCon2019-900898. https:\/\/doi.org\/10.36288\/ROSCon2019-900898","DOI":"10.36288\/ROSCon2019-900898"},{"key":"10440_CR82","doi-asserted-by":"crossref","unstructured":"Timperley CS, Afzal A, Katz DS, Hernandez JM, Le Goues C (2018a) Crashing simulated planes is cheap: Can simulation detect robotics bugs early? In: International Conference on Software Testing, Verification and Validation, IEEE, ICST \u201918, pp 331\u2013342","DOI":"10.29007\/hggx"},{"key":"10440_CR83","doi-asserted-by":"crossref","unstructured":"Timperley CS, Stepney S, Le Goues C (2018b) Bugzoo: a platform for studying software bugs. In: International Conference on Software Engineering: Companion Proceeedings, ICST \u201918, pp 446\u2013447","DOI":"10.1145\/3183440.3195050"},{"key":"10440_CR84","doi-asserted-by":"crossref","unstructured":"Timperley CS, D\u00fcrschmid T, Schmerl B, Garlan D, Le Goues C (2022) ROSDiscover: Statically Detecting Run-Time Architecture Misconfigurations in Robotics Systems. In: International Conference on Software Architecture, ICSA \u201922, pp 112\u2013123","DOI":"10.1109\/ICSA53651.2022.00019"},{"key":"10440_CR85","doi-asserted-by":"crossref","unstructured":"Tomassi DA, Dmeiri N, Wang Y, Bhowmick A, Liu YC, Devanbu PT, Vasilescu B, Rubio-Gonz\u00e1lez C (2019) Bugswarm: Mining and continuously growing a dataset of reproducible failures and fixes. In: International Conference on Software Engineering, ICSE \u201919, pp 339\u2013349","DOI":"10.1109\/ICSE.2019.00048"},{"key":"10440_CR86","doi-asserted-by":"crossref","unstructured":"Vahabzadeh A, Fard AM, Mesbah A (2015) An empirical study of bugs in test code. In: International Conference on Software Maintenance and Evolution, ICSME \u201915, pp 101\u2013110","DOI":"10.1109\/ICSM.2015.7332456"},{"key":"10440_CR87","doi-asserted-by":"crossref","unstructured":"van Tonder R, Le Goues C (2018) Static automated program repair for heap properties. In: International Conference on Software Engineering, pp 151\u2013162","DOI":"10.1145\/3180155.3180250"},{"key":"10440_CR88","unstructured":"Virding R, Wikstr\u00f6m C, Williams M (1996) Concurrent programming in ERLANG. Prentice Hall International (UK) Ltd"},{"key":"10440_CR89","unstructured":"Wall M (2017) European mars lander crashed due to data glitch, esa concludes. Space. https:\/\/www.space.com\/37015-schiaparelli-mars-lander-crash-investigation-complete.html"},{"key":"10440_CR90","doi-asserted-by":"crossref","unstructured":"Wan Z, Lo D, Xia X, Cai L (2017) Bug characteristics in blockchain systems: a large-scale empirical study. In: International Conference on Mining Software Repositories (MSR), MSR \u201917, pp 413\u2013424","DOI":"10.1109\/MSR.2017.59"},{"key":"10440_CR91","doi-asserted-by":"crossref","unstructured":"Wang D, Li S, Xiao G, Liu Y, Sui Y (2021) An Exploratory Study of Autopilot Software Bugs in Unmanned Aerial Vehicles. In: Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC\/FSE \u201921, pp 20\u201331","DOI":"10.1145\/3468264.3468559"},{"key":"10440_CR92","doi-asserted-by":"crossref","unstructured":"Wang J, Dou W, Gao Y, Gao C, Qin F, Yin K, Wei J (2017) A comprehensive study on real world concurrency bugs in node.js. In: International conference on automated software engineering, ASE \u201917, pp 520\u2013531","DOI":"10.1109\/ASE.2017.8115663"},{"key":"10440_CR93","unstructured":"White R (2019) Announcing Official Docker Images for ROS2. https:\/\/discourse.ros.org\/t\/announcing-official-docker-images-for-ros2\/7381. Accessed 9 June 2021"},{"key":"10440_CR94","doi-asserted-by":"crossref","unstructured":"Widyasari R, Sim SQ, Lok C, Qi H, Phan J, Tay Q, Tan C, Wee F, Tan JE, Yieh Y, Goh B, Thung F, Kang HJ, Hoang T, Lo D, Ouh EL (2020) BugsInPy: a database of existing bugs in Python programs to enable controlled testing and debugging studies. In: Joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE \u201920, pp 1556\u20131560","DOI":"10.1145\/3368089.3417943"},{"key":"10440_CR95","doi-asserted-by":"crossref","unstructured":"Witte T, Tichy M (2018) Checking consistency of robot software architectures in ros. In: International workshop on robotics software engineering, RoSE \u201918, pp 1\u20138","DOI":"10.1145\/3196558.3196559"},{"key":"10440_CR96","unstructured":"Wyrobek K (2017) The Origin Story of ROS, the Linux of Robotics. IEEE Spectrum. https:\/\/spectrum.ieee.org\/the-origin-story-of-ros-the-linux-of-robotics"},{"key":"10440_CR97","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-23669-3","volume-title":"Domain-Specific Languages: Effective Modeling, Automation, and Reuse","author":"A W\u0105sowski","year":"2023","unstructured":"W\u0105sowski A, Berger T (2023) Domain-Specific Languages: Effective Modeling, Automation, and Reuse. Springer"},{"key":"10440_CR98","doi-asserted-by":"crossref","unstructured":"Yang J, Yan C, Subramaniam P, Lu S, Cheung A (2018) How not to structure your database-backed web applications: a study of performance bugs in the wild. In: International conference on software engineering, ICSE \u201918, pp 800\u2013810","DOI":"10.1145\/3180155.3180194"},{"key":"10440_CR99","doi-asserted-by":"crossref","unstructured":"Yin Z, Ma X, Zheng J, Zhou Y, Bairavasundaram LN, Pasupathy S (2011) An empirical study on configuration errors in commercial and open source systems. In: Symposium on operating systems principles, pp 159\u2013172","DOI":"10.1145\/2043556.2043572"},{"key":"10440_CR100","doi-asserted-by":"crossref","unstructured":"Zampetti F, Kapur R, Di Penta M, Panichella S (2022) An empirical characterization of software bugs in open-source cyber-physical systems. J Syst Softw 192","DOI":"10.1016\/j.jss.2022.111425"},{"key":"10440_CR101","doi-asserted-by":"crossref","unstructured":"Zhang Y, Chen Y, Cheung SC, Xiong Y, Zhang L (2018) An empirical study on tensorflow program bugs. In: International symposium on software testing and analysis, ISSTA \u201918, pp 129\u2013140","DOI":"10.1145\/3213846.3213866"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-024-10440-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-024-10440-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-024-10440-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,1]],"date-time":"2024-06-01T02:08:40Z","timestamp":1717207720000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-024-10440-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,23]]},"references-count":101,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,5]]}},"alternative-id":["10440"],"URL":"https:\/\/doi.org\/10.1007\/s10664-024-10440-0","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,23]]},"assertion":[{"value":"3 January 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 March 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no competing interests to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interest"}}],"article-number":"57"}}