{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,28]],"date-time":"2026-05-28T18:35:57Z","timestamp":1779993357898,"version":"3.53.1"},"reference-count":294,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T00:00:00Z","timestamp":1712620800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2024,7,31]]},"abstract":"<jats:p>\n            <jats:bold>Artificial Intelligence (AI)<\/jats:bold>\n            can enable the development of next-generation autonomous safety-critical systems in which\n            <jats:bold>Machine Learning (ML)<\/jats:bold>\n            algorithms learn optimized and safe solutions. AI can also support and assist human safety engineers in developing safety-critical systems. However, reconciling both cutting-edge and state-of-the-art AI technology with safety engineering processes and safety standards is an open challenge that must be addressed before AI can be fully embraced in safety-critical systems. Many works already address this challenge, resulting in a vast and fragmented literature. Focusing on the industrial and transportation domains, this survey structures and analyzes challenges, techniques, and methods for developing AI-based safety-critical systems, from traditional functional safety systems to autonomous systems. AI\n            <jats:italic>trustworthiness<\/jats:italic>\n            spans several dimensions, such as engineering, ethics and legal, and this survey focuses on the safety engineering dimension.\n          <\/jats:p>","DOI":"10.1145\/3626314","type":"journal-article","created":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T15:24:57Z","timestamp":1697037897000},"page":"1-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":120,"title":["Artificial Intelligence for Safety-Critical Systems in Industrial and Transportation Domains: A Survey"],"prefix":"10.1145","volume":"56","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6389-648X","authenticated-orcid":false,"given":"Jon","family":"Perez-Cerrolaza","sequence":"first","affiliation":[{"name":"Ikerlan Technology Research Centre, Basque Research and Technology Alliance (BRTA), Mendaro, Guipuzcoa, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7951-4028","authenticated-orcid":false,"given":"Jaume","family":"Abella","sequence":"additional","affiliation":[{"name":"Barcelona Supercomputing Center (BSC), Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7879-4371","authenticated-orcid":false,"given":"Markus","family":"Borg","sequence":"additional","affiliation":[{"name":"RISE Research Institutes of Sweden AB, Lund, Scania, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-5102-3205","authenticated-orcid":false,"given":"Carlo","family":"Donzella","sequence":"additional","affiliation":[{"name":"Exida, Rovereto, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3752-644X","authenticated-orcid":false,"given":"Jes\u00fas","family":"Cerquides","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Research Institute (IIIA-CSIC), Bellaterra, Bareclona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3344-376X","authenticated-orcid":false,"given":"Francisco J.","family":"Cazorla","sequence":"additional","affiliation":[{"name":"BSC and Maspatechnologies S.L., Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1043-8773","authenticated-orcid":false,"given":"Cristofer","family":"Englund","sequence":"additional","affiliation":[{"name":"RISE Research Institutes of Sweden AB, Gothenburg, V\u00e4stra G\u00f6taland, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0003-2243","authenticated-orcid":false,"given":"Markus","family":"Tauber","sequence":"additional","affiliation":[{"name":"Research Studios Austria, Vienna, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0126-1897","authenticated-orcid":false,"given":"George","family":"Nikolakopoulos","sequence":"additional","affiliation":[{"name":"Lule\u00e5\u00a0University of Technology, Lulea, Norrbotten, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5555-9712","authenticated-orcid":false,"given":"Jose Luis","family":"Flores","sequence":"additional","affiliation":[{"name":"Ikerlan Technology Research Centre, BRTA, Arrasate\/Mondragon, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,4,9]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.3390\/su11010189"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793740"},{"key":"e_1_3_2_4_2","article-title":"How Drive.ai is mastering autonomous driving with deep learning > deep learning from the ground up helps drive\u2019s cars handle the challenges of autonomous driving","author":"Ackerman Evan","year":"2017","unstructured":"Evan Ackerman. 2017. How Drive.ai is mastering autonomous driving with deep learning > deep learning from the ground up helps drive\u2019s cars handle the challenges of autonomous driving. IEEE Spectrum (2017). https:\/\/spectrum.ieee.org\/how-driveai-is-mastering-autonomous-driving-with-deep-learning","journal-title":"IEEE Spectrum"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2870052"},{"key":"e_1_3_2_6_2","unstructured":"Prithvi Akella Ugo Rosolia Andrew Singletary and Aaron D Ames. 2020. Formal verification of safety critical autonomous systems via bayesian optimization. arXiv:2009.12909. Retrieved from https:\/\/arxiv.org\/abs\/2009.12909"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2807385"},{"key":"e_1_3_2_8_2","volume-title":"Safety of Machine Learning Systems in Autonomous Driving","author":"Al-Khoury Fadi","year":"2017","unstructured":"Fadi Al-Khoury. 2017. Safety of Machine Learning Systems in Autonomous Driving. Thesis."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2020.1003474"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2021.3075557"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/RTAS48715.2020.000-1"},{"key":"e_1_3_2_12_2","unstructured":"Rob Alexander Hamid Asgari Rob Ashmore Andrew Banks Rajiv Bongirwar Ben Bradshaw John Bragg John Clegg Jane Fenn Christopher Harper David Harvey Nikita Johnson Catherine Menon Roger Rivett Philippa Ryan Mark Sujan Nick Tudor and Stuart Tushingham. 2020. Safety Assurance Objectives for Autonomous Systems."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3100125"},{"key":"e_1_3_2_14_2","volume-title":"Concrete Problems in AI Safety","year":"2016","unstructured":"Dario Amodei, Chris Olah, Jacob Steindhart, Paul Christiano, John Schulman, and Dan Man\u00e9. 2016. Concrete Problems in AI Safety. Report."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2021.01.171"},{"key":"e_1_3_2_16_2","unstructured":"2023. ARP6983 (WIP) - Process Standard for Development and Certification\/Approval of Aeronautical Safety-Related Products Implementing AI. SAE."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2743240"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453444"},{"key":"e_1_3_2_19_2","unstructured":"2021. ASTM F3269-21: Standard Practice for Methods to Safely Bound Behavior of Aircraft Systems Containing Complex Functions Using Run-Time Assurance. ASTM."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN-W50199.2020.00024"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/IRPS45951.2020.9129519"},{"key":"e_1_3_2_22_2","unstructured":"AUTOSAR 2022. AUTOSAR (AUTomotive Open System ARchitecture). Retrieved September 30 2022 from https:\/\/www.autosar.org\/"},{"key":"e_1_3_2_23_2","doi-asserted-by":"crossref","unstructured":"A. Avi\u017eienis J. C. Laprie B. Randell and C. Landwehr. 2004. Basic concepts and taxonomy of dependable and secure computing. In Proceedings of the IEEE Transactions on Dependable and Secure Computing. 11\u201333.","DOI":"10.1109\/TDSC.2004.2"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-018-0637-6"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2018.8500632"},{"key":"e_1_3_2_26_2","unstructured":"Baidu. 2021. Apollo CyberRT framework for Autonomous Driving. Retrieved 30 June 2022 from https:\/\/github.com\/storypku\/CyberRT"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.5898\/JHRI.3.2.Beer"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.3390\/app8122590"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-26250-1_25"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","unstructured":"Raja Ben Abdessalem Annibale Panichella Shiva Nejati Lionel C. Briand and Thomas Stifter. 2018. Testing autonomous cars for feature interaction failures using many-objective search. In Proceedings of the 33rd ACM\/IEEE International Conference on Automated Software Engineering. 143\u2013154. DOI:10.1145\/3238147.3238192","DOI":"10.1145\/3238147.3238192"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2021.3118099"},{"key":"e_1_3_2_33_2","volume-title":"Proceedings of the Critical Automotive Applicat.: Robustness & Safety","year":"2015","unstructured":"Carl Bergenhem, Rolf Johansson, Andreas S\u00f6derberg, Jonas Nilsson, J\u00f6rgen Tryggvesson, Martin T\u00f6rngren, and Stig Ursing. 2015. How to reach complete safety requirement refinement for autonomous vehicles. In Proceedings of the Critical Automotive Applicat.: Robustness & Safety."},{"key":"e_1_3_2_34_2","volume-title":"Proceedings of the Auditing AI-Systems: From Basics to Applicat. (Workshop at Fraunhofer Forum)","year":"2020","unstructured":"Christian Berghoff, Battista Biggio, Elisa Brummel, Vasilios Danos, Thomas Doms, Heiko Ehrich, Thorsten Gantevoort, Barbara Hammer, Joachim Iden, Sven Jacob, Heidy Khlaaf, Lars Komrowski, Robert Kr\u00f6wing, Jan Hendrik Metzen, Matthias Neu, Fabian Petsch, Maximilian Poretschkin, Wojciech Samek, Hendrik Sch\u00e4be, Arndt von Twickel, Martin Vechev, and Thomas Wiegand. 2020. Towards auditable AI systems. In Proceedings of the Auditing AI-Systems: From Basics to Applicat. (Workshop at Fraunhofer Forum)."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-015-1530-5"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.5555\/3042573.3042761"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2018.07.023"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/LES.2019.2953253"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40793-2_15"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-55583-2_31"},{"key":"e_1_3_2_41_2","volume-title":"Proceedings of the 2nd Embedded Real Time Software Congr.","year":"2004","unstructured":"Jp. Blanquart, S. Fleury, M. Hernek, C. Honvault, F. Ingrand, J. Poncet, D. Powell, N. Strady-L\u00e9cubin, and P. Th\u00e9venod. 2004. Software safety supervision on-board autonomous spacecraft. In Proceedings of the 2nd Embedded Real Time Software Congr.."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2019.2914775"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaf2654"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-94238-0_1"},{"key":"e_1_3_2_45_2","unstructured":"Wnuk Boris Duran Christoffer Levandowski Shenjian Gao Yanwen Tan Henrik Kaijser Henrik L\u00f6nn and Jonas T\u00f6rnqvist. 2018. Safely entering the deep: A review of verification and validation for machine learning and a challenge elicitation in the automotive industry. arXiv:1812.05389. Retrieved from https:\/\/arxiv.org\/abs\/1812.05389"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/LATW.2019.8704548"},{"key":"e_1_3_2_47_2","first-page":"2039","article-title":"A survey of iterative learning control","volume":"26","author":"Bristow D. A.","year":"2006","unstructured":"D. A. Bristow, M. Tharayil, and A. G. Alleyne. 2006. A survey of iterative learning control. IEEE Control Systems Magazine 26, 3 (2006), 2039\u20132114.","journal-title":"IEEE Control Systems Magazine"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12567-020-00321-9"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSD.2016.84"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2019.103201"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-83903-1_8"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1038\/538020a"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3301283"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3418538"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW50294.2020.00063"},{"key":"e_1_3_2_56_2","volume-title":"CEN-CENELEC Focus Group Report: RoadMap on Artificial Intelligence (AI)","year":"2020","unstructured":"CENELEC. 2020. CEN-CENELEC Focus Group Report: RoadMap on Artificial Intelligence (AI). Report. CENELEC."},{"key":"e_1_3_2_57_2","unstructured":"CENELEC.2020. EN 50128:2011\/A1:2020 - Railway Applications: Communication signalling and processing systems - Software for railway control and protection systems."},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2017.2670225"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ssci.2020.104832"},{"key":"e_1_3_2_60_2","unstructured":"Bryant Chen Wilka Carvalho Nathalie Baracaldo Heiko Ludwig Benjamin Edwards Taesung Lee Ian M. Molloy and Biplav Srivastava. 2018. Detecting backdoor attacks on deep neural networks by activation clustering. CoRR abs\/1811.03728 (2018). Retrieved from http:\/\/arxiv.org\/abs\/1811.03728"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2021.3091477"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2020.2993926"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/JETCAS.2019.2910232"},{"key":"e_1_3_2_64_2","volume-title":"Proceedings of the Performance Metrics for Intelligent Systems Workshop","author":"Clough B.","year":"2002","unstructured":"B. Clough. 2002. Metrics, schmetrics! how the heck do you determine a UAV\u2019s autonomy anyway. In Proceedings of the Performance Metrics for Intelligent Systems Workshop."},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","unstructured":"Darren Cofer Isaac Amundson Ramachandra Sattigeri Arjun Passi Christopher Boggs Eric Smith Limei Gilham Taejoon Byun and Sanjai Rayadurgam. 2020. Run-Time Assurance for Learning-Enabled Systems. In NASA Formal Methods: 12th International Symposium NFM 2020 Moffett Field CA USA May 11.15 2020 Proceedings Springer-Verlag Moffett Field CA 361\u2013368. DOI:10.1007\/978-3-030-55754-6_21","DOI":"10.1007\/978-3-030-55754-6_21"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/IRC.2020.00062"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2021.3109555"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","unstructured":"Werner Dahm. 2010. Technology Horizons: A Vision for Air Force Science & Technology During 2010-2030. Air University Press Air Force Research Institute. DOI:10.21236\/ADA562237","DOI":"10.21236\/ADA562237"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3361682"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-34419-0_6"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.110941"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/DASC.2016.7778009"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533215"},{"key":"e_1_3_2_74_2","volume-title":"The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World","author":"Domingos Pedro","year":"2018","unstructured":"Pedro Domingos. 2018. The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books, Inc."},{"key":"e_1_3_2_75_2","first-page":"1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Du Min","year":"2020","unstructured":"Min Du, Ruoxi Jia, and Dawn Song. 2020. Robust anomaly detection and backdoor attack detection via differential privacy. In Proceedings of the International Conference on Learning Representations.1\u201319."},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.4271\/12-02-04-0018"},{"key":"e_1_3_2_77_2","volume-title":"EASA Concept Paper: First usable guidance for Level 1 machine learning applications - A deliverable of the EASA AI Roadmap","year":"2021","unstructured":"EASA. 2021. EASA Concept Paper: First usable guidance for Level 1 machine learning applications - A deliverable of the EASA AI Roadmap. Report. European Union Aviation safety Agency (EASA)."},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-68167-2_19"},{"key":"e_1_3_2_79_2","volume-title":"Process Considerations: A Reliable AI Data Labeling Process","author":"Eicher Matthis","year":"2020","unstructured":"Matthis Eicher, Patrick Scharpfenecker, Dieter Ludwig, Felix Friedmann, Florian Netter, and Marius Reuther. 2020. Process Considerations: A Reliable AI Data Labeling Process. Technical Report. Incenda AI and T\u00dcV S\u00dcD."},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/INISTA.2015.7276794"},{"key":"e_1_3_2_81_2","first-page":"6","volume-title":"Proceedings of the 3rd Eur. Embedded Real Time Softw.","author":"Erben Meinhard","year":"2006","unstructured":"Meinhard Erben, Wolf G\u00fcnther, Tobias Sedlmeier, Dieter Lederer, and Klaus-J\u00fcrgen Amsler. 2006. Legal aspects of safety designed software development, especially under european law. In Proceedings of the 3rd Eur. Embedded Real Time Softw.. 6."},{"key":"e_1_3_2_82_2","unstructured":"EU. 2016. Regulation (EU) 2016\/679 of the European parliament and of the council - on the protection of natural persons with regard to the processing of personal data and on the free movement of such data and repealing Directive 95\/46\/EC (General Data Protection Regulation)."},{"key":"e_1_3_2_83_2","unstructured":"EU. 2019. Ethics Guidelines for Trustworthy AI. European Commission - High-Level Expert Group on Artificial Intell. (HLEG AI). Retrieved from https:\/\/digital-strategy.ec.europa.eu\/en\/library\/ethics-guidelines-trustworthy-ai"},{"key":"e_1_3_2_84_2","unstructured":"EU. 2021. Proposal for a Regulation of the European Parliament and the Council - Laying down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts. European Commission. Retrieved from https:\/\/eur-lex.europa.eu\/legal-content\/en\/ALL\/?uri=CELEX:52021PC0206"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/768"},{"key":"e_1_3_2_86_2","volume-title":"Liability for Artificial Intelligence and Other Emerging Digital Technologies","author":"Technologies Expert Group on Liability and New","year":"2019","unstructured":"Expert Group on Liability and New Technologies. 2019. Liability for Artificial Intelligence and Other Emerging Digital Technologies. Report. European Comission."},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9196709"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/3194104.3194109"},{"key":"e_1_3_2_89_2","doi-asserted-by":"crossref","unstructured":"Javier Fern\u00e1ndez Jon Perez Irune Agirre Imanol Allende Jaume Abella and Francisco J. Cazorla. 2021. Towards functional safety compliance of matrix-matrix multiplication for machine learning-based autonomous systems. Journal of Systems Architecture 121 (2021) 102298.","DOI":"10.1016\/j.sysarc.2021.102298"},{"key":"e_1_3_2_90_2","first-page":"59","volume-title":"Computer Safety, Reliability, and Security: 37th International Conference, SAFECOMP 2018, V\u00e4ster\u00e5s, Sweden, September 19-21, 2018, Proceedings","year":"2018","unstructured":"Patrik Feth, Rasmus Adler, Takeshi Fukuda, Tasuku Ishigooka, Satoshi Otsuka, Daniel Schneider, Denis Uecker, and Kentaro Yoshimura. 2018. Multi-aspect safety engineering for highly automated driving. In Computer Safety, Reliability, and Security: 37th International Conference, SAFECOMP 2018, V\u00e4ster\u00e5s, Sweden, September 19-21, 2018, Proceedings. Springer, 59\u201372."},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2018.2876389"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.13140\/RG.2.1.2797.7447"},{"key":"e_1_3_2_93_2","volume-title":"Uncertainty in Deep Learning","author":"Gal Yarin","year":"2016","unstructured":"Yarin Gal. 2016. Uncertainty in Deep Learning. Ph.D. Dissertation. University of Cambridge."},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2019.8917103"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.5555\/2789272.2886795"},{"key":"e_1_3_2_96_2","volume-title":"IJCAI Workshop on Artificial Intell. Safety (AISafety)","year":"2021","unstructured":"Florian Gei\u00dfler, Syed Qutub, Sayanta Roychowdhury, Ali Asgari, Yang Peng, Akash Dhamasia, Karthik Pattabiraman, and Michael Paulitsch. 2021. Towards a safety case for hardware fault tolerance in convolutional neural networks using activation range supervision. In IJCAI Workshop on Artificial Intell. Safety (AISafety)."},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","unstructured":"M. Gharib and A. Bondavalli. On the evaluation measures for machine learning algorithms for safety-critical systems. In 15th European Dependable Computing Conference (EDCC). 141\u2013144. DOI:10.1109\/EDCC.2019.00035","DOI":"10.1109\/EDCC.2019.00035"},{"key":"e_1_3_2_98_2","doi-asserted-by":"publisher","DOI":"10.1145\/3412841.3442074"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-017-1000-6"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464841"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.3141\/2424-07"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSRS.2016.7815842"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1007\/s42979-021-00726-1"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21918"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236009"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2017.04.004"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSCON.2018.8369576"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2020.100037"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409754"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1109\/DASC.1993.283552"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1109\/IRCE50905.2020.9199244"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1002\/j.2334-5837.2016.00179.x"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1145\/3194085.3194090"},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106409"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1109\/JEEIT.2019.8717439"},{"key":"e_1_3_2_116_2","doi-asserted-by":"publisher","DOI":"10.1145\/3451179"},{"key":"e_1_3_2_117_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cosrev.2020.100270"},{"key":"e_1_3_2_118_2","doi-asserted-by":"crossref","unstructured":"Xiaowei Huang Marta Kwiatkowska Sen Wang and Min Wu. 2017. Safety verification of deep neural networks.Computer Aided Verification: 29th International Conference CAV 2017 Heidelberg Germany July 24-28 2017 Proceedings Part I 30. Springer Internat. Publishing 3\u201329.","DOI":"10.1007\/978-3-319-63387-9_1"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-05318-5"},{"key":"e_1_3_2_120_2","unstructured":"IEC 2009. IEC 62267: Railway applications - Automated urban guided transport (AUGT) - Safety requirements."},{"key":"e_1_3_2_121_2","unstructured":"IEC 2010. IEC 61508(-1\/7): Functional safety of electrical\/electronic\/programmable electronic safety-related systems."},{"key":"e_1_3_2_122_2","unstructured":"IEC 2014. IEC 62290-1: Railway applications - Urban guided transport management and command\/control systems - Part 1: System principles and fundamental concepts."},{"key":"e_1_3_2_123_2","unstructured":"IEEE. 2021. IEEE 7000: IEEE Standard Model Process for Addressing Ethical Concerns during System Design."},{"key":"e_1_3_2_124_2","unstructured":"ISO. 2009. ISO 10975: Tractors and machinery for agriculture - Auto-guidance systems for operator-controlled tractors and self-propelled machines - Safety requirements."},{"key":"e_1_3_2_125_2","unstructured":"ISO. 2011. ISO 10218-1: Robots and robotic devices - Safety requirements for industrial robots Part 1: Robots."},{"key":"e_1_3_2_126_2","unstructured":"ISO. 2015. ISO 13849-1: Safety of machinery Safety-related parts of control systems Part 1: General principles for design."},{"key":"e_1_3_2_127_2","unstructured":"ISO. 2016. ISO\/TS 15066: Robots and robotic devices Collaborative robots."},{"key":"e_1_3_2_128_2","unstructured":"ISO. 2017. ISO 16001: Earth-moving machinery - Object detection systems and visibility aids - Performance requirements and tests."},{"key":"e_1_3_2_129_2","unstructured":"ISO. 2018. ISO 18497: Agricultural machinery and tractors - Safety of highly automated agricultural machines - Principles for design."},{"key":"e_1_3_2_130_2","unstructured":"ISO. 2018. ISO 18758-2: Mining and earth-moving machinery - Rock drill rigs and rock reinforcement rigs - Part 2: Safety requirements."},{"key":"e_1_3_2_131_2","unstructured":"ISO. 2018. ISO 25119: Tractors and machinery for agriculture and forestry - Safety-related parts of control systems."},{"key":"e_1_3_2_132_2","unstructured":"ISO 2018. ISO 26262(-1\/11) Road vehicles - Functional safety."},{"key":"e_1_3_2_133_2","unstructured":"ISO 2019. ISO 17757: Earth-moving machinery and mining Autonomous and semi-autonomous machine system safety."},{"key":"e_1_3_2_134_2","unstructured":"ISO. 2019. ISO\/PAS 21448: Road vehicles Safety of the intended functionality (SOTIF)."},{"key":"e_1_3_2_135_2","unstructured":"ISO 2020. ISO 3691-4: Industrial trucks Safety requirements and verification Part 4: Driverless industrial trucks and their systems."},{"key":"e_1_3_2_136_2","unstructured":"ISO 2020. ISO\/TR 4804 Road vehicles Safety and cybersecurity for automated driving systems Design verification and validation."},{"key":"e_1_3_2_137_2","unstructured":"ISO 2021. ISO\/AWI TS 5083 Road vehicles Safety for automated driving systems Design verification and validation."},{"key":"e_1_3_2_138_2","unstructured":"ISO 2021. ISO\/IEC AWI TR 5469: Artificial intelligence Functional safety and AI systems (draft)."},{"key":"e_1_3_2_139_2","unstructured":"ISO. 2021. ISO\/IEC DIS 22989: Information technology - Artificial Intelligence - Artificial intelligence concepts and terminology (draft)."},{"key":"e_1_3_2_140_2","unstructured":"ISO 2021. ISO\/IEC DTR 24368 - Information technology - Artificial intelligence Overview of ethical and societal concerns (Draft)."},{"key":"e_1_3_2_141_2","unstructured":"ISO. 2021. ISO\/IEC TR 24030: Information technology Artificial intelligence (AI) Use cases."},{"key":"e_1_3_2_142_2","unstructured":"ISO 2021. ISO\/TR 22100-5: Safety of machinery - relationship with ISO 12100 - Part 5: Implications of artificial intelligence machine learning."},{"key":"e_1_3_2_143_2","doi-asserted-by":"publisher","unstructured":"Stephen Jacklin Johann Schumann Pramod Gupta Michael Richard Kurt Guenther and Fola Soares. Development of advanced verification and validation procedures and tools for the certification of learning systems in aerospace applications. In Infotech@Aerospace. DOI:10.2514\/6.2005-6912","DOI":"10.2514\/6.2005-6912"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","DOI":"10.1162\/0899766053630350"},{"key":"e_1_3_2_145_2","volume-title":"Rule Extraction from Recurrent Neural Networks","author":"Jacobsson Henrik","year":"2006","unstructured":"Henrik Jacobsson. 2006. Rule Extraction from Recurrent Neural Networks. Thesis."},{"key":"e_1_3_2_146_2","doi-asserted-by":"publisher","DOI":"10.3390\/s18030925"},{"key":"e_1_3_2_147_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2018.8569661"},{"key":"e_1_3_2_148_2","first-page":"29","volume-title":"Proceedings of the 10th European Congress on Embedded Real Time Systems","year":"2020","unstructured":"Eric Jenn, Alexandre Albore, Franck Mamalet, Gr\u00e9gory Flandin, Christophe Gabreau, Herv\u00e9 Delseny, Adrien Gauffriau, Hugues Bonnin, Lucian Alecu, and J\u00e9r\u00e9my Pirard. 2020. Identifying challenges to the certification of machine learning for safety critical systems. In Proceedings of the 10th European Congress on Embedded Real Time Systems. 29\u201331."},{"key":"e_1_3_2_149_2","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2019.8813776"},{"key":"e_1_3_2_150_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN.2019.00025"},{"key":"e_1_3_2_151_2","doi-asserted-by":"publisher","DOI":"10.1109\/AERO.2001.931288"},{"key":"e_1_3_2_152_2","doi-asserted-by":"publisher","DOI":"10.1162\/99608f92.f06c6e61"},{"key":"e_1_3_2_153_2","doi-asserted-by":"publisher","DOI":"10.1145\/3140659.3080246"},{"key":"e_1_3_2_154_2","doi-asserted-by":"publisher","DOI":"10.2514\/1.G003724"},{"key":"e_1_3_2_155_2","doi-asserted-by":"publisher","DOI":"10.1109\/DASC.2016.7778091"},{"key":"e_1_3_2_156_2","volume-title":"Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?","author":"Kalra Nidhi","year":"2016","unstructured":"Nidhi Kalra and Susan M. Paddock. 2016. Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?RAND Corporation."},{"key":"e_1_3_2_157_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2017.05.006"},{"key":"e_1_3_2_158_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-63387-9_5"},{"key":"e_1_3_2_159_2","unstructured":"Alex Kendall Vijay Badrinarayanan and Roberto Cipolla. 2015. Bayesian SegNet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv:1511.02680. Retrieved from https:\/\/arxiv.org\/abs\/1511.02680"},{"key":"e_1_3_2_160_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487679"},{"key":"e_1_3_2_161_2","article-title":"Deep reinforcement learning for autonomous driving: A survey","author":"Kiran B. Ravi","year":"2021","unstructured":"B. Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A. Al Sallab, Senthil Yogamani, and Patrick P\u00e9rez. 2021. Deep reinforcement learning for autonomous driving: A survey. IEEE Trans. on Intelligent Transportation Syst. 23, 6 (2021), 4909\u20134926.","journal-title":"IEEE Trans. on Intelligent Transportation Syst."},{"key":"e_1_3_2_162_2","doi-asserted-by":"publisher","DOI":"10.1016\/0957-4174(91)90175-E"},{"key":"e_1_3_2_163_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-26250-1_26"},{"key":"e_1_3_2_164_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2021.3108035"},{"key":"e_1_3_2_165_2","doi-asserted-by":"publisher","DOI":"10.4271\/2016-01-0128"},{"key":"e_1_3_2_166_2","doi-asserted-by":"publisher","DOI":"10.1109\/MITS.2016.2583491"},{"key":"e_1_3_2_167_2","doi-asserted-by":"publisher","DOI":"10.4271\/2018-01-1071"},{"key":"e_1_3_2_168_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2018.8569971"},{"key":"e_1_3_2_169_2","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2019.8813994"},{"key":"e_1_3_2_170_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-86044-8_3"},{"key":"e_1_3_2_171_2","article-title":"Toward scalable verification for safety-critical deep networks","author":"Kuper Lindsey","year":"2018","unstructured":"Lindsey Kuper, Guy Katz, Justin Gottschlich, Kyle Julian, Clark Barrett, and Mykel Kochenderfer. 2018. Toward scalable verification for safety-critical deep networks. ArXiV (2018).","journal-title":"ArXiV"},{"key":"e_1_3_2_172_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-006-0039-9"},{"key":"e_1_3_2_173_2","unstructured":"Zeshan Kurd and Tim P. Kelly. 2004. Using fuzzy self-organising maps for safety critical systems. In Proceedings of the Reliability Engineering & System Safety. Maritta Heisel Peter Liggesmeyer and Stefan Wittmann (Eds.) Springer Berlin 17\u201330."},{"key":"e_1_3_2_174_2","doi-asserted-by":"publisher","DOI":"10.1007\/11563228_11"},{"key":"e_1_3_2_175_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-020-05872-w"},{"key":"e_1_3_2_176_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICECS46596.2019.8964912"},{"key":"e_1_3_2_177_2","doi-asserted-by":"publisher","DOI":"10.1109\/FDL50818.2020.9232934"},{"key":"e_1_3_2_178_2","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126964"},{"key":"e_1_3_2_179_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2019.106200"},{"key":"e_1_3_2_180_2","doi-asserted-by":"publisher","DOI":"10.23919\/CISTI49556.2020.9141124"},{"key":"e_1_3_2_181_2","volume-title":"Industrial Use of Safety-related Artificial Neural Networks","author":"Lisboa Paulo","year":"2001","unstructured":"Paulo Lisboa. 2001. Industrial Use of Safety-related Artificial Neural Networks. Report. Health & Safety Executive (HSE). Retrieved from http:\/\/www.hse.gov.uk\/research\/crr_pdf\/2001\/crr01327.pdf"},{"key":"e_1_3_2_182_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2016.12.038"},{"key":"e_1_3_2_183_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2020.03.017"},{"key":"e_1_3_2_184_2","doi-asserted-by":"publisher","DOI":"10.1145\/3342355"},{"key":"e_1_3_2_185_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-017-0284-0"},{"key":"e_1_3_2_186_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2022.3196396"},{"key":"e_1_3_2_187_2","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2020.1003021"},{"key":"e_1_3_2_188_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abm6074"},{"key":"e_1_3_2_189_2","unstructured":"Joseph Machrouh Jean-Paul Blanquart Philippe Baufreton J. L. Boulanger Herv\u00e9 Delseny Jean Gassino G\u00e9rard Ladier Emmanuel Ledinot Michel Leeman and Jean-Marc Astruc. 2012. Cross domain comparison of System Assurance. In Embedded Real Time Software and Systems (ERTS)."},{"key":"e_1_3_2_190_2","volume-title":"Proceedings of the Int. Conf. on Learning Representations","author":"Madry Aleksander","year":"2018","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In Proceedings of the Int. Conf. on Learning Representations."},{"key":"e_1_3_2_191_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-59717-0_11"},{"key":"e_1_3_2_192_2","doi-asserted-by":"publisher","DOI":"10.1201\/9781351117821-10"},{"key":"e_1_3_2_193_2","doi-asserted-by":"publisher","DOI":"10.1145\/3487043"},{"key":"e_1_3_2_194_2","volume-title":"Proceedings of the AISafety@IJCAI","author":"McDermid J.","year":"2020","unstructured":"J. McDermid and Yan Jia. 2020. Safety of artificial intelligence: A collaborative model. In Proceedings of the AISafety@IJCAI."},{"key":"e_1_3_2_195_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0065-2458(05)65004-8"},{"key":"e_1_3_2_196_2","unstructured":"Jan Hendrik Metzen Tim Genewein Volker Fischer and Bastian Bischoff. 2017. On detecting adversarial perturbations. In 5th International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_2_197_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2015.2426179"},{"key":"e_1_3_2_198_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2017.10.031"},{"key":"e_1_3_2_199_2","unstructured":"Prabhat Nagarajan et\u00a0al. 2019. Deterministic Implementations for Reproducibility in Deep Reinforcement Learning. arXiv:1809.05676. Retrieved from https:\/\/arxiv.org\/abs\/1809.05676"},{"key":"e_1_3_2_200_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-018-09679-z"},{"key":"e_1_3_2_201_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-85729-410-4_65"},{"key":"e_1_3_2_202_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICUAS51884.2021.9476765"},{"key":"e_1_3_2_203_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-54422-9"},{"key":"e_1_3_2_204_2","doi-asserted-by":"publisher","DOI":"10.1108\/IJQRM-02-2019-0065"},{"key":"e_1_3_2_205_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-62144-5_10"},{"key":"e_1_3_2_206_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.01.012"},{"key":"e_1_3_2_207_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-1-85617-752-8.10045-5"},{"key":"e_1_3_2_208_2","doi-asserted-by":"publisher","DOI":"10.3390\/make2040031"},{"key":"e_1_3_2_209_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2021.3074394"},{"key":"e_1_3_2_210_2","doi-asserted-by":"publisher","DOI":"10.1145\/3549526"},{"key":"e_1_3_2_211_2","doi-asserted-by":"publisher","DOI":"10.1145\/3398665"},{"key":"e_1_3_2_212_2","first-page":"23","volume-title":"Proceedings of the Workshop on Artificial Intelligence Safety","author":"Picardi Chiara","year":"2020","unstructured":"Chiara Picardi, Colin Paterson, Richard David Hawkins, Radu Calinescu, and Ibrahim Habli. 2020. Assurance argument patterns and processes for machine learning in safety-related systems. In Proceedings of the Workshop on Artificial Intelligence Safety. 23\u201330."},{"key":"e_1_3_2_213_2","doi-asserted-by":"publisher","DOI":"10.1145\/3234150"},{"key":"e_1_3_2_214_2","doi-asserted-by":"publisher","DOI":"10.3233\/AIC-2012-0525"},{"key":"e_1_3_2_215_2","doi-asserted-by":"publisher","DOI":"10.5555\/1199801"},{"key":"e_1_3_2_216_2","volume-title":"Proceedings of the Embedded World","author":"Putzer Henrik J.","year":"2021","unstructured":"Henrik J. Putzer et\u00a0al. 2021. Trustworthy AI-based systems with VDE-AR-E 2842-61. In Proceedings of the Embedded World."},{"key":"e_1_3_2_217_2","volume-title":"Proceedings of the 29th USENIX Conference on Security Symposium","author":"Quiring Erwin","year":"2020","unstructured":"Erwin Quiring et\u00a0al. 2020. Adversarial preprocessing: Understanding and preventing image-scaling attacks in machine learning. In Proceedings of the 29th USENIX Conference on Security Symposium."},{"key":"e_1_3_2_218_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW53098.2021.00023"},{"key":"e_1_3_2_219_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3055015"},{"key":"e_1_3_2_220_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3048047"},{"key":"e_1_3_2_221_2","doi-asserted-by":"publisher","DOI":"10.5555\/2971808.2971988"},{"key":"e_1_3_2_222_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09881-0"},{"key":"e_1_3_2_223_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2993730"},{"key":"e_1_3_2_224_2","doi-asserted-by":"publisher","unstructured":"Albert Rizaldi Jonas Keinholz Monika Huber Jochen Feldle Fabian Immler Matthias Althoff Eric Hilgendorf and Tobias Nipkow. 2017. Formalising and monitoring traffic rules for Autonomous Vehicles in Isabelle\/HOL. In Integrated Formal Methods (IFS) 50\u201366. DOI:10.1007\/978-3-319-66845-1_4","DOI":"10.1007\/978-3-319-66845-1_4"},{"key":"e_1_3_2_225_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3099287"},{"key":"e_1_3_2_226_2","first-page":"5","volume-title":"Autonomous Driving \u2013 A Practical Roadmap (2010-01-2335)","author":"Ronald Jurgen","year":"2013","unstructured":"Jurgen Ronald. 2013. Autonomous Driving \u2013 A Practical Roadmap (2010-01-2335). SAE, 5\u201326."},{"key":"e_1_3_2_227_2","unstructured":"RTCA. 2011. DO-178C\/EUROCAE ED-12C - Software Considerations in Airborne Systems and Equipment Certification."},{"key":"e_1_3_2_228_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0048-x"},{"key":"e_1_3_2_229_2","volume-title":"Proceedings of the European Congr. Embedded Real-Time Syst.","author":"Rudolph Alexander","year":"2018","unstructured":"Alexander Rudolph, Stefan Voget, and J\u00fcrgen Mottok. 2018. A consistent safety case argumentation for artificial intelligence in safety related automotive systems. In Proceedings of the European Congr. Embedded Real-Time Syst.."},{"key":"e_1_3_2_230_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSD51259.2020.00109"},{"key":"e_1_3_2_231_2","unstructured":"SAE. 2010. Aerospace Recommended Practice ARP4754 Guidelines For Development Of Civil Aircraft and Systems."},{"key":"e_1_3_2_232_2","unstructured":"SAE. 2014. J3016 - Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems."},{"key":"e_1_3_2_233_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2016.08.030"},{"key":"e_1_3_2_234_2","unstructured":"Rick Salay and Krzysztof Czarnecki. 2018. Using machine learning safely in automotive software: An assessment and adaption of software process requirements in ISO 26262. arXiv:1808.01614. Retrieved from https:\/\/arxiv.org\/abs\/1808.01614"},{"key":"e_1_3_2_235_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-26250-1_23"},{"key":"e_1_3_2_236_2","doi-asserted-by":"publisher","unstructured":"R. Salay R. Queiroz and K. Czarnecki. 2018. An analysis of ISO 26262: Machine learning and safety in automotive. In WCX World Congress Experience (SAE Technical Paper 2018-01-1075) SAE. DOI:10.4271\/2018-01-1075","DOI":"10.4271\/2018-01-1075"},{"key":"e_1_3_2_237_2","volume-title":"Proceedings of the 4th International Conference on Safety and Reliability.","author":"Sallak Mohamed","year":"2006","unstructured":"Mohamed Sallak, Christophe Simon, and Jean-Fran\u00e7ois Aubry. 2006. Evaluating safety integrity level in presence of uncertainty. In Proceedings of the 4th International Conference on Safety and Reliability."},{"key":"e_1_3_2_238_2","volume-title":"Artificial Intelligence Applied to Software Testing","author":"Salvado Jo\u00e3o Alexandre Pedroso","year":"2019","unstructured":"Jo\u00e3o Alexandre Pedroso Salvado. 2019. Artificial Intelligence Applied to Software Testing. Thesis."},{"key":"e_1_3_2_239_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN-W.2017.47"},{"key":"e_1_3_2_240_2","doi-asserted-by":"publisher","DOI":"10.1049\/iet-cdt.2018.5026"},{"key":"e_1_3_2_241_2","doi-asserted-by":"publisher","DOI":"10.1109\/DASC43569.2019.9081649"},{"key":"e_1_3_2_242_2","doi-asserted-by":"crossref","unstructured":"Sebastian Schirmer et\u00a0al. [n. d.]. Considerations of artificial intelligence safety engineering for unmanned aircraft.(Computer Safety Reliability and Security)(2018) 465\u2013472.","DOI":"10.1007\/978-3-319-99229-7_40"},{"key":"e_1_3_2_243_2","volume-title":"Artificial Intelligence and Functional Safety - A summary of the current challenges","author":"Schneider Volker","year":"2021","unstructured":"Volker Schneider. 2021. Artificial Intelligence and Functional Safety - A summary of the current challenges. Report. T\u00dcV SUD Rail GmbH. Retrieved from https:\/\/metsta.fi\/wp-content\/uploads\/2021\/05\/Artificial-Intelligence-and-Functional-Safety.pdf"},{"key":"e_1_3_2_244_2","unstructured":"Catherine D. Schuman Thomas E. Potok Robert M. Patton J. Douglas Birdwell Mark E. Dean Garrett S. Rose and James S. Plank. 2017. A Survey of Neuromorphic Computing and Neural Networks in Hardware. arXiv:1705.06963. Retrieved from https:\/\/arxiv.org\/abs\/1705.06963"},{"key":"e_1_3_2_245_2","doi-asserted-by":"publisher","DOI":"10.5555\/1840986"},{"key":"e_1_3_2_246_2","doi-asserted-by":"publisher","DOI":"10.20378\/irb-47275"},{"key":"e_1_3_2_247_2","first-page":"2503","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems","author":"Sculley D.","year":"2015","unstructured":"D. Sculley et\u00a0al. 2015. Hidden technical debt in machine learning systems. In Proceedings of the 28th International Conference on Neural Information Processing Systems. 2503\u20132511."},{"key":"e_1_3_2_248_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2020.3016104"},{"key":"e_1_3_2_249_2","first-page":"458","volume-title":"Computer Safety, Reliability, and Security: SAFECOMP 2018 Workshops, ASSURE, DECSoS, SASSUR, STRIVE, and WAISE, V\u00e4ster\u00e5s, Sweden, September 18, 2018, Proceedings 37","author":"Shafaei Sina","year":"2018","unstructured":"Sina Shafaei, Stefan Kugele, Mohd Hafeez Osman, and Alois Knoll. 2018. Uncertainty in machine learning: A safety perspective on autonomous driving. In Computer Safety, Reliability, and Security: SAFECOMP 2018 Workshops, ASSURE, DECSoS, SASSUR, STRIVE, and WAISE, V\u00e4ster\u00e5s, Sweden, September 18, 2018, Proceedings 37. Springer Internat. Publishing, 458\u2013464."},{"key":"e_1_3_2_250_2","doi-asserted-by":"crossref","unstructured":"Mahmood Sharif Sruti Bhagavatula Lujo Bauer and Michael K. Reiter. 2016. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. Association for Computing Machinery 1528\u20131540.","DOI":"10.1145\/2976749.2978392"},{"key":"e_1_3_2_251_2","doi-asserted-by":"publisher","DOI":"10.21236\/ADA057655"},{"key":"e_1_3_2_252_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.41"},{"key":"e_1_3_2_253_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jlp.2018.11.002"},{"key":"e_1_3_2_254_2","volume-title":"Robotics 2020 - Multi-Annual Roadmap For Robotics in Europe - Horizon 2020 Call ICT-2017 (ICT-25, ICT-27 & ICT-28)","year":"2016","unstructured":"SPARC. 2016. Robotics 2020 - Multi-Annual Roadmap For Robotics in Europe - Horizon 2020 Call ICT-2017 (ICT-25, ICT-27 & ICT-28). Report. SPARC (The Partnership for Robotics in Europe)."},{"key":"e_1_3_2_255_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2019.8917385"},{"key":"e_1_3_2_256_2","doi-asserted-by":"publisher","DOI":"10.1145\/3302504.3311802"},{"key":"e_1_3_2_257_2","doi-asserted-by":"publisher","DOI":"10.1145\/3316781.3317779"},{"key":"e_1_3_2_258_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISORC49007.2020.00030"},{"key":"e_1_3_2_259_2","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2020.2975764"},{"key":"e_1_3_2_260_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-011-9271-y"},{"key":"e_1_3_2_261_2","doi-asserted-by":"publisher","unstructured":"Brian Taylor Marjorie Darrah and Christina Moats. 2003. Verification and validation of neural networks: A sampling of research in progress. In Proceedings of SPIE - The International Society for Optical Engineering. DOI:10.1117\/12.487527","DOI":"10.1117\/12.487527"},{"key":"e_1_3_2_262_2","volume-title":"Methods and Procedures for the Verification and Validation of Artificial Neural Networks","author":"Taylor Brian J.","year":"2006","unstructured":"Brian J. Taylor. 2006. Methods and Procedures for the Verification and Validation of Artificial Neural Networks. Springer Science & Business Media."},{"key":"e_1_3_2_263_2","doi-asserted-by":"publisher","DOI":"10.1117\/12.487527"},{"key":"e_1_3_2_264_2","first-page":"129","volume-title":"Proceedings of the Computer Safety, Reliability, and Security.","author":"Terrosi Francesco","unstructured":"Francesco Terrosi, Lorenzo Strigini, and Andrea Bondavalli. [n. d.]. Impact of machine learning on safety monitors. In Proceedings of the Computer Safety, Reliability, and Security.Mario Trapp, Francesca Saglietti, Marc Spisl\u00e4nder, and Friedemann Bitsch (Eds.), Springer Int. Publishing, 129\u2013143."},{"key":"e_1_3_2_265_2","doi-asserted-by":"publisher","DOI":"10.1016\/S1474-6670(17)55918-2"},{"key":"e_1_3_2_266_2","doi-asserted-by":"publisher","DOI":"10.56094\/jss.v55i2.46"},{"issue":"29","key":"e_1_3_2_267_2","first-page":"380","article-title":"Testing the intelligence of unmanned autonomous systems","author":"Thompson Miles S.","year":"2008","unstructured":"Miles S. Thompson. 2008. Testing the intelligence of unmanned autonomous systems. ITEA Journal29 (2008), 380\u2013387.","journal-title":"ITEA Journal"},{"key":"e_1_3_2_268_2","doi-asserted-by":"publisher","DOI":"10.1515\/eng-2020-0074"},{"key":"e_1_3_2_269_2","doi-asserted-by":"publisher","unstructured":"Christoph Torens Franz Juenger Sebastian Schirmer Simon Schopferer Theresa D. Maienschein and Johann C. Dauer. 2022. Machine learning verification and safety for unmanned aircraft - A literature study. In AIAA SCITECH 2022 Forum. DOI:10.2514\/6.2022-1133","DOI":"10.2514\/6.2022-1133"},{"key":"e_1_3_2_270_2","first-page":"55","volume-title":"Proceedings of the International Workshop on Formal Techniques for Safety-Critical Systems.","author":"T\u00f6rnblom John","year":"2018","unstructured":"John T\u00f6rnblom and Simin Nadjm-Tehrani. 2018. Formal verification of random forests in safety-critical applications. In Proceedings of the International Workshop on Formal Techniques for Safety-Critical Systems.Springer, 55\u201371."},{"key":"e_1_3_2_271_2","doi-asserted-by":"publisher","DOI":"10.5555\/3241094.3241142"},{"key":"e_1_3_2_272_2","unstructured":"Brandon Tran Jerry Li and Aleksander Ma\u0327dry. 2018. Spectral signatures in backdoor attacks. In 32nd International Conference on Neural Information Processing Systems. Curran Associates Inc. 8011\u20138021."},{"key":"e_1_3_2_273_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-20055-8_48"},{"key":"e_1_3_2_274_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2019.2955903"},{"key":"e_1_3_2_275_2","volume-title":"Basics of Machine Learning with Aspects of Functional Safety and Cybersecurity","year":"2022","unstructured":"TUVR. 2022. Basics of Machine Learning with Aspects of Functional Safety and Cybersecurity. Report. T\u00dcV Rheinland."},{"key":"e_1_3_2_276_2","unstructured":"ULSE. 2020. UL 4600 - Standard for Evaluation of Autonomous Products."},{"key":"e_1_3_2_277_2","volume-title":"A Survey of Introducing Artificial Intelligence Into the Safety Critical System Software Design Process","author":"Varadaraju Rakshith","year":"2007","unstructured":"Rakshith Varadaraju. 2007. A Survey of Introducing Artificial Intelligence Into the Safety Critical System Software Design Process. Report. University of Northern Iowa."},{"key":"e_1_3_2_278_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITA.2016.7888195"},{"key":"e_1_3_2_279_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-47166-2_49"},{"key":"e_1_3_2_280_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.clsr.2015.03.008"},{"key":"e_1_3_2_281_2","unstructured":"VDE. 2021. VDE-AR-E 2842-61: Development and trustworthiness of autonomous\/cognitive systems."},{"key":"e_1_3_2_282_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2794604"},{"key":"e_1_3_2_283_2","doi-asserted-by":"crossref","unstructured":"Francis Rhys Ward and Ibrahim Habli. 2020. An assurance case pattern for the interpretability of machine learning in safety-critical systems.Computer Safety Reliability and Security. SAFECOMP 2020 Workshops: DECSoS 2020 DepDevOps 2020 USDAI 2020 and WAISE 2020 Lisbon Portugal September 15 2020 Proceedings 39. Springer Internat. Publishing 395\u2013407.","DOI":"10.1007\/978-3-030-55583-2_30"},{"key":"e_1_3_2_284_2","unstructured":"Waymo. 2019. AutoML: Automating the design of machine learning models for autonomous driving. Retrieved from https:\/\/blog.waymo.com\/2019\/07\/automl-automating-design-of-machine.html"},{"key":"e_1_3_2_285_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSPEC.2011.5960163"},{"key":"e_1_3_2_286_2","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3969214"},{"key":"e_1_3_2_287_2","doi-asserted-by":"publisher","DOI":"10.1145\/3494523"},{"key":"e_1_3_2_288_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2018.23198"},{"key":"e_1_3_2_289_2","doi-asserted-by":"crossref","unstructured":"Puyudi Yang Jianbo Chen Cho-Jui Hsieh Jane-Ling Wang and Michael I. Jordan. 2020. ML-LOO: Detecting adversarial examples with feature attribution. Proceedings of the AAAI Conference on Artificial Intelligence. 6639\u20136647.","DOI":"10.1609\/aaai.v34i04.6140"},{"key":"e_1_3_2_290_2","volume-title":"Unveiled: BMW\u2019s Scalable AV Architecture","author":"Yoshida Junko","year":"2020","unstructured":"Junko Yoshida. 2020. Unveiled: BMW\u2019s Scalable AV Architecture. IEEE."},{"key":"e_1_3_2_291_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trpro.2017.01.024"},{"key":"e_1_3_2_292_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106296"},{"key":"e_1_3_2_293_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2962027"},{"key":"e_1_3_2_294_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2017.2701846"},{"key":"e_1_3_2_295_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394885.3431623"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626314","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3626314","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:53:37Z","timestamp":1750287217000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626314"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,9]]},"references-count":294,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,7,31]]}},"alternative-id":["10.1145\/3626314"],"URL":"https:\/\/doi.org\/10.1145\/3626314","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,9]]},"assertion":[{"value":"2022-09-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-26","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}