{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T23:57:00Z","timestamp":1784159820030,"version":"3.55.0"},"reference-count":24,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2020,3,2]],"date-time":"2020-03-02T00:00:00Z","timestamp":1583107200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["2018-0-00327"],"award-info":[{"award-number":["2018-0-00327"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We propose a log analysis framework for test driving of autonomous vehicles. The log of a vehicle is a fundamental source to detect and analyze events during driving. A set of dumped logs are, however, usually mixed and fragmented since they are generated concurrently by a number of modules such as sensors, actuators and programs. This makes it hard to analyze them to discover latent errors that could occur due to complex chain reactions among those modules. Our framework provides a logging architecture based on formal specifications, which hierarchically organizes them to find out a priori relationships between them. Then, algorithmic or implementation errors can be detected by examining a posteriori relationships. However, a test in a situation of certain parameters, so called an oracle test, does not necessarily trigger latent violations of the relationships. In our framework, this is remedied by adopting metamorphic testing to quantitatively verify the formal specification. As a working proof, we define three metamorphic relations critical for testing autonomous vehicles and verify them in a quantitative manner based on our logging system.<\/jats:p>","DOI":"10.3390\/s20051356","type":"journal-article","created":{"date-parts":[[2020,3,3]],"date-time":"2020-03-03T03:13:28Z","timestamp":1583205208000},"page":"1356","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["A Formal and Quantifiable Log Analysis Framework for Test Driving of Autonomous Vehicles"],"prefix":"10.3390","volume":"20","author":[{"given":"Kyungbok","family":"Sung","sequence":"first","affiliation":[{"name":"Autonomous Driving Intelligence Research Section, Artificial Intelligence Research Laboratory, Electronics and Telecommunications Research Institute, Daejeon 34129, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kyoung-Wook","family":"Min","sequence":"additional","affiliation":[{"name":"Autonomous Driving Intelligence Research Section, Artificial Intelligence Research Laboratory, Electronics and Telecommunications Research Institute, Daejeon 34129, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeongdan","family":"Choi","sequence":"additional","affiliation":[{"name":"Autonomous Driving Intelligence Research Section, Artificial Intelligence Research Laboratory, Electronics and Telecommunications Research Institute, Daejeon 34129, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Byung-Cheol","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Joongbu University-Goyang, Goyang 10279, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1007\/s42154-018-0009-9","article-title":"Survey on Artificial Intelligence for Vehicles","volume":"1","author":"Li","year":"2018","journal-title":"Automot. Innov."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Oh, S., and Kang, H. (2017). Object detection and classification by decision-level fusion for intelligent vehicle systems. Sensors, 17.","DOI":"10.3390\/s17010207"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"54","DOI":"10.4218\/etrij.2018-0435","article-title":"Dynamic swarm particle for fast motion vehicle tracking","volume":"42","author":"Jati","year":"2020","journal-title":"ETRI J."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bimbraw, K. (2015, January 21\u201323). Autonomous Cars: Past, Present and Future\u2014A Review of the Developments in the Last Century, the Present Scenario and the Expected Future of Autonomous Vehicle Technology. Proceedings of the 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO), Colmar, France.","DOI":"10.5220\/0005540501910198"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"102021","DOI":"10.1109\/ACCESS.2019.2926040","article-title":"MIT advanced vehicle technology study: Large-scale naturalistic driving study of driver behavior and interaction with automation","volume":"7","author":"Fridman","year":"2019","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.aap.2017.11.001","article-title":"Autonomous vehicles\u2019 disengagements: Trends, triggers, and regulatory limitations","volume":"110","author":"Eurich","year":"2018","journal-title":"Accid. Anal. Prev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1016\/j.tra.2016.09.010","article-title":"Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?","volume":"94","author":"Kalra","year":"2016","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_8","unstructured":"Levin, S. (2018). Uber crash shows\u2019 catastrophic failure\u2019of self-driving technology, experts say. The Guard., Available online: https:\/\/www.theguardian.com\/technology\/2018\/mar\/22\/self-driving-car-uber-death-woman-failure-fatal-crash-arizona."},{"key":"ref_9","unstructured":"Wongpiromsarn, T., and Murray, R.M. (2008, January 9\u201311). Formal verification of an autonomous vehicle system. Proceedings of the Conference on Decision and Control, Cancun, Maxico. Available online: http:\/\/www.cds.caltech.edu\/~murray\/papers\/wm08-cdc.html."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Seshia, S.A., Sadigh, D., and Sastry, S.S. (2015, January 8\u201312). Formal methods for semi-autonomous driving. Proceedings of the 2015 52nd ACM\/EDAC\/IEEE Design Automation Conference (DAC), San Francisco, CA, USA.","DOI":"10.1145\/2744769.2747927"},{"key":"ref_11","unstructured":"Chen, T.Y.T., Cheung, S.S.C., and Yiu, S.S.M. (1998). Metamorphic testing: A new approach for generating next test cases. Tech. Rep. HKUST-CS98-01, Dep. Comput. Sci. Hong Kong Univ. Sci. Technol. Hong Kong, 1\u201311. Available online: https:\/\/www.cse.ust.hk\/~scc\/publ\/CS98-01-metamorphictesting.pdf."},{"key":"ref_12","unstructured":"Segura, S., and Zhou, Z.Q. (, January May). Metamorphic testing 20 years later: A hands-on introduction. Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings, Gothenburg, Sweden."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1145\/3241979","article-title":"Metamorphic testing of driverless cars","volume":"62","author":"Zhou","year":"2019","journal-title":"Commun. ACM"},{"key":"ref_14","unstructured":"(2019, December 10). California Code of Regulations, Title 13, Division 1, Chapter 1, Article 3.7. Available online: https:\/\/govt.westlaw.com\/calregs\/Browse\/Home\/California\/CaliforniaCodeofRegulations?guid=I14E801D0D46811DE8879F88E8B0DAAAE&originationContext=documenttoc&transitionType=Default&contextData=(sc.Default)."},{"key":"ref_15","unstructured":"(2019, December 10). Motor Vehicle Management Act, Statutes of the Republic of Korea, Article 27. Available online: https:\/\/elaw.klri.re.kr\/eng_service\/lawView.do?hseq=42015&lang=ENG."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The KITTI dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Rob. Res."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The Cityscapes Dataset for Semantic Urban Scene Understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_18","unstructured":"Jin, X., Deshmukh, J.V., Kapinski, J., Ueda, K., and Butts, K. (,  2014). Challenges of Applying Formal Methods to Automotive Control Systems. Proceedings of the NSF National Workshop on Transportation Cyber-Physical Systems, Available online: https:\/\/cps-vo.org\/node\/11225."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.scico.2017.05.006","article-title":"Formal verification of autonomous vehicle platooning","volume":"148","author":"Kamali","year":"2017","journal-title":"Sci. Comput. Program."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1093\/comjnl\/25.4.465","article-title":"On testing non-testable programs","volume":"25","author":"Weyuker","year":"1982","journal-title":"Comput. J."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Sung, K., Min, K., and Choi, J. (2018, January 11\u201314). Driving information logger with in-vehicle communication for autonomous vehicle research. Proceedings of the International Conference on Advanced Communication Technology, ICACT, Chuncheon-si Gangwon-do, Korea.","DOI":"10.23919\/ICACT.2018.8323731"},{"key":"ref_22","unstructured":"Thorn, E., Kimmel, S., and Chaka, M. (2019, December 10). A Framework for Automated Driving System Testable Cases and Scenarios, Available online: https:\/\/www.nhtsa.gov\/sites\/nhtsa.dot.gov\/files\/documents\/13882-automateddrivingsystems_092618_v1a_tag.pdf."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1109\/TSE.2016.2532875","article-title":"A Survey on Metamorphic Testing","volume":"42","author":"Segura","year":"2016","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_24","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13\u201315). CARLA: An Open Urban Driving Simulator. Proceedings of the 1st Annual Conference on Robot Learning, PMLR, Mountain View, CA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1356\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:03:10Z","timestamp":1760173390000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1356"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,2]]},"references-count":24,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["s20051356"],"URL":"https:\/\/doi.org\/10.3390\/s20051356","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,2]]}}}