{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T00:42:50Z","timestamp":1775263370019,"version":"3.50.1"},"reference-count":15,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,4,24]],"date-time":"2020-04-24T00:00:00Z","timestamp":1587686400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Within a workshop on evaluation methods for automated vehicles (AVs) at the Driving Assessment 2019 symposium in Santa Fe; New Mexico, a heuristic evaluation methodology that aims at supporting the development of human\u2013machine interfaces (HMIs) for AVs was presented. The goal of the workshop was to bring together members of the human factors community to discuss the method and to further promote the development of HMI guidelines and assessment methods for the design of HMIs of automated driving systems (ADSs). The workshop included hands-on experience of rented series production partially automated vehicles, the application of the heuristic assessment method using a checklist, and intensive discussions about possible revisions of the checklist and the method itself. The aim of the paper is to summarize the results of the workshop, which will be used to further improve the checklist method and make the process available to the scientific community. The participants all had previous experience in HMI design of driver assistance systems, as well as development and evaluation methods. They brought valuable ideas into the discussion with regard to the overall value of the tool against the background of the intended application, concrete improvements of the checklist (e.g., categorization of items; checklist items that are currently perceived as missing or redundant in the checklist), when in the design process the tool should be applied, and improvements for the usability of the checklist.<\/jats:p>","DOI":"10.3390\/info11040233","type":"journal-article","created":{"date-parts":[[2020,4,24]],"date-time":"2020-04-24T11:42:14Z","timestamp":1587728534000},"page":"233","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Checklist for Expert Evaluation of HMIs of Automated Vehicles\u2014Discussions on Its Value and Adaptions of the Method within an Expert Workshop"],"prefix":"10.3390","volume":"11","author":[{"given":"Nadja","family":"Sch\u00f6mig","sequence":"first","affiliation":[{"name":"W\u00fcrzburg Institute for Traffic Sciences GmbH (WIVW), D-97209 Veitsh\u00f6chheim, Germany"}]},{"given":"Katharina","family":"Wiedemann","sequence":"additional","affiliation":[{"name":"W\u00fcrzburg Institute for Traffic Sciences GmbH (WIVW), D-97209 Veitsh\u00f6chheim, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4061-8273","authenticated-orcid":false,"given":"Sebastian","family":"Hergeth","sequence":"additional","affiliation":[{"name":"BMW Group, D-80937 M\u00fcnchen, Germany"}]},{"given":"Yannick","family":"Forster","sequence":"additional","affiliation":[{"name":"BMW Group, D-80937 M\u00fcnchen, Germany"}]},{"given":"Jeffrey","family":"Muttart","sequence":"additional","affiliation":[{"name":"Crash Safety Research Center, LLC, 201 W High Street, B8 East Hampton, New York, NY 06424, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1549-1327","authenticated-orcid":false,"given":"Alexander","family":"Eriksson","sequence":"additional","affiliation":[{"name":"Volvo Car Corporation, Torslanda, SE-405 31 G\u00f6teborg, Sweden"}]},{"given":"David","family":"Mitropoulos-Rundus","sequence":"additional","affiliation":[{"name":"Hyundai American Technical Center, Superior Township, MI 48198, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5863-3378","authenticated-orcid":false,"given":"Kevin","family":"Grove","sequence":"additional","affiliation":[{"name":"Virginia Tech Transportation Institute, 3500 Transportation Research Plaza, Blacksburg, VA 24061, USA"}]},{"given":"Josef","family":"Krems","sequence":"additional","affiliation":[{"name":"Department of Behavioural and Social Sciences, Technische Universit\u00e4t Chemnitz, 09111 Chemnitz, Germany"}]},{"given":"Andreas","family":"Keinath","sequence":"additional","affiliation":[{"name":"BMW Group, D-80937 M\u00fcnchen, Germany"}]},{"given":"Alexandra","family":"Neukum","sequence":"additional","affiliation":[{"name":"W\u00fcrzburg Institute for Traffic Sciences GmbH (WIVW), D-97209 Veitsh\u00f6chheim, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1712-6473","authenticated-orcid":false,"given":"Frederik","family":"Naujoks","sequence":"additional","affiliation":[{"name":"BMW Group, D-80937 M\u00fcnchen, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,24]]},"reference":[{"key":"ref_1","unstructured":"National Highway Traffic Safety Administration (2017). Automated Driving Systems 2.0: A Vision for Safety."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1080\/15389588.2019.1603374","article-title":"Test procedure for evaluating the human\u2013machine interface of vehicles with automated driving systems","volume":"20","author":"Naujoks","year":"2019","journal-title":"Traffic Inj. Prev."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Naujoks, F., Hergeth, S., Wiedemann, K., Sch\u00f6mig, N., and Keinath, A. (2018, January 1\u20135). Use cases for assessing, testing, and validating the human machine interface of automated driving systems. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Philadelphia, PA, USA.","DOI":"10.1177\/1541931218621426"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Nielsen, J. (1993). Usability Engineering, Academic Press.","DOI":"10.1016\/B978-0-08-052029-2.50007-3"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.trf.2018.10.012","article-title":"Towards guidelines and verification methods for automated vehicle HMIs","volume":"60","author":"Naujoks","year":"2019","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_6","unstructured":"Naujoks, F., Hergeth, S., Keinath, A., Wiedemann, K., and Sch\u00f6mig, N. (2019, January 10\u201313). Development and application of an expert based assessment for evaluating the usability of SAE Level 3 ADS HMIs. Proceedings of the ESV Conference 2019, Eindhoven, The Netherlands."},{"key":"ref_7","unstructured":"RESPONSE Consortium (2020, April 21). Code of Practice for the Design and Evaluation of ADAS; RESPONSE 3: A PReVENT Project; 2006. Available online: https:\/\/www.acea.be\/uploads\/publications\/20090831_Code_of_Practice_ADAS.pdf."},{"key":"ref_8","unstructured":"ISO 26262 (2008). Road Vehicles Functional Safety, International Organization for Standardization."},{"key":"ref_9","unstructured":"(2020, April 21). UN ECE R79: Uniform Provisions Concerning the Approval of Vehicles with Regard to Steering Equipment. Available online: https:\/\/www.unece.org\/fileadmin\/DAM\/trans\/main\/wp29\/wp29regs\/2017\/R079r3e.pdf."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1177\/0018720814547570","article-title":"Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust","volume":"57","author":"Hoff","year":"2015","journal-title":"Hum. Factors"},{"key":"ref_11","unstructured":"Parasuraman, R., and Davies, R. (1984). Processing resources in attention. Varieties of Attention, Academic Press."},{"key":"ref_12","unstructured":"UNECE (2020, February 18). Uniform Provisions Concerning the Approval of Vehicles with Regard to Automated Lane Keeping System. Informal Document ACSF-25-03. Available online: https:\/\/wiki.unece.org\/download\/attachments\/92013066\/ACSF-25-23%20%28Chairs%29%20Draft%20UN%20Regulation%20for%20ALKS%20for%20GRVA.pdf?api=v2."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"202","DOI":"10.17011\/ht\/urn.2006519","article-title":"Creating a framework for improving the learnability of a complex system","volume":"2","year":"2006","journal-title":"Hum. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Borojeni, S.S., Chuang, L., Heuten, W., and Boll, S. (2016). Assisting Drivers with Ambient Take-Over Requests in Highly Automated Driving. Automotive\u2019 UI: Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Association for Computing Machinery.","DOI":"10.1145\/3003715.3005409"},{"key":"ref_15","unstructured":"ANSI Z535.4-2011 (R2017) (2020, February 18). American National Standard for Product Safety Signs and Labels. Available online: https:\/\/webstore.ansi.org\/preview-pages\/NEMA\/preview_ANSI+Z535.4-2011+(R2017).pdf."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/4\/233\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:33:26Z","timestamp":1760366006000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/4\/233"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,24]]},"references-count":15,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["info11040233"],"URL":"https:\/\/doi.org\/10.3390\/info11040233","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,24]]}}}