{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T15:46:57Z","timestamp":1776095217102,"version":"3.50.1"},"reference-count":49,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2020,3,24]],"date-time":"2020-03-24T00:00:00Z","timestamp":1585008000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Research on external human\u2013machine interfaces (eHMIs) has recently become a major area of interest in the field of human factors research on automated driving. The broad variety of methodological approaches renders the current state of research inconclusive and comparisons between interface designs impossible. To date, there are no standardized test procedures to evaluate and compare different design variants of eHMIs with each other and with interactions without eHMIs. This article presents a standardized test procedure that enables the effective usability evaluation of eHMI design solutions. First, the test procedure provides a methodological approach to deduce relevant use cases for the evaluation of an eHMI. In addition, we define specific usability requirements that must be fulfilled by an eHMI to be effective, efficient, and satisfying. To prove whether an eHMI meets the defined requirements, we have developed a test protocol for the empirical evaluation of an eHMI with a participant study. The article elucidates underlying considerations and details of the test protocol that serves as framework to measure the behavior and subjective evaluations of non-automated road users when interacting with automated vehicles in an experimental setting. The standardized test procedure provides a useful framework for researchers and practitioners.<\/jats:p>","DOI":"10.3390\/info11030173","type":"journal-article","created":{"date-parts":[[2020,3,24]],"date-time":"2020-03-24T13:04:04Z","timestamp":1585055044000},"page":"173","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":38,"title":["Standardized Test Procedure for External Human\u2013Machine Interfaces of Automated Vehicles"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8055-9805","authenticated-orcid":false,"given":"Christina","family":"Ka\u00df","sequence":"first","affiliation":[{"name":"W\u00fcrzburg Institute for Traffic Sciences GmbH, 97209 Veitsh\u00f6chheim, Germany"}]},{"given":"Stefanie","family":"Schoch","sequence":"additional","affiliation":[{"name":"W\u00fcrzburg Institute for Traffic Sciences GmbH, 97209 Veitsh\u00f6chheim, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1712-6473","authenticated-orcid":false,"given":"Frederik","family":"Naujoks","sequence":"additional","affiliation":[{"name":"BMW Group, 80937 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4061-8273","authenticated-orcid":false,"given":"Sebastian","family":"Hergeth","sequence":"additional","affiliation":[{"name":"BMW Group, 80937 Munich, Germany"}]},{"given":"Andreas","family":"Keinath","sequence":"additional","affiliation":[{"name":"BMW Group, 80937 Munich, Germany"}]},{"given":"Alexandra","family":"Neukum","sequence":"additional","affiliation":[{"name":"W\u00fcrzburg Institute for Traffic Sciences GmbH, 97209 Veitsh\u00f6chheim, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Hensch, A.-C., Neumann, I., Beggiato, M., Halama, J., and Krems, J.F. How Should Automated Vehicles Communicate?\u2014Effects of a Light-Based Communication Approach in a Wizard-of-Oz Study. Proceedings of the AHFE 2019 International Conference on Human Factors in Transportation, Washington, DC, USA, 24\u201328 July 2019, Springer.","DOI":"10.1007\/978-3-030-20503-4_8"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1016\/j.aap.2018.03.018","article-title":"What externally presented information do VRUs require when interacting with fully Automated Road Transport Systems in shared space?","volume":"118","author":"Merat","year":"2018","journal-title":"Accid. Anal. Prev."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1007\/s10111-018-0521-z","article-title":"Designing the interaction of automated vehicles with other traffic participants: Design considerations based on human needs and expectations","volume":"21","author":"Schieben","year":"2019","journal-title":"Cogn. Technol. Work"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Mahadevan, K., Somanath, S., and Sharlin, E. (2018, January 21\u201326). Communicating awareness and intent in autonomous vehicle-pedestrian interaction. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montr\u00e9al, QC, Canada.","DOI":"10.1145\/3173574.3174003"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Eisma, Y., Van Bergen, S., Ter Brake, S., Hensen, M., Tempelaar, W., and De Winter, J. (2020). External human-machine interfaces: The effect of display location on crossing intentions and eye movements. Information, 11.","DOI":"10.3390\/info11010013"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kooijman, L., Happee, R., and de Winter, J.C.F. (2019). How do eHMIs affect pedestrians\u2019 crossing behavior? A study using a head-mounted display combined with a motion suit. Information, 10.","DOI":"10.3390\/info10120386"},{"key":"ref_7","unstructured":"Otherson, I., Conti-Kufner, A.S., Dietrich, A., Maruhn, P., and Bengler, K. (2018, January 8\u201310). Designing for Automated Vehicle and Pedestrian Communication: Perspectives on eHMIs from Older and Younger Persons. Proceedings of the Human Factors and Ergonomics Society Europe Chapter 2018 Annual Conference, Berlin, Germany."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1016\/j.apergo.2018.11.002","article-title":"An experimental study to investigate design and assessment criteria: What is important for communication between pedestrians and automated vehicles?","volume":"75","author":"Ackermann","year":"2019","journal-title":"Appl. Ergon."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"B\u00f6ckle, M.-P., Brenden, A.P., Klingeg\u00e5rd, M., Habibovic, A., and Bout, M. (2017, January 24\u201327). SAV2P: Exploring the Impact of an Interface for Shared Automated Vehicles on Pedestrians\u2019 Experience. Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct, Oldenburg, Germany.","DOI":"10.1145\/3131726.3131765"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1353","DOI":"10.1177\/0018720819836343","article-title":"External Human-Machine Interfaces on Automated Vehicles: Effects on Pedestrian Crossing Decisions","volume":"61","author":"Dietrich","year":"2019","journal-title":"Hum. Factors"},{"key":"ref_11","unstructured":"Lagstrom, T., and Malmsten Lundgren, V. (2015). AVIP-Autonomous Vehicles Interaction with Pedestrians. [Master\u2019s Thesis, Chalmers University of Technology]."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1336","DOI":"10.3389\/fpsyg.2018.01336","article-title":"Communicating Intent of Automated Vehicles to Pedestrians","volume":"9","author":"Habibovic","year":"2018","journal-title":"Front. Psychol."},{"key":"ref_13","unstructured":"Gruenefeld, U., Wei\u00df, S., L\u00f6cken, A., Virgilio, I., Kun, A.L., and Boll, S. (2019, January 22\u201325). VRoad: Gesture-based interaction between pedestrians and automated vehicles in virtual reality. Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings, Utrecht, The Netherlands."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Rodr\u00edguez Palmeiro, A. (2017). Interaction between Pedestrians and Wizard of Oz Automated Vehicles. [Master\u2019s Thesis, Technical University Delft].","DOI":"10.1016\/j.trf.2018.07.020"},{"key":"ref_15","first-page":"94","article-title":"Interactions between cyclists and automated vehicles: Results of a photo experiment","volume":"12","author":"Hagenzieker","year":"2020","journal-title":"J. Transp. Saf. Secur."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1007\/978-3-319-73888-8_3","article-title":"External HMIs and their effect on the interaction between pedestrians and automated vehicles","volume":"Volume 722","author":"Karwowski","year":"2018","journal-title":"Intelligent Human Systems Integration"},{"key":"ref_17","unstructured":"Dietrich, A., Willrodt, J.-H., Wagner, K., and Bengler, K. (2018, January 5\u20137). Projection-Based External Human Machine Interfaces-Enabling Interaction between Automated Vehicles and Pedestrians. Proceedings of the Driving Simulation Conference 2018 Europe VR, Antibes, France."},{"key":"ref_18","unstructured":"Clamann, M., Aubert, M., and Cummings, M.L. (2017, January 8\u201312). Evaluation of vehicle-to-pedestrian communication displays for autonomous vehicles. Proceedings of the 96th Annual Transportation Research Board Meeting, Washintgon, DC, USA."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.trf.2018.08.016","article-title":"Investigating pedestrian suggestions for external features on fully autonomous vehicles: A virtual reality experiment","volume":"59","author":"Deb","year":"2018","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Li, Y., Dikmen, M., Hussein, T.G., Wang, Y., and Burns, C. (2018, January 23\u201325). To cross or not to cross: Urgency-based external warning displays on autonomous vehicles to improve pedestrian crossing safety. Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Toronto, ON, Canada.","DOI":"10.1145\/3239060.3239082"},{"key":"ref_21","first-page":"145","article-title":"Comparison of Child and Adult Pedestrian Perspectives of External Features on Autonomous Vehicles Using Virtual Reality Experiment","volume":"Volume 964","author":"Stanton","year":"2019","journal-title":"AHFE 2019: Advances in Human Factors of Transportation"},{"key":"ref_22","unstructured":"Fridman, L., Mehler, B., Xia, L., Yang, Y., Facusse, L.Y., and Reimer, B. (2020, March 24). To Walk or Not to Walk: Crowdsourced Assessment of External Vehicle-to-Pedestrian Displays. Available online: https:\/\/arxiv.org\/abs\/1707.02698."},{"key":"ref_23","unstructured":"Yang, S. (2017). Driver Behavior Impact on Pedestrians\u2019 Crossing Experience in the Conditionally Autonomous Driving Context. [Student\u2019s Thesis, School of Computer Science and Communication]."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"L\u00f6cken, A., Golling, C., and Riener, A. (2019, January 22\u201325). How Should Automated Vehicles Interact with Pedestrians? A Comparative Analysis of Interaction Concepts in Virtual Reality. Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Application, Utrecht, The Netherlands.","DOI":"10.1145\/3342197.3344544"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1049\/iet-its.2017.0321","article-title":"Potential safety effects of a frontal brake light for motor vehicles","volume":"12","author":"Petzoldt","year":"2018","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Naujoks, F., Hergeth, S., Wiedemann, K., Sch\u00f6mig, N., and Keinath, A. (2018, January 1\u20135). Use cases for assessing, testing, and validating the human machine interface of automated driving systems. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Philadelphia, PA, USA.","DOI":"10.1177\/1541931218621426"},{"key":"ref_27","first-page":"551","article-title":"Testing scenarios for human factors research in level 3 automated vehicles","volume":"Volume 597","author":"Stanton","year":"2017","journal-title":"AHFE 2017: Advances in Human Aspects of Transportation"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"McCall, R., McGee, F., Meschtscherjakov, A., Louveton, N., and Engel, T. (2016, January 24\u201326). Towards a taxonomy of autonomous vehicle handover situations. Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ann Arbor, MI, USA.","DOI":"10.1145\/3003715.3005456"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.trf.2016.10.007","article-title":"Human factors of transitions in automated driving: A general framework and literature survey","volume":"43","author":"Lu","year":"2016","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_30","first-page":"708","article-title":"Taxonomy of traffic situations for the interaction between automated vehicles and human road users","volume":"Volume 597","author":"Stanton","year":"2017","journal-title":"AHFE 2017: Advances in Human Aspects of Transportation"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1080\/15389588.2019.1603374","article-title":"Test procedure for evaluating the human\u2013machine interface of vehicles with automated driving systems","volume":"20","author":"Naujoks","year":"2019","journal-title":"Traffic Inj. Prev."},{"key":"ref_32","unstructured":"Alliance of Automobile Manufacturers (2006). Statement of Principles, Criteria and Verification Procedures on Driver Interactions with Advanced In-Vehicle Information and Communication Systems Including, Alliance of Automobile Manufacturers."},{"key":"ref_33","unstructured":"National Highway Traffic Safety Administration (2014). Visual-Manual NHTSA Driver Distraction Guidelines for In-Vehicle Electronic Devices, Department of Transportation."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2757","DOI":"10.3389\/fpsyg.2019.02757","article-title":"External Human-Machine Interfaces for Autonomous Vehicle-to-Pedestrian Communication: A Review of Empirical Work","volume":"10","author":"Rouchitsas","year":"2019","journal-title":"Front. Psychol."},{"key":"ref_35","unstructured":"International Organization for Standardization (2018). Ergonomics of Human-System Interaction\u2014Part 11: Usability: Definitions and Concepts, International Organization for Standardization. ISO 9241-11."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Markkula, G., Madigan, R., Nathanael, D., Portouli, E., Lee, Y.M., Dietrich, A., Billington, J., Schieben, A., and Merat, N. (2020, February 24). Defining Interactions: A Conceptual Framework for Understanding Interactive Behaviour in Human and Automated Road Traffic. Available online: https:\/\/doi.org\/10.31234\/osf.io\/8w9z4.","DOI":"10.31234\/osf.io\/8w9z4"},{"key":"ref_37","unstructured":"Amundsen, F.H., and Hyd\u00e9n, C. (1977). Proceedings of the First Workshop on Traffic Conflicts, Institute of Transport Economics."},{"key":"ref_38","unstructured":"(2020, February 24). Forschungsgesellschaft f\u00fcr Stra\u00dfen- und Verkehrswesen. AG 2 Stra\u00dfenentwurf. Available online: https:\/\/www.fgsv.de\/gremien\/strassenentwurf.html."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kraft, A.-K., Maag, C., and Baumann, M. (2019). How to support cooperative driving by HMI design?. Transp. Res. Interdiscip. Perspect., 3.","DOI":"10.1016\/j.trip.2019.100064"},{"key":"ref_40","unstructured":"SAE International (2020, February 24). Taxonomy and Definitions for Terms Related to Driving Automation Systems for on-Road Motor Vehicles (No. J3016). Available online: https:\/\/saemobilus.sae.org\/content\/j3016_201806."},{"key":"ref_41","unstructured":"Federal Highway Administration (2003). Manual on Uniform Traffic Control Devices, Federal Highway Administration."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2263","DOI":"10.1016\/j.atmosenv.2010.04.028","article-title":"Exposure to particulate matter in traffic: A comparison of cyclists and car passengers","volume":"44","author":"Panis","year":"2010","journal-title":"Atmos. Environ."},{"key":"ref_43","unstructured":"National Highway Traffic Safety Administration (2017). Federal Automated Vehicles Policy 2.0, Department of Transportation."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Fuest, T., Michalowski, L., Tr\u00e4ris, L., Bellem, H., and Bengler, K. (2018, January 4\u20137). Using the Driving Behavior of an Automated Vehicle to Communicate Intentions-A Wizard of Oz Study. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569486"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1016\/j.trf.2019.03.006","article-title":"Deceleration parameters and their applicability as informal communication signal between pedestrians and automated vehicles","volume":"62","author":"Ackermann","year":"2019","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_46","unstructured":"Response Consortium (2020, March 24). Code of Practice for the Design and Evaluation of ADAS; A Prevent Project; Response: 2006; Volume 3. Available online: https:\/\/www.acea.be\/uploads\/publications\/20090831_Code_of_Practice_ADAS.pdf."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"425","DOI":"10.2307\/30036540","article-title":"User acceptance of information technology: Toward a unified view","volume":"27","author":"Venkatesh","year":"2003","journal-title":"MIS Q."},{"key":"ref_48","first-page":"291","article-title":"Towards an assistance strategy that reduces unnecessary collision alarms: An examination of the driver\u2019s perceived need for assistance","volume":"25","author":"Schmidt","year":"2018","journal-title":"J. Exp. Psychol. Appl."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Rettenmaier, M., Pietsch, M., Schmidtler, J., and Bengler, K. (2019, January 9\u201312). Passing through the Bottleneck-The Potential of External Human-Machine Interfaces. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8814082"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/3\/173\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:11:04Z","timestamp":1760173864000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/3\/173"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,24]]},"references-count":49,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["info11030173"],"URL":"https:\/\/doi.org\/10.3390\/info11030173","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,24]]}}}