{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:38:13Z","timestamp":1760240293077,"version":"build-2065373602"},"reference-count":48,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,4,29]],"date-time":"2019-04-29T00:00:00Z","timestamp":1556496000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>The development of automated driving will profit from an agreed-upon methodology to evaluate human\u2013machine interfaces. The present study examines the role of feedback on interaction performance provided directly to participants when interacting with driving automation (i.e., perceived ease of use). In addition, the development of ratings itself over time and use case specificity were examined. In a driving simulator study, N = 55 participants completed several transitions between Society of Automotive Engineers (SAE) level 0, level 2, and level 3 automated driving. One half of the participants received feedback on their interaction performance immediately after each use case, while the other half did not. As expected, the results revealed that participants judged the interactions to become easier over time. However, a use case specificity was present, as transitions to L0 did not show effects over time. The role of feedback also depended on the respective use case. We observed more conservative evaluations when feedback was provided than when it was not. The present study supports the application of perceived ease of use as a diagnostic measure in interaction with automated driving. Evaluations of interfaces can benefit from supporting feedback to obtain more conservative results.<\/jats:p>","DOI":"10.3390\/mti3020029","type":"journal-article","created":{"date-parts":[[2019,4,29]],"date-time":"2019-04-29T21:49:47Z","timestamp":1556574587000},"page":"29","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Tell Them How They Did: Feedback on Operator Performance Helps Calibrate Perceived Ease of Use in Automated Driving"],"prefix":"10.3390","volume":"3","author":[{"given":"Yannick","family":"Forster","sequence":"first","affiliation":[{"name":"Bayerische Motoren Werke (BMW) Group, Knorrstr. 147, 80937 Munich, Germany"},{"name":"Department of Psychology, Chemnitz University of Technology, 09111 Chemnitz, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4061-8273","authenticated-orcid":false,"given":"Sebastian","family":"Hergeth","sequence":"additional","affiliation":[{"name":"Bayerische Motoren Werke (BMW) Group, Knorrstr. 147, 80937 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1712-6473","authenticated-orcid":false,"given":"Frederik","family":"Naujoks","sequence":"additional","affiliation":[{"name":"Bayerische Motoren Werke (BMW) Group, Knorrstr. 147, 80937 Munich, Germany"}]},{"given":"Josef","family":"Krems","sequence":"additional","affiliation":[{"name":"Department of Psychology, Chemnitz University of Technology, 09111 Chemnitz, Germany"}]},{"given":"Andreas","family":"Keinath","sequence":"additional","affiliation":[{"name":"Bayerische Motoren Werke (BMW) Group, Knorrstr. 147, 80937 Munich, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2019,4,29]]},"reference":[{"key":"ref_1","unstructured":"SAE (2018). Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems, SAE International."},{"key":"ref_2","unstructured":"AAM (2006). Statement of Principles, Criteria and Verification Procedures on Driver Interactions with Advanced In-Vehicle Information and Communication Systems, Alliance of Automobile Manufactures."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1561\/1100000069","article-title":"Human-Machine Interaction for Vehicles: Review and Outlook","volume":"11","author":"Kun","year":"2017","journal-title":"FNT Hum. Comput. Interact."},{"key":"ref_4","unstructured":"NHTSA (2012). Visual-Manual NHTSA Driver Distraction Guidelines for In-Vehicle Electronic Devices."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1177\/0018720815625744","article-title":"Keep your scanners peeled: Gaze behavior as a measure of automation trust during highly automated driving","volume":"58","author":"Hergeth","year":"2016","journal-title":"Hum. Factors"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Forster, Y., Naujoks, F., and Neukum, A. (2017, January 11\u201314). Increasing anthropomorphism and trust in automated driving functions by adding speech output. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium, Redondo Beach, CA, USA.","DOI":"10.1109\/IVS.2017.7995746"},{"key":"ref_7","first-page":"431","article-title":"The effect of urgency take-over requests during highly automated driving under distraction conditions","volume":"7","author":"Naujoks","year":"2014","journal-title":"Adv. Hum. Asp. Transp."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Feldh\u00fctter, A., Segler, C., and Bengler, K. (2017, January 17\u201321). Does Shifting Between Conditionally and Partially Automated Driving Lead to a Loss of Mode Awareness?. Proceedings of the International Conference on Applied Human Factors and Ergonomics, Los Angeles, CA, USA.","DOI":"10.1007\/978-3-319-60441-1_70"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Forster, Y., Hergeth, S., Naujoks, F., and Krems, J.F. (2018, January 23\u201325). How Usability can Save the Day. Methodological Considerations for Making Automated Driving a Success Story. Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Toronto, ON, Canada.","DOI":"10.1145\/3239060.3239076"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Naujoks, F., Hergeth, S., Wiedemann, K., Schoemig, N., Forster, Y., and Keinath, A. (2019). Test Procedure for Evaluating the Human-Machine-Interface of Vehicles with Automated Driving. Traffic Injury Prev., in press.","DOI":"10.1080\/15389588.2019.1603374"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1006\/imms.1993.1032","article-title":"Evaluating user-computer interaction: A framework","volume":"38","author":"Sweeney","year":"1993","journal-title":"Int. J. Man. Mach. Stud."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Forster, Y., Hergeth, S., Naujoks, F., Krems, J.F., and Keinath, A. (2018). Self-Report Measures for the Evaluation of Human-Machine Interfaces in Automated Driving, Subimtted and under Review.","DOI":"10.1007\/s10111-019-00599-8"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1016\/j.trf.2019.02.013","article-title":"Learning to Use Automation: Behavioral Changes in Interaction with Automated Driving Systems","volume":"62","author":"Forster","year":"2019","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"319","DOI":"10.2307\/249008","article-title":"Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology","volume":"13","author":"Davis","year":"1989","journal-title":"MIS Q."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Forster, Y., Naujoks, F., and Neukum, A. (2016, January 24\u201326). Your Turn or My Turn? Design of a Human-Machine Interface for Conditional Automation. Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ann Arbor, MI, USA.","DOI":"10.1145\/3003715.3005463"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1016\/j.aap.2017.09.011","article-title":"Assessing the utility of TAM, TPB, and UTAUT for advanced driver assistance systems","volume":"108","author":"Rahman","year":"2017","journal-title":"Accid. Anal. Prev."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1016\/j.trf.2018.06.024","article-title":"User acceptance of automated shuttles in Berlin-Sch\u00f6neberg: A questionnaire study","volume":"58","author":"Nordhoff","year":"2018","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"R\u00f6del, C., Stadler, S., Meschtscherjakov, A., and Tscheligi, M. (2014, January 17\u201319). Towards Autonomous Cars: The Effect of Autonomy Levels on Acceptance and User Experience. Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Seattle, WA, USA.","DOI":"10.1145\/2667317.2667330"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1016\/j.trf.2014.04.009","article-title":"Intention to use a fully automated car. Attitudes and a priori acceptability","volume":"27","author":"Payre","year":"2014","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wintersberger, P., Riener, A., Schartm\u00fcller, C., Frison, A.-K., and Weigl, K. (2018, January 23\u201325). Let Me Finish before I Take Over: Towards Attention Aware Device Integration in Highly Automated Vehicles. Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Toronto, ON, Canada.","DOI":"10.1145\/3239060.3239085"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Frison, A.-K., Wintersberger, P., Riener, A., and Schartm\u00fcller, C. (2017, January 24\u201327). Driving Hotzenplotz: A Hybrid Interface for Vehicle Control Aiming to Maximize Pleasure in Highway Driving. Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Oldenburg, Germany.","DOI":"10.1145\/3122986.3123016"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1016\/j.trf.2017.12.005","article-title":"Predictive shared steering control for driver override in automated driving: A simulator study","volume":"61","author":"Guo","year":"2019","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Large, D., Burnett, G., Crundall, E., Lawson, G., and Skrypchuk, L. (2016, January 24\u201326). Twist It, Touch It, Push It, Swipe It: Evaluating Secondary Input Devices for Use with an Automotive Touchscreen HMI. Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ann Arbor, MI, USA.","DOI":"10.1145\/3003715.3005459"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/S0378-7206(98)00096-2","article-title":"The psychological origins of perceived usefulness and ease-of-use","volume":"35","author":"Karahanna","year":"1999","journal-title":"Inf. Manag."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Forster, Y., Hergeth, S., Naujoks, F., Beggiato, M., Krems, J.F., and Keinath, A. (2019, January 24\u201327). Learning and Development of Mental Models in Interaction with Driving Automation: A Simulator Study. Proceedings of the 10th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design, Santa Fe, NM, USA.","DOI":"10.17077\/drivingassessment.1724"},{"key":"ref_26","unstructured":"Hart, C.A. (2019, March 14). Self Driving Safety Steps into the Unknown. Available online: http:\/\/www.thedrive.com\/tech\/26896\/self-driving-safety-steps-into-the-unknown."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.trf.2015.10.005","article-title":"Learning and development of trust, acceptance and the mental model of ACC. A longitudinal on-road study","volume":"35","author":"Beggiato","year":"2015","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Forster, Y., Hergeth, S., Naujoks, F., and Krems, J.F. (2018, January 23\u201325). Unskilled and Unaware: Subpar Users of Automated Driving Systems Make Spurious Decisions. Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Toronto, ON, Canada.","DOI":"10.1145\/3239092.3265960"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"888","DOI":"10.1037\/0033-2909.84.5.888","article-title":"Attitude-behavior relations: A theoretical analysis and review of empirical research","volume":"84","author":"Ajzen","year":"1977","journal-title":"Psychol. Bull."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hancock, P.A., and Matthews, G. (2018). Workload and Performance: Associations, Insensitivities, and Dissociations. Hum. Factors.","DOI":"10.1177\/0018720818809590"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1145\/175276.175282","article-title":"Measuring Usability: Preference vs. Performance","volume":"37","author":"Nielsen","year":"1994","journal-title":"Commun. ACM"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"229","DOI":"10.3389\/fpsyg.2012.00229","article-title":"Calibration research: Where do we go from here?","volume":"3","author":"Bol","year":"2012","journal-title":"Front. Psychol."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zimmerman, B.J. (2000). Attaining self-regulation: A social cognitive perspective. Handbook of Self-Regulation, Elsevier.","DOI":"10.1016\/B978-012109890-2\/50031-7"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1124","DOI":"10.1126\/science.185.4157.1124","article-title":"Judgment under uncertainty: Heuristics and biases","volume":"185","author":"Tversky","year":"1974","journal-title":"Science"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1016\/j.ergon.2008.01.007","article-title":"The impact of cognitive feedback on judgment performance and trust with decision aids","volume":"38","author":"Seong","year":"2008","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Helldin, T., Falkman, G., Riveiro, M., and Davidsson, S. (2013, January 27\u201330). Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving. Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Eindhoven, The Nethderlands.","DOI":"10.1145\/2516540.2516554"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1518\/hfes.46.1.50.30392","article-title":"Trust in automation: Designing for appropriate reliance","volume":"46","author":"Lee","year":"2004","journal-title":"Hum. Factors"},{"key":"ref_38","unstructured":"ISO (2012). Road Vehicles\u2014Ergonomic Aspects of Transport Information and Control Systems\u2014Calibration Tasks for Methods Which Assess Driver Demand Due to the Use of In-Vehicle Systems, ISO."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.trf.2015.10.022","article-title":"Controllability of Partially Automated Driving functions\u2013Does it matter whether drivers are allowed to take their hands off the steering wheel?","volume":"35","author":"Naujoks","year":"2015","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_40","unstructured":"Manca, L., de Winter, J.C.F., and Happee, R. (2015, January 1\u20133). Visual Displays for Automated Driving: A Survey. Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Nottingham, UK."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Naujoks, F., Hergeth, S., Wiedemann, K., Schoemig, N., and Keinath, A. (2018, January 1\u20135). Use Cases for Assessing, Testing, and Validating the Human Machine Interface of Automated Driving Systems. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Philadelphia, PA, USA.","DOI":"10.1177\/1541931218621426"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Wickens, C.D., Hollands, J.G., Banbury, S., and Parasuraman, R. (2015). Engineering Psychology & Human Performance, Psychology Press.","DOI":"10.4324\/9781315665177"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1037\/0003-066X.39.4.377","article-title":"Learning Outcomes and Their Effects. Useful Categories of Human Performance","volume":"39","author":"Gagne","year":"1984","journal-title":"Am. Psychol."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1177\/0018720816685832","article-title":"Takeover Time in Highly Automated Vehicles: Noncritical Transitions to and From Manual Control","volume":"59","author":"Eriksson","year":"2017","journal-title":"Hum. Factors"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1207\/S15327590IJHC143&4_11","article-title":"Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies","volume":"14","author":"Lewis","year":"2002","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1177\/0018720816678714","article-title":"Prior familiarization with takeover requests affects drivers\u2019 takeover performance and automation trust","volume":"59","author":"Hergeth","year":"2017","journal-title":"Hum. Factors"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1177\/0018720815612319","article-title":"Fully automated driving: Impact of trust and practice on manual control recovery","volume":"58","author":"Payre","year":"2016","journal-title":"Hum. Factors"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.aap.2018.06.003","article-title":"Get ready for automated driving using Virtual Reality","volume":"118","author":"Sportillo","year":"2018","journal-title":"Accid. Anal. Prev."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/3\/2\/29\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:48:04Z","timestamp":1760186884000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/3\/2\/29"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,29]]},"references-count":48,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,6]]}},"alternative-id":["mti3020029"],"URL":"https:\/\/doi.org\/10.3390\/mti3020029","relation":{},"ISSN":["2414-4088"],"issn-type":[{"type":"electronic","value":"2414-4088"}],"subject":[],"published":{"date-parts":[[2019,4,29]]}}}