{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,17]],"date-time":"2026-05-17T04:24:55Z","timestamp":1778991895110,"version":"3.51.4"},"reference-count":52,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T00:00:00Z","timestamp":1626825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Background: As Automated Driving Systems (ADS) technology gets assimilated into the market, the driver\u2019s obligation will be changed to a supervisory role. A key point to consider is the driver\u2019s engagement in the secondary task to maintain the driver\/user in the control loop. This paper aims to monitor driver engagement with a game and identify any impacts the task has on hazard recognition. Methods: We designed a driving simulation using Unity3D and incorporated three tasks: No-task, AR-Video, and AR-Game tasks. The driver engaged in an AR object interception game while monitoring the road for threatening road scenarios. Results: There was a significant difference in the tasks (F(2,33) = 4.34, p = 0.0213), identifying the game-task as significant with respect to reaction time and ideal for the present investigation. Game scoring followed three profiles\/phases: learning, saturation, and decline profile. From the profiles, it is possible to quantify\/infer drivers\u2019 engagement with the game task. Conclusion: The paper proposes alternative monitoring that has utility, i.e., entertaining the user. Further experiments with AR-Games focusing on the real-world car environment will be performed to confirm the performance following the recommendations derived from the current test.<\/jats:p>","DOI":"10.3390\/mti5080040","type":"journal-article","created":{"date-parts":[[2021,7,22]],"date-time":"2021-07-22T22:35:31Z","timestamp":1626993331000},"page":"40","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["User Monitoring in Autonomous Driving System Using Gamified Task: A Case for VR\/AR In-Car Gaming"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3343-3614","authenticated-orcid":false,"given":"Joseph K.","family":"Muguro","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan"},{"name":"School of Engineering, Dedan Kimathi University of Technology, Private Bag, Nyeri 10143, Kenya"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0502-5453","authenticated-orcid":false,"given":"Pringgo Widyo","family":"Laksono","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan"},{"name":"Industrial Engineering, Universitas Sebelas Maret, Surakarta 57126, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuta","family":"Sasatake","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kojiro","family":"Matsushita","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7806-8492","authenticated-orcid":false,"given":"Minoru","family":"Sasaki","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,21]]},"reference":[{"key":"ref_1","unstructured":"Anderson, J.M., Kalra, N., Stanley, K.D., Sorensen, P., Samaras, C., and Oluwatola, O.A. (2014). Brief History and Current State of Autonomous Vehicles. Auton. Veh. Technol., 55\u201374."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"117","DOI":"10.3389\/frobt.2019.00117","article-title":"Pedestrian Trust in Automated Vehicles: Role of Traffic Signal and AV Driving Behavior","volume":"6","author":"Jayaraman","year":"2019","journal-title":"Front. Robot. AI"},{"key":"ref_3","unstructured":"Standard, S. (2020, December 17). J3016B: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles-SAE International. Available online: https:\/\/www.sae.org\/standards\/content\/j3016_201806\/."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Radlmayr, J., Gold, C., Lorenz, L., Farid, M., and Bengler, K. (2014, January 27\u201331). How Traffic Situations and Non-Driving Related Tasks Affect the Take-Over Quality in Highly Automated Driving. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, San Diego, CA, USA.","DOI":"10.1177\/1541931214581434"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Dingus, T.A., Klauer, S.G., Neale, V.L., Petersen, A., Lee, S.E., Sudweeks, J., Perez, M.A., Hankey, J., Ramsey, D., and Gupta, S. (2006). The 100-Car Naturalistic Driving Study: Phase II\u2014Results of the 100-Car Field Experiment, PsycEXTRA Dataset; Department of Transportation. National Highway Traffic Safety Administration.","DOI":"10.1037\/e624282011-001"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"105266","DOI":"10.1016\/j.aap.2019.105266","article-title":"How does drivers\u2019 visual search change as a function of experience? A systematic review and meta-analysis","volume":"132","author":"Robbins","year":"2019","journal-title":"Accid. Anal. Prev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1016\/j.aap.2016.04.002","article-title":"Is take-over time all that matters? The impact of visual-cognitive load on driver take-over quality after conditionally automated driving","volume":"92","author":"Zeeb","year":"2016","journal-title":"Accid. Anal. Prev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1016\/j.trf.2019.06.008","article-title":"Response times and gaze behavior of truck drivers in time critical conditional automated driving take-overs","volume":"64","author":"Lotz","year":"2019","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1676","DOI":"10.1177\/1541931215591362","article-title":"Distraction Becomes Engagement in Automated Driving","volume":"59","author":"Miller","year":"2015","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2403","DOI":"10.1016\/j.promfg.2015.07.499","article-title":"Vigilance Decrement and Passive Fatigue Caused by Monotony in Automated Driving","volume":"3","author":"Cingel","year":"2015","journal-title":"Procedia Manuf."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1016\/j.tra.2018.05.003","article-title":"Psychophysical assessment of a driver\u2019s mental state in autonomous vehicles","volume":"124","author":"Arakawa","year":"2019","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.aap.2018.03.013","article-title":"Asleep at the automated wheel\u2014Sleepiness and fatigue during highly automated driving","volume":"126","author":"Vogelpohl","year":"2019","journal-title":"Accid. Anal. Prev."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.trf.2015.06.003","article-title":"Are we there yet? Trip duration and mood during travel","volume":"33","author":"Morris","year":"2015","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.tra.2018.12.015","article-title":"How do activities conducted while commuting influence mode choice? Using revealed preference models to inform public transportation advantage and autonomous vehicle scenarios","volume":"124","author":"Malokin","year":"2019","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1016\/j.chb.2017.06.019","article-title":"From road distraction to safe driving: Evaluating the effects of boredom and gamification on driving behaviour, physiological arousal, and subjective experience","volume":"75","author":"Steinberger","year":"2017","journal-title":"Comput. Hum. Behav."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/j.aap.2017.04.017","article-title":"Take-over performance in evasive manoeuvres","volume":"106","author":"Happee","year":"2017","journal-title":"Accid. Anal. Prev."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lakier, M., Nacke, L.E., Igarashi, T., and Vogel, D. (2019, January 22\u201325). Cross-Car, Multiplayer Games for Semi-Autonomous Driving. Proceedings of the Annual Symposium on Computer-Human Interaction in Play, Barcelona, Spain.","DOI":"10.1145\/3311350.3347166"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Broy, N., Goebl, S., Hauder, M., Kothmayr, T., Kugler, M., Reinhart, F., Salfer, M., Schlieper, K., and Andr\u00e9, E. (2011, January 30). A cooperative in-car game for heterogeneous players. Proceedings of the 3rd International Conference on Intelligent Information Processing, Salzburg, Austria.","DOI":"10.1145\/2381416.2381443"},{"key":"ref_19","first-page":"1","article-title":"Driving with the Fishes","volume":"2","author":"Paredes","year":"2018","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hock, P., Benedikter, S., Gugenheimer, J., and Rukzio, E. (2017, January 6\u201311). Carvr: Enabling in-car virtual reality entertainment. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3025665"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1016\/j.trf.2019.02.020","article-title":"From car sickness to autonomous car sickness: A review","volume":"62","author":"Iskander","year":"2019","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"McGill, M., Ng, A., and Brewster, S. (2017, January 6\u201311). I am the passenger: How visual motion cues can influence sickness for in-car VR. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3026046"},{"key":"ref_23","unstructured":"(2020, December 17). Audi, Holoride: Virtual Reality Meets the Real World|audi.com. Available online: https:\/\/www.audi.com\/en\/experience-audi\/mobility-and-trends\/digitalization\/holoride-virtual-reality-meets-the-real-world.html."},{"key":"ref_24","unstructured":"Genty, E. (2020, December 18). Renault SYMBIOZ Demo Car: Technologies for Traveling Well-Being\u2014Groupe Renault. Available online: https:\/\/group.renault.com\/en\/news-on-air\/news\/renault-symbioz-demo-car-technologies-for-traveling-well-being\/."},{"key":"ref_25","unstructured":"Brunnberg, L., Juhlin, O., and Gustafsson, A. (2017, January 28\u201330). Games for passengers. Proceedings of the 4th International Conference on Movement Computing, London, UK."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3161176","article-title":"On Active Passengering","volume":"1","author":"Matsumura","year":"2018","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.ijcci.2015.03.001","article-title":"In-car game design for children: Promoting interactions inside and outside the car","volume":"2","author":"Zuckerman","year":"2014","journal-title":"Int. J. Child-Comput. Interact."},{"key":"ref_28","unstructured":"Statt, N. (2020, December 17). Tesla Arcade Hands-On: Using a Model 3 Steering Wheel as a Game Controller\u2014The Verge. Available online: https:\/\/www.theverge.com\/2019\/6\/18\/18684297\/tesla-arcade-model-3-hands-on-beach-buggy-racing-2-demo-electric-cars."},{"key":"ref_29","unstructured":"Schrader, S. (2020, December 17). The New Mercedes-Benz CLA Lets You Play Mario Kart on Its Infotainment Screen. Available online: https:\/\/www.thedrive.com\/news\/26761\/the-new-mercedes-benz-cla-lets-you-play-mario-kart-on-its-infotainment-screen."},{"key":"ref_30","unstructured":"Tassi, P. (2020, December 17). Nintendo Forms Unusual Partnership with Mercedes for \u2018Mario Kart 8\u2032 DLC. Available online: https:\/\/www.forbes.com\/sites\/insertcoin\/2014\/05\/29\/nintendo-forms-unusual-partnership-with-mercedes-for-mario-kart-8-dlc\/?sh=4836785b1a67."},{"key":"ref_31","unstructured":"CNN (2020, December 17). Amazon\u2019s Zoox Reveals Its Robotaxi\u2014CNN, CNN News. Available online: https:\/\/edition.cnn.com\/2020\/12\/14\/cars\/zoox-amazon-robotaxi\/index.html."},{"key":"ref_32","unstructured":"GM (2020, December 17). Chevrolet EN-V 2.0 Vehicle Sharing Pilot Program Concludes, GM Corporate Newsroom. Available online: https:\/\/media.gm.com\/media\/cn\/en\/gm\/news.detail.html\/content\/Pages\/news\/cn\/en\/2017\/May\/0505_Chevrolet-EN-V.html."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1007\/s10055-019-00420-x","article-title":"Challenges in passenger use of mixed reality headsets in cars and other transportation","volume":"24","author":"McGill","year":"2020","journal-title":"Virtual Real."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"200020","DOI":"10.1016\/j.gvc.2021.200020","article-title":"Virtual reality game level layout design for real environment constraints","volume":"4","author":"Liu","year":"2021","journal-title":"Graph. Vis. Comput."},{"key":"ref_35","first-page":"1","article-title":"Effects of non-driving related tasks on mental workload and take-over times during conditional automated driving","volume":"13","author":"Hetfleisch","year":"2021","journal-title":"Eur. Transp. Res. Rev."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1016\/j.trf.2018.07.025","article-title":"Situation awareness increases when drivers have more time to take over the wheel in a Level 3 automated car: A simulator study","volume":"58","author":"Vlakveld","year":"2018","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lindemann, P., Muller, N., and Rigolll, G. (2019, January 9\u201312). Exploring the Use of Augmented Reality Interfaces for Driver Assistance in Short-Notice Takeovers. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8814237"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Lindemann, P., Lee, T.-Y., and Rigoll, G. (2018, January 16\u201320). Supporting Driver Situation Awareness for Autonomous Urban Driving with an Augmented-Reality Windshield Display. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Munich, Germany.","DOI":"10.1109\/ISMAR-Adjunct.2018.00104"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.apergo.2018.04.006","article-title":"An interview study exploring Tesla drivers\u2019 behavioural adaptation","volume":"72","author":"Lin","year":"2018","journal-title":"Appl. Ergon."},{"key":"ref_40","unstructured":"BBCNews (2020, December 17). Tesla in Fatal California Crash Was on Autopilot\u2014BBC News. Available online: https:\/\/www.bbc.com\/news\/world-us-canada-43604440."},{"key":"ref_41","unstructured":"Lubben, A. (2020, December 17). Self-Driving Uber Killed a Pedestrian as Human Safety Driver Watched. Available online: https:\/\/www.vice.com\/en\/article\/kzxq3y\/self-driving-uber-killed-a-pedestrian-as-human-safety-driver-watched."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.annals.2018.10.009","article-title":"Autonomous vehicles and the future of urban tourism","volume":"74","author":"Cohen","year":"2019","journal-title":"Ann. Tour. Res."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.trf.2016.07.009","article-title":"The impact of roadside landscape colors on driver\u2019s mean heart rate considering driving time","volume":"42","author":"Wang","year":"2016","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1016\/S0001-4575(02)00014-3","article-title":"Monotony of road environment and driver fatigue: A simulator study","volume":"35","author":"Thiffault","year":"2003","journal-title":"Accid. Anal. Prev."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.tra.2018.03.005","article-title":"It takes two to Tango: Automated vehicles and human beings do the dance of driving\u2014Four social considerations for policy","volume":"122","author":"Straub","year":"2019","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.trf.2019.08.013","article-title":"Real autonomous driving from a passenger\u2019s perspective: Two experimental investigations using gaze behaviour and trust ratings in field and simulator","volume":"66","author":"Strauch","year":"2019","journal-title":"Transp. Res. Part F: Traffic Psychol. Behav."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.trc.2017.01.001","article-title":"Are you in the loop? Using gaze dispersion to understand driver visual attention during vehicle automation","volume":"76","author":"Louw","year":"2017","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Feldh\u00fctter, A., Gold, C., Schneider, S., and Bengler, K. (2017). How the Duration of Automated Driving Influences Take-Over Perfor-mance and Gaze Behavior. Advances in Ergonomic Design of Systems, Products and Processes, Springer.","DOI":"10.1007\/978-3-662-53305-5_22"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Peillard, E., Thebaud, T., Normand, J.-M., Argelaguet, F., Moreau, G., Lecuyer, A., and Norrnand, J.-M. (2019, January 23\u201327). Virtual Objects Look Farther on the Sides: The Anisotropy of Distance Perception in Virtual Reality. Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan.","DOI":"10.1109\/VR.2019.8797826"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Eghbali, P., V\u00e4\u00e4n\u00e4nen, K., and Jokela, T. (2019, January 26\u201329). Social acceptability of virtual reality in public spaces: Experiential factors and design recom-mendations. Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia, Pisa, Italy.","DOI":"10.1145\/3365610.3365647"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Li, J., George, C., Ngao, A., Holl\u00e4nder, K., Mayer, S., and Butz, A. (2021). Rear-Seat Productivity in Virtual Reality: Investigating VR Interaction in the Confined Space of a Car. Multimodal Technol. Interact., 5.","DOI":"10.3390\/mti5040015"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"192","DOI":"10.3182\/20100831-4-FR-2021.00035","article-title":"Effect of Driver\u2019s Head Tilt Strategy on Motion Sickness Incidence","volume":"43","author":"Wada","year":"2010","journal-title":"IFAC Proc. Vol."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/5\/8\/40\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:32:47Z","timestamp":1760164367000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/5\/8\/40"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,21]]},"references-count":52,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["mti5080040"],"URL":"https:\/\/doi.org\/10.3390\/mti5080040","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202106.0459.v1","asserted-by":"object"}]},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,21]]}}}