{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T05:43:59Z","timestamp":1780638239220,"version":"3.54.1"},"reference-count":44,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2022,12,19]],"date-time":"2022-12-19T00:00:00Z","timestamp":1671408000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Spanish MICINN\/FEDER through the Artificial Intelligence based modular Architecture Implementation and Validation for Autonomous Driving (AIVATAR) project","award":["PID2021-126623OB-I00"],"award-info":[{"award-number":["PID2021-126623OB-I00"]}]},{"name":"Spanish MICINN\/FEDER through the Artificial Intelligence based modular Architecture Implementation and Validation for Autonomous Driving (AIVATAR) project","award":["P2018\/NMT-4331"],"award-info":[{"award-number":["P2018\/NMT-4331"]}]},{"name":"RoboCity2030-DIH-CM project","award":["PID2021-126623OB-I00"],"award-info":[{"award-number":["PID2021-126623OB-I00"]}]},{"name":"RoboCity2030-DIH-CM project","award":["P2018\/NMT-4331"],"award-info":[{"award-number":["P2018\/NMT-4331"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous vehicles are the near future of the automobile industry. However, until they reach Level 5, humans and cars will share this intermediate future. Therefore, studying the transition between autonomous and manual modes is a fascinating topic. Automated vehicles may still need to occasionally hand the control to drivers due to technology limitations and legal requirements. This paper presents a study of driver behaviour in the transition between autonomous and manual modes using a CARLA simulator. To our knowledge, this is the first take-over study with transitions conducted on this simulator. For this purpose, we obtain driver gaze focalization and fuse it with the road\u2019s semantic segmentation to track to where and when the user is paying attention, besides the actuators\u2019 reaction-time measurements provided in the literature. To track gaze focalization in a non-intrusive and inexpensive way, we use a method based on a camera developed in previous works. We devised it with the OpenFace 2.0 toolkit and a NARMAX calibration method. It transforms the face parameters extracted by the toolkit into the point where the user is looking on the simulator scene. The study was carried out by different users using our simulator, which is composed of three screens, a steering wheel and pedals. We distributed this proposal in two different computer systems due to the computational cost of the simulator based on the CARLA simulator. The robot operating system (ROS) framework is in charge of the communication of both systems to provide portability and flexibility to the proposal. Results of the transition analysis are provided using state-of-the-art metrics and a novel driver situation-awareness metric for 20 users in two different scenarios.<\/jats:p>","DOI":"10.3390\/s22249993","type":"journal-article","created":{"date-parts":[[2022,12,19]],"date-time":"2022-12-19T09:31:01Z","timestamp":1671442261000},"page":"9993","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Driver Take-Over Behaviour Study Based on Gaze Focalization and Vehicle Data in CARLA Simulator"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5101-0485","authenticated-orcid":false,"given":"Javier","family":"Araluce","sequence":"first","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, 28805 Alcal\u00e1 de Henares, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0087-3077","authenticated-orcid":false,"given":"Luis M.","family":"Bergasa","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, 28805 Alcal\u00e1 de Henares, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8875-1866","authenticated-orcid":false,"given":"Manuel","family":"Oca\u00f1a","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, 28805 Alcal\u00e1 de Henares, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8145-9045","authenticated-orcid":false,"given":"Elena","family":"L\u00f3pez-Guill\u00e9n","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, 28805 Alcal\u00e1 de Henares, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rodrigo","family":"Guti\u00e9rrez-Moreno","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, 28805 Alcal\u00e1 de Henares, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J. Felipe","family":"Arango","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, 28805 Alcal\u00e1 de Henares, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,19]]},"reference":[{"key":"ref_1","unstructured":"Jimenez, F. (2017). Intelligent Vehicles: Enabling Technologies and Future Developments, Butterworth-Heinemann."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1016\/j.tra.2018.05.005","article-title":"A taxonomy of autonomous vehicle handover situations","volume":"124","author":"McCall","year":"2019","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4318","DOI":"10.1109\/TITS.2019.2939676","article-title":"A Dual-Cameras-Based Driver Gaze Mapping System with an Application on Non-Driving Activities Monitoring","volume":"21","author":"Yang","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_4","unstructured":"Santos, G.H.G., and Larocca, A.P.C. (2019, January 10\u201314). Drivers Take-Over Performance from Partial Automation to Manual Driving. Proceedings of the 2019 33th Congresso de Pesquisa e Ensino em Transporte da ANPET, Balne\u00e1rio Cambori\u00fa, Brasil."},{"key":"ref_5","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017). CARLA: An open urban driving simulator. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Baltrusaitis, T., Zadeh, A., Lim, Y.C., and Morency, L.P. (2018, January 15\u201319). Openface 2.0: Facial behavior analysis toolkit. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00019"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Araluce, J., Bergasa, L.M., Oca\u00f1a, M., L\u00f3pez-Guill\u00e9n, E., Revenga, P.A., Arango, J.F., and P\u00e9rez, O. (2021). Gaze Focalization System for Driving Applications Using OpenFace 2.0 Toolkit with NARMAX Algorithm in Accidental Scenarios. Sensors, 21.","DOI":"10.3390\/s21186262"},{"key":"ref_8","unstructured":"Bergh\u00f6fer, F.L., Purucker, C., Naujoks, F., Wiedemann, K., and Marberger, C. (2018, January 1\u20135). Prediction of take-over time demand in conditionally automated driving-results of a real world driving study. Proceedings of the Human Factors and Ergonomics Society Europe, Philadelphia, PA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lotz, A., and Weissenberger, S. (2018). Predicting take-over times of truck drivers in conditional autonomous driving. International Conference on Applied Human Factors and Ergonomics, Springer.","DOI":"10.1007\/978-3-319-93885-1_30"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1109\/TIV.2019.2955364","article-title":"Looking at the driver\/rider in autonomous vehicles to predict take-over readiness","volume":"5","author":"Deo","year":"2019","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Du, N., Zhou, F., Pulver, E., Tilbury, D., Robert, L.P., Pradhan, A.K., and Yang, X.J. (2020, January 25\u201330). Predicting Takeover Performance in Conditionally Automated Driving. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3334480.3382963"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"105748","DOI":"10.1016\/j.aap.2020.105748","article-title":"Predicting driver takeover performance in conditionally automated driving","volume":"148","author":"Du","year":"2020","journal-title":"Accid. Anal. Prev."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Pakdamanian, E., Sheng, S., Baee, S., Heo, S., Kraus, S., and Feng, L. (2021, January 8\u201313). Deeptake: Prediction of driver takeover behavior using multimodal data. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan.","DOI":"10.1145\/3411764.3445563"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.aap.2017.03.011","article-title":"Transition to manual: Comparing simulator with on-road control transitions","volume":"102","author":"Eriksson","year":"2017","journal-title":"Accid. Anal. Prev."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1016\/j.trf.2014.09.005","article-title":"Transition to manual: Driver behaviour when resuming control from a highly automated vehicle","volume":"27","author":"Merat","year":"2014","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.trc.2020.01.006","article-title":"Examining the effects of emotional valence and arousal on takeover performance in conditionally automated driving","volume":"112","author":"Du","year":"2020","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1016\/j.trf.2019.09.001","article-title":"How does training effect users\u2019 attitudes and skills needed for highly automated driving?","volume":"66","author":"Ebnali","year":"2019","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/j.aap.2015.02.023","article-title":"What determines the take-over time? An integrated model approach of driver take-over after automated driving","volume":"78","author":"Zeeb","year":"2015","journal-title":"Accid. Anal. Prev."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Rangesh, A., Deo, N., Greer, R., Gunaratne, P., and Trivedi, M.M. (2021). Autonomous Vehicles that Alert Humans to Take-Over Controls: Modeling with Real-World Data. arXiv.","DOI":"10.1109\/ITSC48978.2021.9564434"},{"key":"ref_20","first-page":"431","article-title":"The effect of urgency of take-over requests during highly automated driving under distraction conditions","volume":"7","author":"Naujoks","year":"2014","journal-title":"Adv. Hum. Asp. Transp."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1177\/1541931218621005","article-title":"The effect of whole-body haptic feedback on driver\u2019s perception in negotiating a curve","volume":"Volume 62","author":"Pakdamanian","year":"2018","journal-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"987","DOI":"10.1080\/10447318.2017.1404778","article-title":"The effects of vibration patterns of take-over request and non-driving tasks on taking-over control of automated vehicles","volume":"34","author":"Wan","year":"2018","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"913","DOI":"10.3758\/s13428-013-0422-2","article-title":"PyGaze: An open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments","volume":"46","author":"Dalmaijer","year":"2013","journal-title":"Behav. Res. Methods"},{"key":"ref_24","first-page":"2055668318773991","article-title":"Head-mounted eye gaze tracking devices: An overview of modern devices and recent advances","volume":"5","author":"Cognolato","year":"2018","journal-title":"J. Rehabil. Assist. Technol. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Shen, J., Zafeiriou, S., Chrysos, G.G., Kossaifi, J., Tzimiropoulos, G., and Pantic, M. (2015, January 7\u201313). The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results. Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), Santiago, Chile.","DOI":"10.1109\/ICCVW.2015.132"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Xia, Y., Zhang, D., Kim, J., Nakayama, K., Zipser, K., and Whitney, D. (2018, January 2\u20136). Predicting driver attention in critical situations. Proceedings of the Asian Conference on Computer Vision, Perth, Australia.","DOI":"10.1007\/978-3-030-20873-8_42"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mizuno, N., Yoshizawa, A., Hayashi, A., and Ishikawa, T. (2017, January 26\u201328). Detecting driver\u2019s visual attention area by using vehicle-mounted device. Proceedings of the 2017 IEEE 16th International Conference on Cognitive Informatics & Cognitive Computing (ICCI* CC), Oxford, UK.","DOI":"10.1109\/ICCI-CC.2017.8109772"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2014","DOI":"10.1109\/TITS.2015.2396031","article-title":"Driver gaze tracking and eyes off the road detection system","volume":"16","author":"Vicente","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Naqvi, R.A., Arsalan, M., Batchuluun, G., Yoon, H.S., and Park, K.R. (2018). Deep learning-based gaze detection system for automobile drivers using a NIR camera sensor. Sensors, 18.","DOI":"10.3390\/s18020456"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1167","DOI":"10.1109\/TITS.2012.2187517","article-title":"Gaze fixation system for the evaluation of driver distractions induced by IVIS","volume":"13","author":"Bergasa","year":"2012","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hofbauer, M., Kuhn, C.B., P\u00fcttner, L., Petrovic, G., and Steinbach, E. (2020, January 2\u20134). Measuring Driver Situation Awareness Using Region-of-Interest Prediction and Eye Tracking. Proceedings of the 2020 IEEE International Symposium on Multimedia (ISM), Naples, Italy.","DOI":"10.1109\/ISM.2020.00022"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Langner, T., Seifert, D., Fischer, B., Goehring, D., Ganjineh, T., and Rojas, R. (2016, January 16\u201321). Traffic awareness driver assistance based on stereovision, eye-tracking, and head-up display. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487485"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Mori, M., Miyajima, C., Angkititrakul, P., Hirayama, T., Li, Y., Kitaoka, N., and Takeda, K. (2012, January 16\u201319). Measuring driver awareness based on correlation between gaze behavior and risks of surrounding vehicles. Proceedings of the 2012 15th International IEEE Conference on Intelligent Transportation Systems, Anchorage, AK, USA.","DOI":"10.1109\/ITSC.2012.6338802"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., and Ng, A.Y. (2009, January 12\u201317). ROS: An open-source Robot Operating System. Proceedings of the ICRA Workshop on Open Source Software, Kobe, Japan.","DOI":"10.1109\/MRA.2010.936956"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1559","DOI":"10.1080\/00207728808964057","article-title":"Identification of non-linear output-affine systems using an orthogonal least-squares algorithm","volume":"19","author":"Billings","year":"1988","journal-title":"Int. J. Syst. Sci."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Fang, J., Yan, D., Qiao, J., Xue, J., Wang, H., and Li, S. (2019, January 27\u201330). DADA-2000: Can Driving Accident be Predicted by Driver Attention\u0192 Analyzed by A Benchmark. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, NZ, USA.","DOI":"10.1109\/ITSC.2019.8917218"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Nagaraju, D., Ansah, A., Ch, N.A.N., Mills, C., Janssen, C.P., Shaer, O., and Kun, A.L. (2021, January 9\u201314). How will drivers take back control in automated vehicles?. A driving simulator test of an interleaving framework. In Proceedings of the 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Leeds, UK.","DOI":"10.1145\/3409118.3475128"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1109\/THMS.2018.2844251","article-title":"The Effects of Lead Time of Take-Over Request and Nondriving Tasks on Taking-Over Control of Automated Vehicles","volume":"48","author":"Wan","year":"2018","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1109\/TITS.2017.2750080","article-title":"Erfnet: Efficient residual factorized convnet for real-time semantic segmentation","volume":"19","author":"Romera","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_40","unstructured":"Chen, L., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zabihi, S., Zabihi, S., Beauchemin, S.S., and Bauer, M.A. (2017, January 11\u201314). Detection and recognition of traffic signs inside the attentional visual field of drivers. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995781"},{"key":"ref_42","unstructured":"(2022, December 12). ScenarioRunner Is a Module That Allows Traffic Scenario Definition and Execution for the CARLA Simulator. Available online: https:\/\/carla-scenariorunner.readthedocs.io\/en\/latest\/."},{"key":"ref_43","unstructured":"Blackman, S., and Popoli, R. (1999). Design and Analysis of Modern Tracking Systems, Artech House."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Hooey, B.L., Gore, B.F., Wickens, C.D., Scott-Nash, S., Socash, C., Salud, E., and Foyle, D.C. (2011). Modeling pilot situation awareness. Human Modelling in Assisted Transportation, Springer.","DOI":"10.1007\/978-88-470-1821-1_22"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/24\/9993\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:43:47Z","timestamp":1760147027000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/24\/9993"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,19]]},"references-count":44,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["s22249993"],"URL":"https:\/\/doi.org\/10.3390\/s22249993","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,19]]}}}