{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T17:41:38Z","timestamp":1770918098193,"version":"3.50.1"},"reference-count":25,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,1,28]],"date-time":"2023-01-28T00:00:00Z","timestamp":1674864000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Hyundai Mobis","award":["092021C26S03000"],"award-info":[{"award-number":["092021C26S03000"]}]},{"name":"Hyundai Mobis","award":["2022R1F1A1072626"],"award-info":[{"award-number":["2022R1F1A1072626"]}]},{"name":"Hyundai Mobis","award":["5199990814084"],"award-info":[{"award-number":["5199990814084"]}]},{"name":"Korea Institute of Police Technology (KIPoT)","award":["092021C26S03000"],"award-info":[{"award-number":["092021C26S03000"]}]},{"name":"Korea Institute of Police Technology (KIPoT)","award":["2022R1F1A1072626"],"award-info":[{"award-number":["2022R1F1A1072626"]}]},{"name":"Korea Institute of Police Technology (KIPoT)","award":["5199990814084"],"award-info":[{"award-number":["5199990814084"]}]},{"name":"National Research Foundation of Korea (NRF)","award":["092021C26S03000"],"award-info":[{"award-number":["092021C26S03000"]}]},{"name":"National Research Foundation of Korea (NRF)","award":["2022R1F1A1072626"],"award-info":[{"award-number":["2022R1F1A1072626"]}]},{"name":"National Research Foundation of Korea (NRF)","award":["5199990814084"],"award-info":[{"award-number":["5199990814084"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Driver\u2019s hands on\/off detection is very important in current autonomous vehicles for safety. Several studies have been conducted to create a precise algorithm. Although many studies have proposed various approaches, they have some limitations, such as robustness and reliability. Therefore, we propose a deep learning model that utilizes in-vehicle data. We also established a data collection system, which collects in-vehicle data that are auto-labeled for efficient and reliable data acquisition. For a robust system, we devised a confidence logic that prevents outliers\u2019 sway. To evaluate our model in more detail, we suggested a new metric to explain the events, considering state transitions. In addition, we conducted an extensive experiment on the new drivers to demonstrate our model\u2019s generalization ability. We verified that the proposed system achieved a better performance than in previous studies, by resolving their drawbacks. Our model detected hands on\/off transitions in 0.37 s on average, with an accuracy of 95.7%.<\/jats:p>","DOI":"10.3390\/s23031442","type":"journal-article","created":{"date-parts":[[2023,1,30]],"date-time":"2023-01-30T02:01:18Z","timestamp":1675044078000},"page":"1442","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Deep Learning-Based Driver\u2019s Hands on\/off Prediction System Using In-Vehicle Data"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1184-341X","authenticated-orcid":false,"given":"Hyeongoo","family":"Pyeon","sequence":"first","affiliation":[{"name":"Graduate School of Automotive Engineering, Kookmin University, Seoul 02707, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hanwul","family":"Kim","sequence":"additional","affiliation":[{"name":"Steering Control Logic Engineering Cell, Hyundai MOBIS Technical Center, Yongin-si 16891, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rak Chul","family":"Kim","sequence":"additional","affiliation":[{"name":"Graduate School of Automotive Engineering, Kookmin University, Seoul 02707, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0498-5631","authenticated-orcid":false,"given":"Geesung","family":"Oh","sequence":"additional","affiliation":[{"name":"Graduate School of Automotive Engineering, Kookmin University, Seoul 02707, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1917-699X","authenticated-orcid":false,"given":"Sejoon","family":"Lim","sequence":"additional","affiliation":[{"name":"Department of Automobile and IT Convergence, Kookmin University, Seoul 02707, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,28]]},"reference":[{"key":"ref_1","unstructured":"Cao, Y., Griffon, T., Fahrenkrog, F., Schneider, M., Naujoks, F., Tango, F., Wolter, S., Knapp, A., Page, Y., and Mallada, J. (2022, October 27). L3Pilot-Code of Practice for the Development of Automated Driving Functions; 2022. Available online: https:\/\/bast.opus.hbz-nrw.de\/opus45-bast\/frontdoor\/deliver\/index\/docId\/2630\/file\/L3Pilot_CoP-ADF_v1.0.pdf."},{"key":"ref_2","unstructured":"(2022, October 27). MotorTrend Mercedes-Benz Drive Pilot Autonomous First \u201cDrive\u201d: We Try a World\u2019s First Driverless System. Available online: https:\/\/www.motortrend.com\/news\/mercedes-benz-drive-pilot-eqs-autonomous-driverless-first-drive-review\/."},{"key":"ref_3","unstructured":"Mousel, T., and Treis, A. (2017, January 6). Hands Off Detection Requirements for UN R79 Regulated Lane Keeping Assist Systems. Proceedings of the 25th International Technical Conference on the Enhanced Safety of Vehicles (ESV) National Highway Traffic Safety Administration, Detroit, MI, USA."},{"key":"ref_4","unstructured":"Sakai, M., and Fuchs, R. (2022, October 27). Hands On\/Off Detection Based on EPS Sensors. Available online: https:\/\/www.jtekt.co.jp\/e\/engineering-journal\/assets\/1017\/1017e_06.pdf."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"772","DOI":"10.1109\/TITS.2019.2956598","article-title":"Driver intervention detection via real-time transfer function estimation","volume":"22","author":"Schinkel","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3444","DOI":"10.1109\/JSEN.2018.2805381","article-title":"Online sensing of human steering intervention torque for autonomous driving actuation systems","volume":"18","author":"Wang","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_7","unstructured":"Moreillon, M. (2017). 7th International Munich Chassis Symposium 2016, Springer."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1750","DOI":"10.1109\/TIE.2021.3059537","article-title":"Hybrid-learning-based driver steering intention prediction using neuromuscular dynamics","volume":"69","author":"Xing","year":"2021","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"103083","DOI":"10.1016\/j.ergon.2021.103083","article-title":"A method for fatigue detection based on Driver\u2019s steering wheel grip","volume":"82","author":"Li","year":"2021","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"M\u00fchlbacher-Karrer, S., Faller, L., Hamid, R., and Zangl, H. (2016, January 20\u201322). A wireless steering wheel gripping sensor for hands on\/off detection. Proceedings of the 2016 IEEE Sensors Applications Symposium (SAS), Catania, Italy.","DOI":"10.1109\/SAS.2016.7479878"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chen, H., Yang, Z., Huang, C., and Zhang, Q. (218, January 9\u201313). Drive Safe Inspector: A Wearable-Based Fine-Grained Technique for Driver Hand Position Detection. Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/GLOCOM.2018.8647653"},{"key":"ref_12","unstructured":"Johansson, E., and Linder, R. (2022, October 27). System for Hands-on Steering Wheel Detection Using Machine Learning. Available online: https:\/\/odr.chalmers.se\/items\/3d8ba74d-c0b1-4a6d-be14-f89d87981179\/full."},{"key":"ref_13","unstructured":"Hoang Ngan Le, T., Zheng, Y., Zhu, C., Luu, K., and Savvides, M. (July, January 26). Multiple scale faster-rcnn approach to driver\u2019s cell-phone usage and hands on steering wheel detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Rangesh, A., and Trivedi, M. (2018, January 18\u201322). Handynet: A one-stop solution to detect, segment, localize & analyze driver hands. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00158"},{"key":"ref_15","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst., 28."},{"key":"ref_16","unstructured":"Yudkin, P., Friedman, E., Zvitia, O., and Elbaz, G. (2022). Hands-Up: Leveraging Synthetic Data for Hands-On-Wheel Detection. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"106657","DOI":"10.1016\/j.asoc.2020.106657","article-title":"Distracted driver detection by combining in-vehicle and image data using deep learning","volume":"96","author":"Omerustaoglu","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4125865","DOI":"10.1155\/2019\/4125865","article-title":"Driver distraction identification with an ensemble of convolutional neural networks","volume":"2019","author":"Eraqi","year":"2019","journal-title":"J. Adv. Transp."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Jegham, I., Khalifa, A., Alouani, I., and Mahjoub, M. (2018, January 16\u201319). Safe driving: Driver action recognition using surf keypoints. Proceedings of the 2018 30th International Conference on Microelectronics (ICM), Sousse, Tunisia.","DOI":"10.1109\/ICM.2018.8704009"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Borghi, G., Frigieri, E., Vezzani, R., and Cucchiara, R. (2018, January 15\u201319). Hands on the wheel: A dataset for driver hand detection and tracking. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00090"},{"key":"ref_21","unstructured":"(2022, December 17). Vector Informatik GmbH Vector. Available online: https:\/\/www.vector.com\/kr\/ko\/company\/about-vector\/#c55657."},{"key":"ref_22","unstructured":"(2022, December 17). pywin32 PyPI. Available online: https:\/\/pypi.org\/project\/pywin32\/."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_24","unstructured":"Kingma, D., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_25","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2\u20134). TensorFlow: A system for Large-Scale machine learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), Savannah, GA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1442\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:17:51Z","timestamp":1760120271000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1442"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,28]]},"references-count":25,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23031442"],"URL":"https:\/\/doi.org\/10.3390\/s23031442","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,28]]}}}