{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,7]],"date-time":"2026-01-07T08:09:53Z","timestamp":1767773393020,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,4,20]],"date-time":"2022-04-20T00:00:00Z","timestamp":1650412800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Research Project of China Disabled Persons\u2019 Federation - on Assistive Technology","award":["2021CDPFAT-09"],"award-info":[{"award-number":["2021CDPFAT-09"]}]},{"DOI":"10.13039\/501100018617","name":"Liaoning Revitalization Talents Program","doi-asserted-by":"publisher","award":["XLYC1908007"],"award-info":[{"award-number":["XLYC1908007"]}],"id":[{"id":"10.13039\/501100018617","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100017683","name":"Dalian Science and Technology Innovation Fund","doi-asserted-by":"publisher","award":["2019J11CY001","2021JJ12GX028"],"award-info":[{"award-number":["2019J11CY001","2021JJ12GX028"]}],"id":[{"id":"10.13039\/501100017683","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Head pose and eye gaze are vital clues for analysing a driver\u2019s visual attention. Previous approaches achieve promising results from point clouds in constrained conditions. However, these approaches face challenges in the complex naturalistic driving scene. One of the challenges is that the collected point cloud data under non-uniform illumination and large head rotation is prone to partial facial occlusion. It causes bad transformation during failed template matching or incorrect feature extraction. In this paper, a novel estimation method is proposed for predicting accurate driver head pose and gaze zone using an RGB-D camera, with an effective point cloud fusion and registration strategy. In the fusion step, to reduce bad transformation, continuous multi-frame point clouds are registered and fused to generate a stable point cloud. In the registration step, to reduce reliance on template registration, multiple point clouds in the nearest neighbor gaze zone are utilized as a template point cloud. A coarse transformation computed by the normal distributions transform is used as the initial transformation, and updated with particle filter. A gaze zone estimator is trained by combining the head pose and eye image features, in which the head pose is predicted by point cloud registration, and the eye image features are extracted via multi-scale spare coding. Extensive experiments demonstrate that the proposed strategy achieves better results on head pose tracking, and also has a low error on gaze zone classification.<\/jats:p>","DOI":"10.3390\/s22093154","type":"journal-article","created":{"date-parts":[[2022,4,21]],"date-time":"2022-04-21T03:46:11Z","timestamp":1650512771000},"page":"3154","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Driver\u2019s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion"],"prefix":"10.3390","volume":"22","author":[{"given":"Yafei","family":"Wang","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, Dalian Maritime University, Dalian 116026, China"}]},{"given":"Guoliang","family":"Yuan","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Dalian Maritime University, Dalian 116026, China"}]},{"given":"Xianping","family":"Fu","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Dalian Maritime University, Dalian 116026, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3017","DOI":"10.1109\/TITS.2015.2462084","article-title":"Driver behavior analysis for safe driving: A survey","volume":"16","author":"Kaplan","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mittal, A., Kumar, K., Dhamija, S., and Kaur, M. (2016, January 17\u201318). Head movement-based driver drowsiness detection: A review of state-of-art techniques. Proceedings of the 2016 IEEE International Conference on Engineering and Technology (ICETECH), Coimbatore, India.","DOI":"10.1109\/ICETECH.2016.7569378"},{"key":"ref_3","unstructured":"Wang, J., Chai, W., Venkatachalapathy, A., Tan, K.L., Haghighat, A., Velipasalar, S., Adu-Gyamfi, Y., and Sharma, A. (2021). A Survey on Driver Behavior Analysis from In-Vehicle Cameras. IEEE Trans. Intell. Transp. Syst., 1\u201324."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Fanelli, G., Gall, J., and Van Gool, L. (2011, January 20\u201325). Real time head pose estimation with random regression forests. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995458"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1731","DOI":"10.1007\/s00371-020-01934-1","article-title":"RGB-D-based gaze point estimation via multi-column CNNs and facial landmarks global optimization","volume":"37","author":"Zhang","year":"2021","journal-title":"Vis. Comput."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, Y., Yuan, G., Mi, Z., Peng, J., Ding, X., Liang, Z., and Fu, X. (2019). Continuous driver\u2019s gaze zone estimation using rgb-d camera. Sensors, 19.","DOI":"10.3390\/s19061287"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Meyer, G.P., Gupta, S., Frosio, I., Reddy, D., and Kautz, J. (2015, January 7\u201313). Robust model-based 3d head pose estimation. Proceedings of the IEEE international Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.416"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1855","DOI":"10.1109\/TITS.2014.2332613","article-title":"Driver monitoring based on low-cost 3-D sensors","volume":"15","author":"Armingol","year":"2014","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"B\u00e4r, T., Reuter, J.F., and Z\u00f6llner, J.M. (2012, January 16\u201319). Driver head pose and gaze estimation based on multi-template icp 3-d point cloud alignment. Proceedings of the 2012 15th International IEEE Conference on Intelligent Transportation Systems, Anchorage, AK, USA.","DOI":"10.1109\/ITSC.2012.6338678"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hu, T., Jha, S., and Busso, C. (November, January 19). Robust driver head pose estimation in naturalistic conditions from point-cloud data. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA.","DOI":"10.1109\/IV47402.2020.9304592"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hu, T., Jha, S., and Busso, C. (2021). Temporal head pose estimation from point cloud in naturalistic driving conditions. IEEE Trans. Intell. Transp. Syst., Early Access.","DOI":"10.1109\/TITS.2021.3075350"},{"key":"ref_12","unstructured":"Huang, X., Mei, G., Zhang, J., and Abbas, R. (2021). A comprehensive survey on point cloud registration. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Cheng, L., Chen, S., Liu, X., Xu, H., Wu, Y., Li, M., and Chen, Y. (2018). Registration of laser scanning point clouds: A review. Sensors, 18.","DOI":"10.3390\/s18051641"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Padeleris, P., Zabulis, X., and Argyros, A.A. (2012, January 16\u201321). Head pose estimation on depth data based on particle swarm optimization. Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.","DOI":"10.1109\/CVPRW.2012.6239236"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Schwarz, A., Haurilet, M., Martinez, M., and Stiefelhagen, R. (2017, January 21\u201326). Driveahead-a large-scale driver head pose dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.155"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Borghi, G., Venturelli, M., Vezzani, R., and Cucchiara, R. (2017, January 21\u201326). Poseidon: Face-from-depth for driver pose estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.583"},{"key":"ref_17","unstructured":"Venturelli, M., Borghi, G., Vezzani, R., and Cucchiara, R. (2016, January 4). Deep head pose estimation from depth data for in-car automotive applications. Proceedings of the International Workshop on Understanding Human Activities through 3D Sensors, Cancun, Mexico."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Saeed, A., and Al-Hamadi, A. (2015, January 27\u201330). Boosted human head pose estimation using kinect camera. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada.","DOI":"10.1109\/ICIP.2015.7351101"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ribeiro, R.F., and Costa, P.D. (2019, January 14\u201318). Driver gaze zone dataset with depth data. Proceedings of the 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France.","DOI":"10.1109\/FG.2019.8756592"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Aoki, Y., Goforth, H., Srivatsan, R.A., and Lucey, S. (2019, January 15\u201320). Pointnetlk: Robust & efficient point cloud registration using pointnet. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00733"},{"key":"ref_21","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21\u201326). Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Huang, X., Mei, G., and Zhang, J. (2020, January 13\u201319). Feature-metric registration: A fast semi-supervised approach for robust point cloud registration without correspondences. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01138"},{"key":"ref_23","unstructured":"Qi, C.R., Yi, L., Su, H., and Guibas, L.J. (2017, January 4\u20139). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/0262-8856(92)90066-C","article-title":"Object modelling by registration of multiple range images","volume":"10","author":"Chen","year":"1992","journal-title":"Image Vis. Comput."},{"key":"ref_25","unstructured":"Besl, P.J., and McKay, N.D. (1992, January 30). Method for registration of 3-D shapes. Proceedings of the Sensor Fusion IV: Control Paradigms and Data Structures, SPIE, Boston, MA, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yang, J., Li, H., and Jia, Y. (2013, January 1\u20138). Go-icp: Solving 3d registration efficiently and globally optimally. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.184"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Pavlov, A.L., Ovchinnikov, G.W., Derbyshev, D.Y., Tsetserukou, D., and Oseledets, I.V. (2018, January 21\u201325). AA-ICP: Iterative closest point with Anderson acceleration. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8461063"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jha, S., and Busso, C. (2017, January 16\u201319). Challenges in head pose estimation of drivers in naturalistic recordings using existing tools. Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan.","DOI":"10.1109\/ITSC.2017.8317870"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1109\/MIS.2016.47","article-title":"Driver gaze region estimation without use of eye movement","volume":"31","author":"Fridman","year":"2016","journal-title":"IEEE Intell. Syst."},{"key":"ref_30","unstructured":"Wang, Y., Zhao, T., Ding, X., Bian, J., and Fu, X. (2017, January 13\u201316). Head pose-free eye gaze prediction for driver attention study. Proceedings of the 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju, Korea."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"18560","DOI":"10.1109\/ACCESS.2021.3054951","article-title":"A Novel Driving Behavior Learning and Visualization Method with Natural Gaze Prediction","volume":"9","author":"Yuan","year":"2021","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Tayibnapis, I.R., Choi, M.K., and Kwon, S. (2018, January 12\u201314). Driver\u2019s gaze zone estimation by transfer learning. Proceedings of the 2018 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA.","DOI":"10.1109\/ICCE.2018.8326308"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Bi, Q., Ji, X., and Sun, Y. (2020, January 6\u20138). Research on Driver\u2019s Gaze Zone Estimation Based on Transfer Learning. Proceedings of the 2020 IEEE International Conference on Information Technology, Big Data and Artificial Intelligence (ICIBA), Chongqing, China.","DOI":"10.1109\/ICIBA50161.2020.9276968"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Shehu, I.S., Wang, Y., Athuman, A.M., and Fu, X. (2021). Remote Eye Gaze Tracking Research: A Comparative Evaluation on Past and Recent Progress. Electronics, 10.","DOI":"10.37247\/PAELEC.1.22.12"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Khan, M.Q., and Lee, S. (2019). Gaze and eye tracking: Techniques and applications in ADAS. Sensors, 19.","DOI":"10.3390\/s19245540"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.knosys.2016.07.038","article-title":"Appearance-based gaze estimation using deep features and random forest regression","volume":"110","author":"Wang","year":"2016","journal-title":"Knowl.-Based Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.knosys.2017.10.010","article-title":"Learning a gaze estimator with neighbor selection from large-scale synthetic eye images","volume":"139","author":"Wang","year":"2018","journal-title":"Knowl.-Based Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2739","DOI":"10.1109\/TITS.2016.2526050","article-title":"Driver-gaze zone estimation using Bayesian filtering and Gaussian processes","volume":"17","author":"Lundgren","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"107630","DOI":"10.1016\/j.knosys.2021.107630","article-title":"Self-calibrated driver gaze estimation via gaze pattern learning","volume":"235","author":"Yuan","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_40","unstructured":"Jha, S., and Busso, C. (2020). Estimation of Driver\u2019s Gaze Region from Head Position and Orientation using Probabilistic Confidence Regions. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1049\/iet-cvi.2015.0296","article-title":"\u2018Owl\u2019and \u2018Lizard\u2019: Patterns of head pose and eye pose in driver gaze classification","volume":"10","author":"Fridman","year":"2016","journal-title":"IET Comput. Vis."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ledezma, A., Zamora, V., Sipele, \u00d3., Sesmero, M.P., and Sanchis, A. (2021). Implementing a Gaze Tracking Algorithm for Improving Advanced Driver Assistance Systems. Electronics, 10.","DOI":"10.3390\/electronics10121480"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Araluce, J., Bergasa, L.M., Oca\u00f1a, M., L\u00f3pez-Guill\u00e9n, E., Revenga, P.A., Arango, J.F., and P\u00e9rez, O. (2021). Gaze Focalization System for Driving Applications Using OpenFace 2.0 Toolkit with NARMAX Algorithm in Accidental Scenarios. Sensors, 21.","DOI":"10.3390\/s21186262"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1109\/TITS.2019.2892155","article-title":"Driver monitoring using sparse representation with part-based temporal face descriptors","volume":"21","author":"Chiou","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1109\/TCE.2021.3127006","article-title":"Driver Gaze Zone Estimation via Head Pose Fusion Assisted Supervision and Eye Region Weighted Encoding","volume":"67","author":"Yang","year":"2021","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Magnusson, M., Andreasson, H., Nuchter, A., and Lilienthal, A.J. (2009, January 12\u201317). Appearance-based loop detection from 3D laser data using the normal distributions transform. Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan.","DOI":"10.1109\/ROBOT.2009.5152712"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1922","DOI":"10.1109\/TPAMI.2015.2500221","article-title":"Real-time head pose tracking with online face template reconstruction","volume":"38","author":"Li","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2014","DOI":"10.1109\/TITS.2015.2396031","article-title":"Driver gaze tracking and eyes off the road detection system","volume":"16","author":"Vicente","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Martins, P., and Batista, J. (2008, January 17\u201319). Accurate single view model-based head pose estimation. Proceedings of the 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition, Amsterdam, The Netherlands.","DOI":"10.1109\/AFGR.2008.4813369"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Baltrusaitis, T., Zadeh, A., Lim, Y.C., and Morency, L.P. (2018, January 15\u201319). Openface 2.0: Facial behavior analysis toolkit. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00019"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3154\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:57:35Z","timestamp":1760137055000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3154"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,20]]},"references-count":50,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["s22093154"],"URL":"https:\/\/doi.org\/10.3390\/s22093154","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,4,20]]}}}