{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T11:12:58Z","timestamp":1778152378719,"version":"3.51.4"},"reference-count":51,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2018,8,1]],"date-time":"2018-08-01T00:00:00Z","timestamp":1533081600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Facial landmarking locates the key facial feature points on facial data, which provides not only information on semantic facial structures, but also prior knowledge for other kinds of facial analysis. However, most of the existing works still focus on the 2D facial image which may suffer from lighting condition variations. In order to address this limitation, this paper presents a coarse-to-fine approach to accurately and automatically locate the facial landmarks by using deep feature fusion on 3D facial geometry data. Specifically, the 3D data is converted to 2D attribute maps firstly. Then, the global estimation network is trained to predict facial landmarks roughly by feeding the fused CNN (Convolutional Neural Network) features extracted from facial attribute maps. After that, input the local fused CNN features extracted from the local patch around each landmark estimated previously, and other local models are trained separately to refine the locations. Tested on the Bosphorus and BU-3DFE datasets, the experimental results demonstrated effectiveness and accuracy of the proposed method for locating facial landmarks. Compared with existed methods, our results have achieved state-of-the-art performance.<\/jats:p>","DOI":"10.3390\/sym10080308","type":"journal-article","created":{"date-parts":[[2018,8,1]],"date-time":"2018-08-01T11:22:34Z","timestamp":1533122554000},"page":"308","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["A Coarse-to-Fine Approach for 3D Facial Landmarking by Using Deep Feature Fusion"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9402-5839","authenticated-orcid":false,"given":"Kai","family":"Wang","sequence":"first","affiliation":[{"name":"School of Electrical and Information Engineering, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xi","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Management, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wanshun","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Electrical and Information Engineering, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianhua","family":"Zou","sequence":"additional","affiliation":[{"name":"School of Electrical and Information Engineering, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,8,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1109\/34.927467","article-title":"Active Appearance Models","volume":"23","author":"Cootes","year":"2001","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1006\/cviu.1995.1004","article-title":"Active shape models\u2014Their training and application","volume":"61","author":"Cootes","year":"1995","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Cristinacce, D., and Cootes, T.F. (2006, January 4\u20137). Feature Detection and Tracking with Constrained Local Models. Proceedings of the British Machine Vision Conference 2006, Edinburgh, UK.","DOI":"10.5244\/C.20.95"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kazemi, V., and Sullivan, J. (2014, January 23\u201328). One Millisecond Face Alignment with an Ensemble of Regression Trees. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.241"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ren, S., Cao, X., Wei, Y., and Sun, J. (2014, January 23\u201328). Face Alignment at 3000 FPS via Regressing Local Binary Features. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.218"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Xiong, X., and Torre, F.D.L. (2013, January 23\u201328). Supervised Descent Method and Its Applications to Face Alignment. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.75"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Dollar, P., Welinder, P., and Perona, P. (2010, January 13\u201318). Cascaded pose regression. Proceedings of the Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5540094"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1016\/j.imavis.2011.11.008","article-title":"Regression-based intensity estimation of facial action units","volume":"30","author":"Savran","year":"2012","journal-title":"Image Vis. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1109\/LSP.2014.2347011","article-title":"Random Cascaded-Regression Copse for Robust Facial Landmark Detection","volume":"22","author":"Feng","year":"2014","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_10","unstructured":"Zhu, S., Li, C., Chen, C.L., and Tang, X. (2015, January 7\u201312). Face alignment by coarse-to-fine shape searching. Proceedings of the Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhu, X., Lei, Z., Liu, X., Shi, H., and Li, S.Z. (2016, January 27\u201330). Face Alignment Across Large Poses: A 3D Solution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.23"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Jourabloo, A., and Liu, X. (2016, January 11\u201318). Pose-Invariant 3D Face Alignment. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.421"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1109\/TPAMI.2007.1017","article-title":"Three-dimensional face recognition in the presence of facial expressions: An annotated deformable model approach","volume":"29","author":"Kakadiaris","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","unstructured":"Perakis, P., Theoharis, T., Passalis, G., and Kakadiaris, I.A. (2009, January 29). Automatic 3D facial region retrieval from multi-pose facial datasets. Proceedings of the Eurographics Conference on 3D Object Retrieval, Munich, Germany."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Perakis, P., Passalis, G., Theoharis, T., Toderici, G., and Kakadiaris, I.A. (2009, January 28\u201330). Partial matching of interpose 3D facial data for face recognition. Proceedings of the IEEE International Conference on Biometrics: Theory, Applications, and Systems, Washington, DC, USA.","DOI":"10.1109\/BTAS.2009.5339019"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1487","DOI":"10.1016\/j.patrec.2006.02.015","article-title":"Combining local features for robust nose location in 3D facial data","volume":"27","author":"Xu","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"D\u2019Hose, J., Colineau, J., Bichon, C., and Dorizzi, B. (2007, January 27\u201329). Precise Localization of Landmarks on 3D Faces using Gabor Wavelets. Proceedings of the IEEE International Conference on Biometrics: Theory, Applications, and Systems, Crystal City, VA, USA.","DOI":"10.1109\/BTAS.2007.4401927"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Colbry, D., Stockman, G., and Jain, A. (2005, January 20\u201325). Detection of Anchor Points for 3D Face Veri.cation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA.","DOI":"10.1109\/CVPR.2005.441"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Bevilacqua, V., Casorio, P., and Mastronardi, G. (2008, January 15\u201318). Extending Hough Transform to a Points\u2019 Cloud for 3D-Face Nose-Tip Detection. Proceedings of the International Conference on Intelligent Computing: Advanced Intelligent Computing Theories and Applications\u2014With Aspects of Artificial Intelligence, Shanghai, China.","DOI":"10.1007\/978-3-540-85984-0_144"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1417","DOI":"10.1109\/TSMCB.2011.2148711","article-title":"Accurate landmarking of three-dimensional facial data in the presence of facial expressions and occlusions using a three-dimensional statistical facial feature model","volume":"41","author":"Zhao","year":"2011","journal-title":"IEEE Trans. Syst. Man Cybern. Part B Cybern."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1109\/TMM.2009.2017629","article-title":"3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model","volume":"11","author":"Nair","year":"2009","journal-title":"IEEE Trans. Multimedia"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jahanbin, S., Choi, H., Jahanbin, R., and Bovik, A.C. (2008, January 12\u201315). Automated facial feature detection and face recognition using Gabor features on range and portrait images. Proceedings of the IEEE International Conference on Image Processing, San Diego, CA, USA.","DOI":"10.1109\/ICIP.2008.4712368"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2816","DOI":"10.1109\/TMM.2017.2713408","article-title":"Multimodal 2D+3D Facial Expression Recognition with Deep Fusion Convolutional Neural Network","volume":"19","author":"Huibin","year":"2017","journal-title":"IEEE Trans. Multimedia"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7\u201312). FaceNet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"ref_25","unstructured":"Sun, Y., Liang, D., Wang, X., and Tang, X. (arXiv, 2015). DeepID3: Face Recognition with Very Deep Neural Networks, arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Taigman, Y., Yang, M., Ranzato, M., and Wolf, L. (2014, January 23\u201328). DeepFace: Closing the Gap to Human-Level Performance in Face Verification. Proceedings of the Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.220"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chang, F.J., Tran, A.T., Hassner, T., Masi, I., Nevatia, R., and Medioni, G. (2018, January 15\u201319). ExpNet: Landmark-Free, Deep, 3D Facial Expressions. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00027"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sun, Y., Wang, X., and Tang, X. (2013, January 23\u201328). Deep Convolutional Network Cascade for Facial Point Detection. Proceedings of the Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.446"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhang, J., Shan, S., Kan, M., and Chen, X. (2014). Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-10605-2_1"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Luo, P., Chen, C.L., and Tang, X. (2014, January 6\u201312). Facial Landmark Detection by Deep Multi-task Learning. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10599-4_7"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Yang, J., Liu, Q., and Zhang, K. (2017, January 21\u201326). Stacked Hourglass Network for Robust Facial Landmark Localisation. Proceedings of the Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.253"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kumar, A., and Chellappa, R. (2018, January 18\u201322). Disentangling 3D Pose in A Dendritic CNN for Unconstrained 2D Face Alignment. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00052"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Bulat, A., and Tzimiropoulos, G. (2017, January 22\u201329). How Far are We from Solving the 2D and 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks). Proceedings of the International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.116"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1109\/TPAMI.2006.15","article-title":"Matching 2.5D Face Scans to 3D Models","volume":"28","author":"Lu","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","unstructured":"Dibeklioglu, H., Salah, A.A., and Akarun, L. (October, January 29). 3D Facial Landmarking under Expression, Pose, and Occlusion Variations. Proceedings of the IEEE International Conference on Biometrics: Theory, Applications and Systems, Arlington, VA, USA."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1016\/j.patcog.2005.09.009","article-title":"3D face detection using curvature analysis","volume":"39","author":"Colombo","year":"2006","journal-title":"Pattern Recognit."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Boehnen, C., and Russ, T. (2005, January 5\u20137). A Fast Multi-Modal Approach to Facial Feature Detection. Proceedings of the Seventh IEEE Workshops on Application of Computer Vision, Breckenridge, CO, USA.","DOI":"10.1109\/ACVMOT.2005.5"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1016\/S0167-8655(02)00066-1","article-title":"Facial feature detection and face recognition from 2D and 3D images","volume":"23","author":"Wang","year":"2002","journal-title":"Pattern Recognit. Lett."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1007\/BF03253251","article-title":"Robust facial landmarking for registration","volume":"62","author":"Salah","year":"2007","journal-title":"Ann. T\u00e9l\u00e9commun."},{"key":"ref_40","unstructured":"Lu, X., and Jain, A.K. (2006, January 10\u201312). Automatic Feature Extraction for Multiview 3D Face Recognition. Proceedings of the International Conference on Automatic Face and Gesture Recognition, Southampton, UK."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Savran, A., and Akarun, L. (2008). Bosphorus Database for 3D Face Analysis. Biometrics and Identity Management, Springer.","DOI":"10.1007\/978-3-540-89991-4_6"},{"key":"ref_42","unstructured":"Yin, L., Wei, X., Sun, Y., Wang, J., and Rosato, M.J. (2006, January 10\u201312). A 3D facial expression database for facial behavior research. Proceedings of the FGR\u201906 International Conference on Automatic Face and Gesture Recognition, Southampton, UK."},{"key":"ref_43","unstructured":"Simonyan, K., and Zisserman, A. (arXiv, 2014). Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv."},{"key":"ref_44","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 26\u201329). ImageNet classification with deep convolutional neural networks. Proceedings of the International Conference on Neural Information Processing Systems, Doha, Qatar."},{"key":"ref_45","unstructured":"Ioffe, S., and Szegedy, C. (arXiv, 2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1109\/TIFS.2010.2054081","article-title":"Regional registration for expression resistant 3-D face recognition","volume":"5","author":"Akarun","year":"2010","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1007\/s11263-012-0605-9","article-title":"A Machine-Learning Approach to Keypoint Detection and Landmarking on 3D Meshes","volume":"102","author":"Creusot","year":"2013","journal-title":"Int. J. Comput. Vis."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1717","DOI":"10.1109\/TCYB.2014.2359056","article-title":"3-D Facial Landmark Localization With Asymmetry Patterns and Shape Regression from Incomplete Local Features","volume":"45","author":"Sukno","year":"2017","journal-title":"IEEE Trans. Cybern."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Camg\u00f6z, N.C., G\u00f6kberk, B., and Akarun, L. (2015, January 16\u201319). Facial landmark localization in depth images using Supervised Descent Method. Proceedings of the Signal Processing and Communications Applications Conference, Malatya, Turkey.","DOI":"10.1109\/SIU.2015.7130256"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Fanelli, G., Dantone, M., and Gool, L.V. (2013, January 22\u201326). Real time 3D face alignment with Random Forests-based Active Appearance Models. Proceedings of the IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, Shanghai, China.","DOI":"10.1109\/FG.2013.6553713"},{"key":"ref_51","unstructured":"Sun, J., Huang, D., Wang, Y., and Chen, L. (October, January 29). A coarse-to-fine approach to robust 3D facial landmarking via curvature analysis and Active Normal Model. Proceedings of the IEEE International Joint Conference on Biometrics, Clearwater, FL, USA."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/10\/8\/308\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:15:45Z","timestamp":1760195745000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/10\/8\/308"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,1]]},"references-count":51,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2018,8]]}},"alternative-id":["sym10080308"],"URL":"https:\/\/doi.org\/10.3390\/sym10080308","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,8,1]]}}}