{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T12:44:32Z","timestamp":1774529072519,"version":"3.50.1"},"reference-count":58,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2022,9,2]],"date-time":"2022-09-02T00:00:00Z","timestamp":1662076800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Kwangwoon University","award":["S2949268"],"award-info":[{"award-number":["S2949268"]}]},{"name":"Korea Technology and Information Promotion Agency for SMEs","award":["S2949268"],"award-info":[{"award-number":["S2949268"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Treatment of facial palsy is essential because neglecting this disorder can lead to serious sequelae and further damage. For an objective evaluation and consistent rehabilitation training program of facial palsy patients, a clinician\u2019s evaluation must be simultaneously performed alongside quantitative evaluation. Recent research has evaluated facial palsy using 68 facial landmarks as features. However, facial palsy has numerous features, whereas existing studies use relatively few landmarks; moreover, they do not confirm the degree of improvement in the patient. In addition, as the face of a normal person is not perfectly symmetrical, it must be compared with previous images taken at a different time. Therefore, we introduce three methods to numerically approach measuring the degree of facial palsy after extracting 478 3D facial landmarks from 2D RGB images taken at different times. The proposed numerical approach performs registration to compare the same facial palsy patients at different times. We scale landmarks by performing scale matching before global registration. After scale matching, coarse registration is performed with global registration. Point-to-plane ICP is performed using the transformation matrix obtained from global registration as the initial matrix. After registration, the distance symmetry, angular symmetry, and amount of landmark movement are calculated for the left and right sides of the face. The degree of facial palsy at a certain point in time can be approached numerically and can be compared with the degree of palsy at other times. For the same facial expressions, the degree of facial palsy at different times can be measured through distance and angle symmetry. For different facial expressions, the simultaneous degree of facial palsy in the left and right sides can be compared through the amount of landmark movement. Through experiments, the proposed method was tested using the facial palsy patient database at different times. The experiments involved clinicians and confirmed that using the proposed numerical approach can help assess the progression of facial palsy.<\/jats:p>","DOI":"10.3390\/s22176636","type":"journal-article","created":{"date-parts":[[2022,9,8]],"date-time":"2022-09-08T04:18:32Z","timestamp":1662610712000},"page":"6636","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Numerical Approach to Facial Palsy Using a Novel Registration Method with 3D Facial Landmark"],"prefix":"10.3390","volume":"22","author":[{"given":"Junsik","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Electronic Engineering, Kwangwoon University, Seoul 01897, Korea"}]},{"given":"Hyungwha","family":"Jeong","sequence":"additional","affiliation":[{"name":"Department of Plastic Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul 05505, Korea"}]},{"given":"Jeongmok","family":"Cho","sequence":"additional","affiliation":[{"name":"Department of Plastic Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul 05505, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3477-7120","authenticated-orcid":false,"given":"Changsik","family":"Pak","sequence":"additional","affiliation":[{"name":"Department of Plastic Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul 05505, Korea"}]},{"given":"Tae Suk","family":"Oh","sequence":"additional","affiliation":[{"name":"Department of Plastic Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul 05505, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6208-9704","authenticated-orcid":false,"given":"Joon Pio","family":"Hong","sequence":"additional","affiliation":[{"name":"Department of Plastic Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul 05505, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6595-6415","authenticated-orcid":false,"given":"Soonchul","family":"Kwon","sequence":"additional","affiliation":[{"name":"Graduate School of Smart Convergence, Kwangwoon University, Seoul 01897, Korea"}]},{"given":"Jisang","family":"Yoo","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Kwangwoon University, Seoul 01897, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1197","DOI":"10.1136\/bmj.2.5003.1197","article-title":"Facial palsy","volume":"2","author":"Cawthorne","year":"1956","journal-title":"Br. Med J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1159\/000007990","article-title":"Peripheral facial palsy: Etiology, diagnosis and treatment","volume":"41","author":"Roob","year":"1999","journal-title":"Eur. Neurol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"E283","DOI":"10.1002\/lary.24542","article-title":"Etiology, diagnosis, and management of facial palsy: 2000 patients at a facial nerve center","volume":"124","author":"Hohman","year":"2014","journal-title":"Laryngoscope"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1177\/0269215510395634","article-title":"Facial exercise therapy for facial palsy: Systematic review and meta-analysis","volume":"25","author":"Pereira","year":"2011","journal-title":"Clin. Rehabil."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"618","DOI":"10.1016\/j.annemergmed.2017.08.039","article-title":"Managing peripheral facial palsy","volume":"71","author":"Garro","year":"2018","journal-title":"Ann. Emerg. Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1016\/j.otc.2018.07.011","article-title":"Facial rehabilitation: Evaluation and treatment strategies for the patient with facial palsy","volume":"51","author":"Robinson","year":"2018","journal-title":"Otolaryngol. Clin. North Am."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.bjps.2007.03.037","article-title":"Automatic three-dimensional quantitative analysis for evaluation of facial movement","volume":"61","author":"Hontanilla","year":"2008","journal-title":"J. Plast. Reconstr. Aesthetic Surg."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"102485","DOI":"10.1016\/j.jelekin.2020.102485","article-title":"Quantitative analysis of movements in facial nerve palsy with surface electromyography and kinematic analysis","volume":"56","author":"Demeco","year":"2021","journal-title":"J. Electromyogr. Kinesiol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"812961","DOI":"10.1155\/2015\/812961","article-title":"A bidimensional system of facial movement analysis conception and reliability in adults","volume":"2015","author":"Baude","year":"2015","journal-title":"BioMed Res. Int."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1364\/OSAC.405996","article-title":"Optical analysis of facial nerve degeneration in Bell\u2019s palsy","volume":"4","author":"Yitzhak","year":"2021","journal-title":"OSA Contin."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1016\/j.bjps.2020.08.050","article-title":"Three-dimensional scanners for soft-tissue facial assessment in clinical practice","volume":"74","author":"Petrides","year":"2021","journal-title":"J. Plast. Reconstr. Aesthetic Surg."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"755","DOI":"10.1016\/j.anl.2022.01.007","article-title":"New method to evaluate sequelae of static facial asymmetry in patients with facial palsy using three-dimensional scanning analysis","volume":"49","author":"Azuma","year":"2022","journal-title":"Auris Nasus Larynx"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Cheng, X., and Da, F. (2018, January 16\u201317). 3D Facial landmark localization based on two-step keypoint detection. Proceedings of the 2018 International Conference on Audio, Language and Image Processing (ICALIP), Shanghai, China.","DOI":"10.1109\/ICALIP.2018.8455372"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jbi.2016.03.011","article-title":"Facegram\u2013Objective quantitative analysis in facial reconstructive surgery","volume":"61","author":"Horta","year":"2016","journal-title":"J. Biomed. Inform."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gaber, A., Taher, M.F., and Wahed, M.A. (2015, January 25\u201329). Quantifying facial paralysis using the kinect v2. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.","DOI":"10.1109\/EMBC.2015.7318899"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Vinokurov, N., Arkadir, D., Linetsky, E., Bergman, H., and Weinshall, D. (2015). Quantifying hypomimia in parkinson patients using a depth camera. International Symposium on Pervasive Computing Paradigms for Mental Health, Springer.","DOI":"10.1007\/978-3-319-32270-4_7"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Barrios Dell\u2019Olio, G., and Sra, M. (2021, January 10\u201314). FaraPy: An Augmented Reality Feedback System for Facial Paralysis using Action Unit Intensity Estimation. Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology, Virtual.","DOI":"10.1145\/3472749.3474803"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Parra-Dominguez, G.S., Sanchez-Yanez, R.E., and Garcia-Capulin, C.H. (2021). Facial paralysis detection on images using key point analysis. Appl. Sci., 11.","DOI":"10.3390\/app11052435"},{"key":"ref_19","unstructured":"Grishchenko, I., Ablavatski, A., Kartynnik, Y., Raveendran, K., and Grundmann, M. (2020). Attention Mesh: High-fidelity Face Mesh Prediction in Real-time. arXiv."},{"key":"ref_20","unstructured":"Kartynnik, Y., Ablavatski, A., Grishchenko, I., and Grundmann, M. (2019). Real-time facial surface geometry from monocular video on mobile GPUs. arXiv."},{"key":"ref_21","unstructured":"Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.L., Yong, M.G., and Lee, J. (2019). Mediapipe: A framework for building perception pipelines. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhao, R., Wang, Y., Benitez-Quiroz, C.F., Liu, Y., and Martinez, A.M. (15\u201316, January 8\u201310). Fast and precise face alignment and 3D shape reconstruction from a single 2D image. Proceedings of the European Conference on Computer Vision-ECCV 2016 Workshops, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-48881-3_41"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Bulat, A., and Tzimiropoulos, G. (2016). Two-stage convolutional part heatmap regression for the 1st 3d face alignment in the wild (3dfaw) challenge. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-48881-3_43"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gou, C., Wu, Y., Wang, F.Y., and Ji, Q. (2016). Shape augmented regression for 3D face alignment. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-48881-3_42"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Bulat, A., and Tzimiropoulos, G. (2017, January 22\u201327). How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.116"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jackson, A.S., Bulat, A., Argyriou, V., and Tzimiropoulos, G. (2017, January 22\u201327). Large pose 3D face reconstruction from a single image via direct volumetric CNN regression. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.117"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"12813","DOI":"10.1038\/s41598-019-49256-0","article-title":"SPOT3D: Spatial positioning toolbox for head markers using 3D scans","volume":"9","author":"Taberna","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Feng, Z.H., Huber, P., Kittler, J., Hancock, P., Wu, X.J., Zhao, Q., Koppen, P., and R\u00e4tsch, M. (2018, January 15\u201319). Evaluation of dense 3D reconstruction from 2D face images in the wild. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00123"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2250","DOI":"10.1109\/TPAMI.2017.2750687","article-title":"consistent 3D face alignment","volume":"40","author":"Tulyakov","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhang, H., Li, Q., and Sun, Z. (2018, January 20\u201324). Joint voxel and coordinate regression for accurate 3d facial landmark localization. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8546220"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1109\/TPAMI.2018.2866051","article-title":"Hierarchical binary CNNs for landmark localization with limited resources","volume":"42","author":"Bulat","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"7653","DOI":"10.1109\/ACCESS.2022.3141791","article-title":"Deep Learning-Based Facial Landmarks Localization Using Compound Scaling","volume":"10","author":"Colaco","year":"2022","journal-title":"IEEE Access"},{"key":"ref_33","unstructured":"Bazarevsky, V., Kartynnik, Y., Vakunov, A., Raveendran, K., and Grundmann, M. (2019). Blazeface: Sub-millisecond neural face detection on mobile gpus. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"895","DOI":"10.1016\/j.jcms.2019.03.022","article-title":"Assessment of reproducibility and reliability of facial expressions using 3D handheld scanner","volume":"47","author":"Sekerci","year":"2019","journal-title":"J. Cranio-Maxillofac. Surg."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.bjoms.2017.11.015","article-title":"Three-dimensional superimposition for patients with facial palsy: An innovative method for assessing the success of facial reanimation procedures","volume":"56","author":"Sforza","year":"2018","journal-title":"Br. J. Oral Maxillofac. Surg."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.1016\/j.joms.2017.11.017","article-title":"A quantitative assessment of lip movements in different facial expressions through 3-dimensional on 3-dimensional superimposition: A cross-sectional study","volume":"76","author":"Gibelli","year":"2018","journal-title":"J. Oral Maxillofac. Surg."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40510-015-0106-9","article-title":"Facial asymmetry assessment in adults using three-dimensional surface imaging","volume":"16","author":"Patel","year":"2015","journal-title":"Prog. Orthod."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1097\/SCS.0b013e3182a2e99d","article-title":"Quantitative facial asymmetry: Using three-dimensional photogrammetry to measure baseline facial surface symmetry","volume":"25","author":"Taylor","year":"2014","journal-title":"J. Craniofacial Surg."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1016\/j.anl.2015.01.002","article-title":"Quantitative analysis of facial palsy using a three-dimensional facial motion measurement system","volume":"42","author":"Katsumi","year":"2015","journal-title":"Auris Nasus Larynx"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"468","DOI":"10.1016\/j.otohns.2007.12.017","article-title":"Novel 3-D video for quantification of facial movement","volume":"138","author":"Mehta","year":"2008","journal-title":"Otolaryngol. Neck Surg."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"11893","DOI":"10.1007\/s11042-015-2696-0","article-title":"Automatic evaluation of the degree of facial nerve paralysis","volume":"75","author":"Wang","year":"2016","journal-title":"Multimed. Tools Appl."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1109\/JBHI.2017.2707588","article-title":"An unobtrusive computerized assessment framework for unilateral peripheral facial paralysis","volume":"22","author":"Guo","year":"2017","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"26756","DOI":"10.3390\/s151026756","article-title":"A smartphone-based automatic diagnosis system for facial nerve palsy","volume":"15","author":"Kim","year":"2015","journal-title":"Sensors"},{"key":"ref_44","unstructured":"Azoulay, O., Ater, Y., Gersi, L., Glassner, Y., Bryt, O., and Halperin, D. (2022, June 20). Mobile Application for Diagnosis of Facial Palsy. Available online: https:\/\/www.semanticscholar.org\/paper\/Mobile-Application-for-Diagnosis-of-Facial-Palsy-Azoulay-Ater\/890826f9a7e95232380a022f144f9a1d3b2c35ed."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1186\/s12880-016-0117-0","article-title":"Efficient quantitative assessment of facial paralysis using iris segmentation and active contour-based key points detection with hybrid classifier","volume":"16","author":"Barbosa","year":"2016","journal-title":"BMC Med Imaging"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Ngo, T.H., Chen, Y.W., Seo, M., Matsushiro, N., and Xiong, W. (2016, January 25\u201328). Quantitative analysis of facial paralysis based on three-dimensional features. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7532572"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Liu, Y., Xu, Z., Ding, L., Jia, J., and Wu, X. (2021, January 16\u201318). Automatic Assessment of Facial Paralysis Based on Facial Landmarks. Proceedings of the 2021 IEEE 2nd International Conference on Pattern Recognition and Machine Learning (PRML), Chengdu, China.","DOI":"10.1109\/PRML52754.2021.9520746"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Ekman, P., and Friesen, W.V. (1978). Facial action coding system. Environ. Psychol. Nonverbal Behav.","DOI":"10.1037\/t27734-000"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"313","DOI":"10.2307\/2371990","article-title":"Euler transformations","volume":"66","author":"Agnew","year":"1944","journal-title":"Am. J. Math."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/0262-8856(92)90066-C","article-title":"Object modelling by registration of multiple range images","volume":"10","author":"Chen","year":"1992","journal-title":"Image Vis. Comput."},{"key":"ref_51","first-page":"586","article-title":"Method for registration of 3-D shapes","volume":"Volume 1611","author":"Besl","year":"1992","journal-title":"Sensor Fusion IV: Control Paradigms and Data Structures, Proceedings of the ROBOTICS \u201991, Boston, MA, USA, 14\u201315 November 1991"},{"key":"ref_52","unstructured":"(2022, June 20). Reineg. Muscles of the Face, Colorful Anatomy info Poster. Available online: https:\/\/stock.adobe.com\/us\/search?k=facial+muscle&search_type=recentsearch&asset_id=309366859."},{"key":"ref_53","unstructured":"Nakao, N., Ohyama, W., Wakabayashi, T., and Kimura, F. (2007, January 12\u201313). Automatic Detection of Facial Midline as a Guide for Facial Feature Extraction. Proceedings of the 7th International Workshop on Pattern Recognition in Information Systems, Funchal, Portugal."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Galv\u00e1nek, M., Furmanov\u00e1, K., Chal\u00e1s, I., and Sochor, J. (2015, January 22\u201324). Automated facial landmark detection, comparison and visualization. Proceedings of the 31st Spring Conference on Computer Graphics, Smolenice, Slovakia.","DOI":"10.1145\/2788539.2788540"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Lee, Y., Kumar, Y.S., Lee, D., Kim, J., Kim, J., Yoo, J., and Kwon, S. (2020). An extended method for saccadic eye movement measurements using a head-mounted display. Healthcare, 8.","DOI":"10.3390\/healthcare8020104"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"55","DOI":"10.5565\/rev\/elcvia.142","article-title":"Automatic detection of facial midline and its contributions to facial feature extraction","volume":"6","author":"Ohyama","year":"2007","journal-title":"ELCVIA Electron. Lett. Comput. Vis. Image Anal."},{"key":"ref_57","unstructured":"Zhou, Q.Y., Park, J., and Koltun, V. (2018). Open3D: A modern library for 3D data processing. arXiv."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"698","DOI":"10.1109\/TPAMI.1987.4767965","article-title":"Least-squares fitting of two 3-D point sets","volume":"PAMI-9","author":"Arun","year":"1987","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/17\/6636\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:22:18Z","timestamp":1760142138000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/17\/6636"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,2]]},"references-count":58,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22176636"],"URL":"https:\/\/doi.org\/10.3390\/s22176636","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,2]]}}}