{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T03:52:03Z","timestamp":1790999523817,"version":"4.1.0"},"reference-count":41,"publisher":"Public Library of Science (PLoS)","issue":"12","license":[{"start":{"date-parts":[[2022,12,14]],"date-time":"2022-12-14T00:00:00Z","timestamp":1670976000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Quantitative grading and classification of the severity of facial paralysis (FP) are important for selecting the treatment plan and detecting subtle improvement that cannot be detected clinically. To date, none of the available FP grading systems have gained widespread clinical acceptance. The work presented here describes the development and testing of a system for FP grading and assessment which is part of a comprehensive evaluation system for FP. The system is based on the Kinect v2 hardware and the accompanying software SDK 2.0 in extracting the real time facial landmarks and facial animation units (FAUs). The aim of this paper is to describe the development and testing of the FP assessment phase (first phase) of a larger comprehensive evaluation system of FP. The system includes two phases; FP assessment and FP classification. A dataset of 375 records from 13 unilateral FP patients was compiled for this study. The FP assessment includes three separate modules. One module is the symmetry assessment of both facial sides at rest and while performing five voluntary facial movements. Another module is responsible for recognizing the facial movements. The last module assesses the performance of each facial movement for both sides of the face depending on the involved FAUs. The study validates that the FAUs captured using the Kinect sensor can be processed and used to develop an effective tool for the automatic evaluation of FP. The developed FP grading system provides a detailed quantitative report and has significant advantages over the existing grading scales. It is fast, easy to use, user-independent, low cost, quantitative, and automated and hence it is suitable to be used as a clinical tool.<\/jats:p>","DOI":"10.1371\/journal.pone.0277297","type":"journal-article","created":{"date-parts":[[2022,12,14]],"date-time":"2022-12-14T18:30:32Z","timestamp":1671042632000},"page":"e0277297","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":21,"title":["Comprehensive assessment of facial paralysis based on facial animation units"],"prefix":"10.1371","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3380-645X","authenticated-orcid":true,"given":"Amira","family":"Gaber","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mona F.","family":"Taher","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manal","family":"Abdel Wahed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nevin Mohieldin","family":"Shalaby","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Gaber","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2022,12,14]]},"reference":[{"issue":"11","key":"pone.0277297.ref001","doi-asserted-by":"crossref","DOI":"10.3390\/fi10110111","article-title":"Neurologist Standard Classification of Facial Nerve Paralysis with Deep Neural Networks","volume":"10","author":"A. Song","year":"2018","journal-title":"Future Internet"},{"key":"pone.0277297.ref002","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/0194599813505967","article-title":"Clinical practice guideline","volume":"149","author":"R. Baugh","year":"2013","journal-title":"Otolaryngol-Head Neck Surgery"},{"key":"pone.0277297.ref003","doi-asserted-by":"crossref","first-page":"466","DOI":"10.1080\/01676830802352543","article-title":"Facial Nerve Palsy: Anatomy, Etiology, Evaluation, and Management","author":"I. Mavrikakis","year":"2008","journal-title":"Orbit"},{"issue":"4","key":"pone.0277297.ref004","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1589\/jpts.25.515","article-title":"Evaluation and Grading Systems of Facial Paralysis for Facial Rehabilitation","volume":"25","author":"K. Sundaraj","year":"2013","journal-title":"Journal of Physical Therapy Science"},{"issue":"2","key":"pone.0277297.ref005","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/S1672-2930(08)50016-X","article-title":"Facial Grading System: Physical and Psychological Impairments to Be Considered","volume":"3","author":"M. -Y. Zhai","year":"2008","journal-title":"Journal of Otology"},{"issue":"1","key":"pone.0277297.ref006","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1055\/s-2004-823119","article-title":"Approaches to Grading Facial Nerve Function","volume":"18","author":"M. J. Brenner","year":"2004","journal-title":"Seminars in Plastic Surgery"},{"key":"pone.0277297.ref007","volume-title":"Department of Otorhinolaryngology","author":"M. Kanerva","year":"2008"},{"issue":"2","key":"pone.0277297.ref008","doi-asserted-by":"crossref","first-page":"223e","DOI":"10.1097\/PRS.0000000000001447","article-title":"Clinician-Graded Electronic Facial Paralysis Assessment: The eFACE","volume":"136","author":"C. A. Banks","year":"2015","journal-title":"Plastic and Reconstructive Surgery"},{"issue":"10","key":"pone.0277297.ref009","doi-asserted-by":"crossref","first-page":"2274","DOI":"10.1002\/lary.27739","article-title":"Reliability of grading of facial palsy using a video tutorial with synchronous video recording","volume":"129","author":"G. F. Volk","year":"2019","journal-title":"Laryngoscope"},{"key":"pone.0277297.ref010","article-title":"A review of image-based automatic facial landmark identification techniques","author":"B. Johnston","year":"2018","journal-title":"EURASIP Journal on Image and Video Processing"},{"issue":"10","key":"pone.0277297.ref011","doi-asserted-by":"crossref","DOI":"10.3390\/sym11101189","article-title":"Facial Expression Recognition: A Survey","volume":"11","author":"Y. Huang","year":"2019","journal-title":"Symmetry"},{"key":"pone.0277297.ref012","unstructured":"G.-S. J. Hsu, W.-F. Huang and J.-H. Kang, \"Hierarchical Network for Facial Palsy Detection,\" in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018."},{"issue":"1","key":"pone.0277297.ref013","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1097\/PRS.0000000000003447","article-title":"Validation of the Clinician-Graded Electronic Facial Paralysis Assessment","volume":"140","author":"L. Chong","year":"2017","journal-title":"Plastic and Reconstructive Surgery"},{"issue":"2","key":"pone.0277297.ref014","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1097\/PRS.0000000000007572","article-title":"The Auto-eFACE: Machine Learning-Enhanced Program Yields Automated Facial Palsy Assessment Tool","volume":"147","author":"M. Q. Miller","year":"2021","journal-title":"Plastic and Reconstructive Surgery"},{"key":"pone.0277297.ref015","first-page":"253","article-title":"Automatic and Objective Facial Palsy Grading Index Prediction Using Deep Feature Regression","author":"A. Raj","year":"2020","journal-title":"Springer Nature Switzerland"},{"key":"pone.0277297.ref016","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-14815-7_6","article-title":"GY MEDIC: Analysis and Rehabilitation System for Patients with Facial Paralysis","author":"G. M. Guanoluisa","year":"2019","journal-title":"Integrated Uncertainty in Knowledge Modelling and Decision Making IUKM"},{"key":"pone.0277297.ref017","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-25965-5_26","article-title":"GY MEDIC v2: Quantification of Facial Asymmetry in Patients with Automated Bell\u2019s Palsy by AI","author":"G. M. Guanoluisa","year":"2019","journal-title":"Augmented Reality, Virtual Reality, and Computer Graphics AVR"},{"issue":"1","key":"pone.0277297.ref018","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.bjoms.2017.11.015","article-title":"Three-dimensional superimposition for patients with facial palsy: an innovative method for assessing the success of facial reanimation procedures","volume":"56","author":"C. Sforza","year":"2018","journal-title":"British Journal of Oral and Maxillofacial Surgery"},{"issue":"4","key":"pone.0277297.ref019","doi-asserted-by":"crossref","DOI":"10.1001\/jamafacial.2018.0030","article-title":"A Machine Learning Approach for Automated Facial Measurements in Facial Palsy","volume":"20","author":"D. L. Guarin","year":"2018","journal-title":"JAMA Facial Plastic Surgery"},{"issue":"3","key":"pone.0277297.ref020","article-title":"Analysis of Facial Information for Healthcare Applications: A Survey on Computer Vision-Based Approaches","volume":"11","author":"M. Leo","year":"2020","journal-title":"Information"},{"key":"pone.0277297.ref021","first-page":"26757","article-title":"A smartphone-based automatic diagnosis system for facial nerve palsy","volume":"15","author":"H. Kim","year":"2015","journal-title":"Sensors 2015"},{"key":"pone.0277297.ref022","first-page":"2751","article-title":"Automatic recognition of facial movement for paralyzed face","volume":"24","author":"T. Wang","year":"2014","journal-title":"Biomed. Mater. Eng."},{"issue":"4","key":"pone.0277297.ref023","doi-asserted-by":"crossref","first-page":"2325","DOI":"10.14419\/ijet.v7i4.13619","article-title":"Evaluation of 3d facial paralysis using fuzzy logic","volume":"7","author":"Banita","year":"2018","journal-title":"International Journal of Engineering & Technology"},{"key":"pone.0277297.ref024","doi-asserted-by":"crossref","unstructured":"T. Mallick, P. Goyal, P. P. Das and A. K. Majumdar, \"Facial Emotion Recognition from Kinect Data\u2013An Appraisal of Kinect Face Tracking Library,\" in International Conference on Computer Vision Theory and Applications, 2016.","DOI":"10.5220\/0005713405250532"},{"issue":"1","key":"pone.0277297.ref025","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.bjps.2007.03.037","article-title":"Automatic three-dimensional quantitative analysis for evaluation of facial movement","volume":"61","author":"B. Hontanilla","year":"2008","journal-title":"J Plast Reconstr Aesthet Surg"},{"issue":"5","key":"pone.0277297.ref026","doi-asserted-by":"crossref","DOI":"10.1088\/2057-1976\/ac107c","article-title":"SVM classification of facial functions based on facial landmarks and animation Units","volume":"7","author":"A. Gaber","year":"2021","journal-title":"Biomed. Phys. Eng. Express"},{"issue":"65","key":"pone.0277297.ref027","article-title":"Classification of Facial Paralysis based on Machine Learning Techniques","volume":"21","author":"A. Gaber","year":"2022","journal-title":"BioMedical Engineering OnLine"},{"key":"pone.0277297.ref028","volume-title":"Manual for the Facial Action Coding System","author":"P. Ekman","year":"1977"},{"key":"pone.0277297.ref029","volume-title":"Facial Action Coding System Investigator\u201cs Guide","author":"P. Ekman","year":"2002"},{"key":"pone.0277297.ref030","doi-asserted-by":"crossref","unstructured":"A. Gaber, M. F. Taher and M. A. Wahed, \"Automated Grading of Facial Paralysis using the Kinect v2: A Proof of Concept Study,\" in International Conference on Virtual Rehabilitation ICVR, Valencia, 2015.","DOI":"10.1109\/ICVR.2015.7358577"},{"key":"pone.0277297.ref031","doi-asserted-by":"crossref","unstructured":"A. Gaber, M. F. Taher and M. A. Wahed, \"Quantifying Facial Paralysis using the Kinect v2,\" in International Conference of the IEEE Engineering in Medecine and Biology Society, EMBC\u201915, Milan, 2015.","DOI":"10.1109\/EMBC.2015.7318899"},{"issue":"4","key":"pone.0277297.ref032","doi-asserted-by":"crossref","first-page":"109","DOI":"10.21595\/vp.2020.21283","article-title":"A pilot study on automated quantitative grading of facial functions","volume":"30","author":"A. Gaber","year":"2020","journal-title":"Vibroengineering PROCEDIA"},{"key":"pone.0277297.ref033","doi-asserted-by":"crossref","unstructured":"A. Gaber, \"Thesis: \"Automated Grading of Facial Paralysis using the Kinect V2\",\" Egypt, 2016.","DOI":"10.1109\/ICVR.2015.7358577"},{"issue":"3","key":"pone.0277297.ref034","first-page":"61","article-title":"Real Time Facial Emotion Recognition using Kinect V2 Sensor","volume":"17","author":"H. A. Alabbasi","year":"2015","journal-title":"IOSR Journal of Computer Engineering (IOSR-JCE)"},{"issue":"4","key":"pone.0277297.ref035","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1631\/FITEE.1400209","article-title":"Using Kinect for real-time emotion recognition via facial expressions","volume":"16","author":"Q.-R. MAO","year":"2015","journal-title":"Frontiers of Information Technology & Electronic Engineering"},{"issue":"2","key":"pone.0277297.ref036","first-page":"421","article-title":"Facial Emotion Expressions Recognition with Brain Activities using Kinect Sensor V2","volume":"2","author":"H. A. Alabasi","year":"2015","journal-title":"International Research Journal of Engineering and Technology (IRJET)"},{"issue":"7","key":"pone.0277297.ref037","doi-asserted-by":"crossref","first-page":"3418","DOI":"10.1109\/TLA.2016.7587650","article-title":"Facial Expression Analysis with Kinect for the Diagnosis of Paralysis Using Nottingham System","volume":"14","author":"R. C. Carro","year":"2016","journal-title":"IEEE Latin America Transactions"},{"issue":"2","key":"pone.0277297.ref038","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1097\/SAP.0b013e3182223d96","article-title":"Evolution of the 3-Dimensional Video System for Facial Motion Analysis","volume":"69","author":"Tzou","year":"2012","journal-title":"Annals of Plastic Surgery"},{"issue":"10","key":"pone.0277297.ref039","first-page":"1313","article-title":"Measurement of facial movements with Photoshop software during treatment of facial nerve palsy","volume":"16","author":"A. A. Pourmomeny","year":"2011","journal-title":"Journal of Research in Medical Sciences"},{"issue":"5","key":"pone.0277297.ref040","first-page":"10928","article-title":"Outcome Prediction of Bell\u2019s Palsy by Kinect II","volume":"14","author":"F. Caisberger","year":"2019","journal-title":"Biomedical Journal of Scientific & Technical Research"},{"issue":"19","key":"pone.0277297.ref041","doi-asserted-by":"crossref","DOI":"10.1007\/s11042-015-2696-0","article-title":"Automatic Evaluation of the Degree of Facial Nerve Paralysis","volume":"75","author":"T. Wang","year":"2015","journal-title":"Multimedia Tools and Applications"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0277297","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,14]],"date-time":"2022-12-14T18:31:01Z","timestamp":1671042661000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0277297"}},"subtitle":[],"editor":[{"given":"Debotosh","family":"Bhattacharjee","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2022,12,14]]},"references-count":41,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,12,14]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0277297","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,14]]}}}