{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T16:25:33Z","timestamp":1790180733366,"version":"4.1.0"},"reference-count":36,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,3,6]],"date-time":"2025-03-06T00:00:00Z","timestamp":1741219200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"CHAMO Innovation","award":["9252"],"award-info":[{"award-number":["9252"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The Modified Mallet Score (MMS) is widely used to assess upper limb function but requires evaluation by experienced clinicians. This study automated MMS assessments using smartphone videos, artificial intelligence (AI), and new algorithms. A total of 125 videos covering all MMS grades were recorded from four neurotypical participants. For all recordings, an expert physician provided manual scores as the ground truth. The OpenPose BODY25 model extracted body keypoint data, which were used to calculate joint angles for an automated scoring algorithm. The algorithm\u2019s scores were compared to the ground truth and expert manual scoring. High accuracy was achieved for the global abduction, hand-to-neck, hand-on-spine, and hand-to-mouth movements, with Pearson correlation coefficients (PCCs) &gt; 0.9 and a low root mean square error (RMSE). Although slightly less accurate for global external rotation, the algorithm still showed strong agreement. This study demonstrates the potential of using AI and smartphone videos for reliable, remote upper limb assessments.<\/jats:p>","DOI":"10.3390\/s25051619","type":"journal-article","created":{"date-parts":[[2025,3,6]],"date-time":"2025-03-06T06:02:13Z","timestamp":1741240933000},"page":"1619","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Automated Assessment of Upper Extremity Function with the Modified Mallet Score Using Single-Plane Smartphone Videos"],"prefix":"10.3390","volume":"25","author":[{"given":"Cancan","family":"Su","sequence":"first","affiliation":[{"name":"Children\u2019s Hospital of Eastern Ontario Research Institute, Ottawa, ON K1H 8L1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lianne","family":"Brandt","sequence":"additional","affiliation":[{"name":"Children\u2019s Hospital of Eastern Ontario Research Institute, Ottawa, ON K1H 8L1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-1543-5766","authenticated-orcid":false,"given":"Guangwen","family":"Sun","sequence":"additional","affiliation":[{"name":"Children\u2019s Hospital of Eastern Ontario Research Institute, Ottawa, ON K1H 8L1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaitlynn","family":"Sampel","sequence":"additional","affiliation":[{"name":"Children\u2019s Hospital of Eastern Ontario Research Institute, Ottawa, ON K1H 8L1, Canada"},{"name":"Department of Mechanical Engineering, University of Ottawa, Ottawa, ON K1N 6N5, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edward D.","family":"Lemaire","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, University of Ottawa, Ottawa, ON K1H 8M2, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2481-5952","authenticated-orcid":false,"given":"Kevin","family":"Cheung","sequence":"additional","affiliation":[{"name":"Department of Surgery, Division of Plastic Surgery, Children\u2019s Hospital of Eastern Ontario, Ottawa, ON K1H 8L1, Canada"},{"name":"Department of Surgery, University of Ottawa, Ottawa, ON K1N 6N5, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Albert","family":"Tu","sequence":"additional","affiliation":[{"name":"Department of Surgery, University of Ottawa, Ottawa, ON K1N 6N5, Canada"},{"name":"Department of Surgery, Division of Neurosurgery, Children\u2019s Hospital of Eastern Ontario, Ottawa, ON K1H 8L1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7025-7501","authenticated-orcid":false,"given":"Natalie","family":"Baddour","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, University of Ottawa, Ottawa, ON K1N 6N5, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Francisco-Mart\u00ednez, C., Prado-Olivarez, J., Padilla-Medina, J.A., D\u00edaz-Carmona, J., P\u00e9rez-Pinal, F.J., Barranco-Guti\u00e9rrez, A.I., and Mart\u00ednez-Nolasco, J.J. (2021). Upper Limb Movement Measurement Systems for Cerebral Palsy: A Systematic Literature Review. Sensors, 21.","DOI":"10.3390\/s21237884"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1257","DOI":"10.1007\/s11999-011-2065-x","article-title":"Assessment tools and classification systems used for the upper extremity in children with cerebral palsy","volume":"470","author":"Wagner","year":"2012","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Gupta, S., Srinivasan, N., Mahajan, J., Song, A., Chu, A., and McGrath, A. (2022). Outcome measures in OBPP. Brachial Plex. Inj. New Tech. Ideas, 75.","DOI":"10.5772\/intechopen.98796"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1111\/j.1469-8749.2003.tb00911.x","article-title":"Melbourne Assessment of Unilateral Upper Limb Function: Construct validity and correlation with the Pediatric Evaluation of Disability Inventory","volume":"45","year":"2003","journal-title":"Dev. Med. Child. Neurol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1097\/01202412-200609000-00004","article-title":"Interobserver reliability of the Mallet score","volume":"15","author":"Ritt","year":"2006","journal-title":"J. Pediatr. Orthop. B"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Al-Qattan, M.M., and El-Sayed, A.A. (2014). Obstetric brachial plexus palsy: The mallet grading system for shoulder function-revisited. Biomed. Res. Int., 2014.","DOI":"10.1155\/2014\/398121"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1007\/s00402-010-1067-6","article-title":"Soft tissue rebalancing procedures with and without internal rotation osteotomy for shoulder deformity in children with persistent obstetric brachial plexus palsy","volume":"130","author":"Synder","year":"2010","journal-title":"Arch. Orthop. Trauma. Surg."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1007\/s00381-010-1174-2","article-title":"Shoulder function and anatomy in complete obstetric brachial plexus palsy: Long-term improvement after triangle tilt surgery","volume":"26","author":"Nath","year":"2010","journal-title":"Childs Nerv. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1177\/17531934231196118","article-title":"Interrater reliability of face-to-face, tele- and video-based assessments with the modified Mallet classification in brachial plexus birth injuries","volume":"49","author":"Unes","year":"2024","journal-title":"J. Hand Surg. Eur. Vol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1097\/BPO.0000000000002212","article-title":"Comparison of Shoulder Motion Measurements by Visual Estimate, Goniometer and Motion Capture","volume":"42","author":"Russo","year":"2022","journal-title":"J. Pediatr. Orthop."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e951","DOI":"10.1097\/BPO.0000000000002775","article-title":"Shoulder Motion Overestimated by Mallet Scores","volume":"44","author":"Lovette","year":"2024","journal-title":"J. Pediatr. Orthop."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Grip, H., K\u00e4llstr\u00f6mer, A., and \u00d6hberg, F. (2022). Validity and Reliability of Wearable Motion Sensors for Clinical Assessment of Shoulder Function in Brachial Plexus Birth Injury. Sensors, 22.","DOI":"10.3390\/s22239557"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.15453\/2168-6408.1901","article-title":"Three-dimensional motion analysis for occupational therapy upper extremity assessment and rehabilitation: A scoping review","volume":"10","author":"Hill","year":"2022","journal-title":"Open J. Occup. Ther."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Dev, T., Selvaraj, S., Magimairaj, H.P., and Balasubramanian, S. (2022). Accuracy of Single RGBD Camera-Based Upper-Limb Movement Tracking Using OpenPose. Converging Clinical and Engineering Research on Neurorehabilitation, I.V. Proceedings of the 5th International Conference on Neurorehabilitation (ICNR2020), Online, 13\u201316 October 2020, Springer.","DOI":"10.1007\/978-3-030-70316-5_41"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1007\/s00530-021-00815-4","article-title":"A review of computer vision-based approaches for physical rehabilitation and assessment","volume":"28","author":"Debnath","year":"2022","journal-title":"Multimed. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Albuquerque, P., Machado, J.P., Verlekar, T.T., Correia, P.L., and Soares, L.D. (2021). Remote Gait Type Classification System Using Markerless 2D Video. Diagnostics, 11.","DOI":"10.3390\/diagnostics11101824"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ramesh, S.H., Lemaire, E.D., Cheung, K., Tu, A., and Baddour, N. (2023, January 18\u201320). Automated Stride Detection from OpenPose Keypoints Using Handheld Smartphone Video. Proceedings of the 2023 IEEE Sensors Applications Symposium (SAS), Ottawa, ON, Canada.","DOI":"10.1109\/SAS58821.2023.10254104"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhang, F., Juneau, P., McGuirk, C., Tu, A., Cheung, K., Baddour, N., and Lemaire, E. (2021, January 23\u201325). Comparison of OpenPose and HyperPose artificial intelligence models for analysis of hand-held smartphone videos. Proceedings of the 2021 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Lausanne, Switzerland.","DOI":"10.1109\/MeMeA52024.2021.9478740"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ramesh, S.H., Lemaire, E.D., Tu, A., Cheung, K., and Baddour, N. (2023). Automated Implementation of the Edinburgh Visual Gait Score (EVGS) Using OpenPose and Handheld Smartphone Video. Sensors, 23.","DOI":"10.3390\/s23104839"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1019","DOI":"10.1016\/S0031-3203(98)80010-1","article-title":"New algorithms for 2D and 3D point matching: Pose estimation and correspondence","volume":"31","author":"Gold","year":"1998","journal-title":"Pattern Recognit."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Konrad, J., Wang, M., and Ishwar, P. (2012, January 16\u201321). 2d-to-3d image conversion by learning depth from examples. Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.","DOI":"10.1109\/CVPRW.2012.6238903"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"594","DOI":"10.52082\/jssm.2021.594","article-title":"The Calculation, Thresholds and Reporting of Inter-Limb Strength Asymmetry: A Systematic Review","volume":"20","author":"Parkinson","year":"2021","journal-title":"J. Sports Sci. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1779","DOI":"10.1109\/TIM.2011.2108076","article-title":"Distance and angle measurement of objects on an oblique plane based on pixel number variation of CCD images","volume":"60","author":"Hsu","year":"2011","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Elassal, N., and Elder, J.H. (2017, January 17\u201319). Estimating Camera Tilt from Motion without Tracking. Proceedings of the 2017 14th Conference on Computer and Robot Vision (CRV), Edmonton, AB, Canada.","DOI":"10.1109\/CRV.2017.36"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"103056","DOI":"10.1016\/j.dsp.2021.103056","article-title":"Estimation of partially occluded 2D human joints with a Bayesian approach","volume":"114","author":"Dursun","year":"2021","journal-title":"Digit. Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Rafi, U., Gall, J., and Leibe, B. (2015, January 7\u201312). A semantic occlusion model for human pose estimation from a single depth image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA.","DOI":"10.1109\/CVPRW.2015.7301338"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhang, T., Huang, B., and Wang, Y. (2020, January 14\u201319). Object-occluded human shape and pose estimation from a single color image. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00740"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"S16","DOI":"10.1016\/j.gaitpost.2015.06.036","article-title":"Three dimensional movement analysis of the upper limb during activities of daily living in children with obstetric brachial plexus palsy: Comparison with healthy controls","volume":"42","author":"Mahon","year":"2015","journal-title":"Gait Posture"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1109\/MSP.2011.941097","article-title":"What is a savitzky-golay filter? [lecture notes]","volume":"28","author":"Schafer","year":"2011","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.gaitpost.2018.02.028","article-title":"Trunk involvement in performing upper extremity activities while seated in neurological patients with a flaccid trunk\u2013A review","volume":"62","author":"Peeters","year":"2018","journal-title":"Gait Posture"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1177\/1545968315613863","article-title":"Compensatory versus noncompensatory shoulder movements used for reaching in stroke","volume":"30","author":"Levin","year":"2016","journal-title":"Neurorehabilit. Neural Repair"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"521","DOI":"10.21037\/apm-21-3710","article-title":"Effects of physical therapy-based rehabilitation on recovery of upper limb motor function after stroke in adults: A systematic review and meta-analysis of randomized controlled trials","volume":"11","author":"Huang","year":"2022","journal-title":"Ann. Palliat. Med."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"S137","DOI":"10.1016\/j.hansur.2019.02.004","article-title":"Assessment of the upper limb of the tetraplegic patient","volume":"41s","author":"Moiziard","year":"2022","journal-title":"Hand Surg. Rehabil."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1016\/j.pmrj.2011.03.014","article-title":"A systematic literature review of outcome measures for upper extremity function using the international classification of functioning, disability, and health as reference","volume":"3","author":"Velstra","year":"2011","journal-title":"PMR"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1016\/j.apmr.2014.09.016","article-title":"Evaluation of measures of upper limb functioning and disability in people with Parkinson disease: A systematic review","volume":"96","author":"Proud","year":"2015","journal-title":"Arch. Phys. Med. Rehabil."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"563","DOI":"10.1515\/cdbme-2023-1141","article-title":"Accuracy Evaluation of 3D Pose Estimation with MediaPipe Pose for Physical Exercises","volume":"9","author":"Dill","year":"2023","journal-title":"Curr. Dir. Biomed. Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/5\/1619\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:48:16Z","timestamp":1760028496000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/5\/1619"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,6]]},"references-count":36,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["s25051619"],"URL":"https:\/\/doi.org\/10.3390\/s25051619","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,6]]}}}