{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T12:13:13Z","timestamp":1791288793841,"version":"4.3.0"},"reference-count":61,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Markerless motion tracking methods have promise for use in a range of domains, including clinical settings where traditional marker-based systems for human pose estimation are not feasible. Artificial intelligence (AI)\u2013based systems can offer a markerless, lightweight approach to motion capture. However, the accuracy of such systems, such as MediaPipe, for tracking fine upper limb movements involving the hand has not been explored.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n The aim of this study is to evaluate the 2D accuracy of MediaPipe against a known standard.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Participants (N=10) performed a touchscreen-based shape-tracing task requiring them to trace the trajectory of a moving cursor using their index finger. Cursor trajectories created a reoccurring or random shape at 5 different speeds (500-2500 ms, in increments of 500 ms). Movement trajectories on each trial were simultaneously captured by the touchscreen and a separate video camera. Movement coordinates for each trial were extracted from the touchscreen and compared to those predicted by MediaPipe. Specifically, following resampling, normalization, and Procrustes transformations, root-mean-squared error (RMSE; primary outcome measure) was calculated between predicted coordinates and those generated by the touchscreen computer.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Although there was some size distortion in the frame-by-frame estimates predicted by MediaPipe, shapes were similar between the 2 methods and transformations improved the general overlap and similarity of the shapes. The resultant mean RMSE between predicted coordinates and those generated by the touchscreen was 0.28 (SD 0.06) normalized px. Equivalence testing revealed that accuracy differed between MediaPipe and the touchscreen, but that the true difference was between 0 and 0.30 normalized px (\n t<\/jats:italic>\n 114<\/jats:sub>\n =\u22123.02;\n P<\/jats:italic>\n =.002). Additional analyses revealed no differences in resultant RMSE between methods when comparing across lower frame rates (30 and 60 frames per second [FPS]), although there was greater RMSE for 120 FPS than for 60 FPS (\n t<\/jats:italic>\n 35.43<\/jats:sub>\n =\u22122.51;\n P<\/jats:italic>\n =.03).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Overall, we quantified similarities between one AI-based approach to motion capture and a known standard for tracking fine upper limb movements, informing applications of such systems in domains such as clinical and research settings. Future work should address accuracy in 3 dimensions to further validate the use of AI-based systems, including MediaPipe, in such domains.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/56682","type":"journal-article","created":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T02:00:23Z","timestamp":1734573623000},"page":"e56682-e56682","source":"Crossref","is-referenced-by-count":27,"title":["Quantifying Similarities Between MediaPipe and a Known Standard to Address Issues in Tracking 2D Upper Limb Trajectories: Proof of Concept Study"],"prefix":"10.2196","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-4813-1097","authenticated-orcid":false,"given":"Vaidehi","family":"Wagh","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1062-3490","authenticated-orcid":false,"given":"Matthew W","family":"Scott","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6552-6682","authenticated-orcid":false,"given":"Sarah N","family":"Kraeutner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2024,12,17]]},"reference":[{"issue":"15","key":"R1","doi-asserted-by":"publisher","first-page":"4063","DOI":"10.3390\/s20154063","article-title":"Sensors and systems for physical rehabilitation and health monitoring\u2014a review","volume":"20","author":"Nascimento","journal-title":"Sensors (Basel)"},{"issue":"6","key":"R2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0286751","article-title":"Strengths and weaknesses of the MABC-2 as a diagnostic tool for developmental coordination disorder: an online survey of occupational therapists and physiotherapists","volume":"18","author":"Hadwin","journal-title":"PLoS ONE"},{"issue":"4","key":"R3","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1177\/026921559100500403","article-title":"The unreliability of sensory assessments","volume":"5","author":"Lincoln","journal-title":"Clin Rehabil"},{"issue":"4","key":"R4","doi-asserted-by":"publisher","DOI":"10.2196\/29769","article-title":"Application of inertial measurement units and machine learning classification in cerebral palsy: randomized controlled trial","volume":"8","author":"Khaksar","journal-title":"JMIR Rehabil Assist Technol"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","DOI":"10.2196\/50571","article-title":"Movement component analysis of reaching strategies in individuals with stroke: preliminary study","volume":"10","author":"Ota","journal-title":"JMIR Rehabil Assist Technol"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1007\/s42452-019-1925-y","article-title":"Challenges and opportunities in IoT healthcare systems: a systematic review","volume":"2","author":"Selvaraj","journal-title":"SN Appl Sci"},{"issue":"1","key":"R7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.bspc.2007.09.001","article-title":"Human motion tracking for rehabilitation\u2014a survey","volume":"3","author":"Zhou","journal-title":"Biomed Signal Process Control"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1186\/s12984-023-01186-9","article-title":"A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation","volume":"20","author":"Lam","journal-title":"J Neuroeng Rehabil"},{"key":"R9","doi-asserted-by":"crossref","unstructured":"Cao Z Simon T Wei SE Sheikh Y . Realtime multi-person 2d pose estimation using part affinity fields. Presented at: Proceedings of the IEEE conference on computer vision and pattern recognition 2017; 2017:7291-7299; Honolulu, HI. [doi: 10.1109\/CVPR.2017.143]","DOI":"10.1109\/CVPR.2017.143"},{"issue":"9","key":"R10","doi-asserted-by":"publisher","first-page":"1281","DOI":"10.1038\/s41593-018-0209-y","article-title":"DeepLabCut: markerless pose estimation of user-defined body parts with deep learning","volume":"21","author":"Mathis","journal-title":"Nat Neurosci"},{"key":"R11","unstructured":"Lugaresi C Tang J Nash H et al. Mediapipe: a framework for building perception pipelines. Google Research. 2019. URL: https:\/\/research.google\/pubs\/mediapipe-a-framework-for-perceiving-and-processing-reality\/ [Accessed 23-11-2024]"},{"key":"R12","doi-asserted-by":"crossref","unstructured":"Singh AK Kumbhare VA Arthi K . Real-time human pose detection and recognition using mediapipe. Presented at: International Conference on Soft Computing and Signal Processing; Jun 18, 2021:145-154; Singapore. [doi: 10.1007\/978-981-16-7088-6_12]","DOI":"10.1007\/978-981-16-7088-6_12"},{"key":"R13","unstructured":"Ahmadyan A Hou T Wei J Zhang L Ablavatski A Grundmann M . Instant 3D object tracking with applications in augmented reality. arXiv. Preprint posted online onJun23, 2020. [doi: 10.48550\/arXiv.2006.13194]"},{"key":"R14","doi-asserted-by":"crossref","unstructured":"Dijkstra-Soudarissanane S Gunkel SN Reinders V . Virtual visits: life-size immersive communication. Presented at: Proceedings of the 13th ACM Multimedia Systems Conference; Jun 14, 2022:310-314; Ireland.","DOI":"10.1145\/3524273.3532903"},{"key":"R15","volume":"2582","author":"Halder","journal-title":"J homepage"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Zhang M Schulze J Zhang D . Faceatlasar: atlas of facial acupuncture points in augmented reality. arXiv. Preprint posted online onNov29, 2021. [doi: 10.48550\/arXiv.2111.14755]","DOI":"10.5121\/csit.2021.110701"},{"key":"R17","doi-asserted-by":"crossref","unstructured":"Karg P Kreimeier J G\u00f6tzelmann T . Build-and-touch: a low-cost, DIY, open-source approach towards touchable virtual reality. Presented at: The 14th PErvasive Technologies Related to Assistive Environments Conference; Jun 29, 2021:258-259; Corfu, Greece.","DOI":"10.1145\/3453892.3462217"},{"key":"R18","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2022.104089","article-title":"Human motion intention recognition method with visual, audio, and surface electromyography modalities for a mechanical hand in different environments","volume":"79","author":"Xiao","journal-title":"Biomed Signal Process Control"},{"key":"R19","doi-asserted-by":"crossref","unstructured":"Koh WK Nguyen QH Yang YO Xu T Nguyen BP Chua MCH . End-to-end hand rehabilitation system with single-shot gesture classification for stroke patients. In: Soft Computing: Biomedical and Related Applications. Springer; 2021:59-67. [doi: 10.1007\/978-3-030-76620-7_5]","DOI":"10.1007\/978-3-030-76620-7_5"},{"key":"R20","doi-asserted-by":"publisher","DOI":"10.2196\/52661","article-title":"Health care professional-supported co-design of a mime therapy-based serious game for facial rehabilitation","volume":"12","author":"Sousa","journal-title":"JMIR Serious Games"},{"key":"R21","doi-asserted-by":"crossref","unstructured":"Bhatambarekar G Bhinge SA Bhale K Dandgaval A Ramteke K . A study on physiotherapy exercise corrector. Presented at: 2022 International Conference on SMART Generation Computing, Communication and Networking (SMART GENCON); 2022:1-6; Bangalore, India. [doi: 10.1109\/SMARTGENCON56628.2022.10084127]","DOI":"10.1109\/SMARTGENCON56628.2022.10084127"},{"issue":"1","key":"R22","doi-asserted-by":"publisher","DOI":"10.2196\/33771","article-title":"The classification of abnormal hand movement to aid in autism detection: machine learning study","volume":"7","author":"Lakkapragada","journal-title":"JMIR Biomed Eng"},{"issue":"02","key":"R23","doi-asserted-by":"publisher","first-page":"102","DOI":"10.4236\/jcc.2023.112008","article-title":"Detection of 3D human posture based on improved Mediapipe","volume":"11","author":"Lin","journal-title":"JCC"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","first-page":"012004","DOI":"10.1088\/1757-899X\/1279\/1\/012004","article-title":"Knee flexion\/extension angle measurement for gait analysis using machine learning solution \u201cMediaPipe Pose\u201d and its comparison with Kinovea","volume":"1279","author":"Gupta","journal-title":"IOP Conf Ser: Mater Sci Eng"},{"key":"R25","doi-asserted-by":"publisher","DOI":"10.1016\/j.measurement.2023.112826","article-title":"Reliability and validity analysis of MediaPipe-based measurement system for some human rehabilitation motions","volume":"214","author":"Latreche","journal-title":"Measurement (Lond)"},{"key":"R26","doi-asserted-by":"crossref","unstructured":"Palani P Panigrahi S Jammi SA Thondiyath A . Real-time joint angle estimation using mediapipe framework and inertial sensors. Presented at: 2022 IEEE 22nd International Conference on Bioinformatics and Bioengineering (BIBE); 2022:128-133; Taichung, Taiwan. [doi: 10.1109\/BIBE55377.2022.00035]","DOI":"10.1109\/BIBE55377.2022.00035"},{"key":"R27","doi-asserted-by":"crossref","unstructured":"Amprimo G Ferraris C Masi G Pettiti G Priano L . GMH-d: combining google mediapipe and rgb-depth cameras for hand motor skills remote assessment. Presented at: 2022 IEEE International Conference on Digital Health (ICDH); 2022:132-141; Barcelona, Spain. [doi: 10.1109\/ICDH55609.2022.00029]","DOI":"10.1109\/ICDH55609.2022.00029"},{"issue":"20","key":"R28","doi-asserted-by":"publisher","first-page":"7992","DOI":"10.3390\/s22207992","article-title":"Video-based hand movement analysis of Parkinson patients before and after medication using high-frame-rate videos and MediaPipe","volume":"22","author":"G\u00fcney","journal-title":"Sensors (Basel)"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"Ghanbari S Ashtyani ZP Masouleh MT . User identification based on hand geometrical biometrics using media-pipe. Presented at: 2022 30th International Conference on Electrical Engineering (ICEE); 2022:373-378; Tehran, Iran, Islamic Republic of. [doi: 10.1109\/ICEE55646.2022.9827056]","DOI":"10.1109\/ICEE55646.2022.9827056"},{"key":"R30","doi-asserted-by":"crossref","unstructured":"Srinivasan HK Mathunny JJ Devaraj A Karthik V . Validation of an automated step length measurement method in sprinting athletes using computer vision and pose estimation. Presented at: 2023 International Conference on Recent Advances in Electrical, Electronics, Ubiquitous Communication, and Computational Intelligence (RAEEUCCI); 2023:1-5; Chennai, India. [doi: 10.1109\/RAEEUCCI57140.2023.10134177]","DOI":"10.1109\/RAEEUCCI57140.2023.10134177"},{"issue":"1","key":"R31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11332-023-01075-7","article-title":"Validity of neural networks in determining lower limb kinematics in stationary cycling","volume":"20","author":"Bini","journal-title":"Sport Sci Health"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","first-page":"3","DOI":"10.3390\/s23010003","article-title":"Validation of angle estimation based on body tracking data from RGB-D and RGB cameras for biomechanical assessment","volume":"23","author":"Lafayette","journal-title":"Sensors (Basel)"},{"key":"R33","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2019.06.003","article-title":"Hand pose estimation and tracking in real and virtual interaction: a review","volume":"89","author":"Ahmad","journal-title":"Image Vis Comput"},{"issue":"4","key":"R34","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1016\/j.pmr.2015.06.008","article-title":"Upper limb motor impairment after stroke","volume":"26","author":"Raghavan","journal-title":"Phys Med Rehabil Clin N Am"},{"issue":"3","key":"R35","doi-asserted-by":"publisher","first-page":"540","DOI":"10.1016\/j.apmr.2014.09.016","article-title":"Evaluation of measures of upper limb functioning and disability in people with Parkinson disease: a systematic review","volume":"96","author":"Proud","journal-title":"Arch Phys Med Rehabil"},{"issue":"3","key":"R36","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1007\/s11832-014-0593-0","article-title":"Prevalence and pattern of upper limb involvement in cerebral palsy","volume":"8","author":"Makki","journal-title":"J Child Orthop"},{"issue":"1","key":"R37","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-85391-3","article-title":"Unimanual and bimanual motor performance in children with developmental coordination disorder (DCD) provide evidence for underlying motor control deficits","volume":"11","author":"Grohs","journal-title":"Sci Rep"},{"key":"R38","unstructured":"Van Rossum G Drake FL . Python 3 Reference Manual. CreateSpace; 2009."},{"key":"R39","doi-asserted-by":"publisher","DOI":"10.1016\/j.bbr.2018.10.030","article-title":"Movement related sensory feedback is not necessary for learning to execute a motor skill","volume":"359","author":"Ingram","journal-title":"Behav Brain Res"},{"key":"R40","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v031.i07","article-title":"Computing and visualizing dynamic time warping alignments in R: the dtw package","volume":"31","author":"Giorgino","journal-title":"J Stat Softw"},{"issue":"2","key":"R41","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1016\/j.neuron.2013.10.058","article-title":"Scale-invariant movement encoding in the human motor system","volume":"81","author":"Kadmon Harpaz","journal-title":"Neuron"},{"issue":"11","key":"R42","first-page":"120","volume":"25","author":"Bradski","journal-title":"Dr Dobb\u2019s J Softw Tools Prof Program"},{"key":"R43","unstructured":"Zhang F Bazarevsky V Vakunov A et al. Mediapipe hands: on-device real-time hand tracking. arXiv. Preprint posted online on2020. arXiv:2006.10214"},{"key":"R44","unstructured":"Hand landmarks detection guide. MediaPipe Solutions. URL: https:\/\/developers.google.com\/mediapipe\/solutions\/vision\/hand_landmarker#get_started [Accessed 15-11-2024]"},{"key":"R45","unstructured":"Google colaboratory. Welcome to Colab. URL: https:\/\/colab.research.google.com\/ [Accessed 15-11-2024]"},{"issue":"3","key":"R46","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: fundamental algorithms for scientific computing in Python","volume":"17","author":"Virtanen","journal-title":"Nat Methods"},{"key":"R47","volume":"12","author":"Pedregosa","journal-title":"J Mach Learn Res"},{"key":"R48","unstructured":"R core team: R: A language and environment for statistical computing. R. URL: https:\/\/www.R-project.org [Accessed 15-11-2024]"},{"key":"R49","unstructured":"Oksanen J Simpson GL Blanchet FG Kindt R Legendre P Minchin PR . Vegan: community ecology package. R package version 2.6-4. R. URL: https:\/\/CRAN.R-project.org\/package=vegan [Accessed 15-11-2024]"},{"key":"R50","doi-asserted-by":"crossref","unstructured":"Caldwell AR . Exploring equivalence testing with the updated TOSTER R package. PsyArXiv. Preprint posted online onNov17, 2022. [doi: 10.31234\/osf.io\/ty8de]","DOI":"10.31234\/osf.io\/ty8de"},{"key":"R51","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.12995","article-title":"Applications and limitations of current markerless motion capture methods for clinical gait biomechanics","volume":"10","author":"Wade","journal-title":"PeerJ"},{"issue":"3","key":"R52","doi-asserted-by":"publisher","first-page":"1179","DOI":"10.3390\/ijerph19031179","article-title":"Automatic markerless motion detector method against traditional digitisation for 3-dimensional movement kinematic analysis of ball kicking in soccer field context","volume":"19","author":"Palucci Vieira","journal-title":"IJERPH"},{"key":"R53","doi-asserted-by":"publisher","DOI":"10.1016\/j.humov.2014.11.013","article-title":"Recognizing upper limb movements with wrist worn inertial sensors using k-means clustering classification","volume":"40","author":"Biswas","journal-title":"Hum Mov Sci"},{"issue":"8","key":"R54","doi-asserted-by":"publisher","first-page":"1477","DOI":"10.1007\/s11548-022-02637-9","article-title":"Robust hand tracking for surgical telestration","volume":"17","author":"M\u00fcller","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"R55","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2018.12.001","article-title":"Home-based technologies for stroke rehabilitation: a systematic review","volume":"123","author":"Chen","journal-title":"Int J Med Inform"},{"issue":"6","key":"R56","doi-asserted-by":"publisher","first-page":"731","DOI":"10.1111\/1440-1630.12615","article-title":"Effectiveness of occupational therapy intervention for people with Parkinson\u2019s disease: systematic review","volume":"66","author":"Welsby","journal-title":"Aus Occup Therapy J"},{"issue":"12","key":"R57","doi-asserted-by":"publisher","first-page":"1797","DOI":"10.3390\/healthcare11121797","article-title":"Feasibility of a home-based mirror therapy program in children with unilateral spastic cerebral palsy","volume":"11","author":"Ortega-Mart\u00ednez","journal-title":"Healthcare (Basel)"},{"issue":"5","key":"R58","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0284086","article-title":"Combined action observation and motor imagery improves learning of activities of daily living in children with developmental coordination disorder","volume":"18","author":"Scott","journal-title":"PLoS ONE"},{"issue":"5","key":"R59","doi-asserted-by":"publisher","DOI":"10.2196\/36388","article-title":"Evaluation and mitigation of racial bias in clinical machine learning models: scoping review","volume":"10","author":"Huang","journal-title":"JMIR Med Inform"},{"key":"R60","unstructured":"TraceLab. GitHub repository. URL: https:\/\/github.com\/LBRF\/TraceLab [Accessed 23-11-2024]"},{"key":"R61","unstructured":"MediaPipe. GitHub repository. URL: https:\/\/github.com\/NIMBL-UBC\/MediaPipe [Accessed 23-11-2024]"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/formative.jmir.org\/2024\/1\/e56682","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:45:58Z","timestamp":1782841558000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2024\/1\/e56682"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,17]]},"references-count":61,"URL":"https:\/\/doi.org\/10.2196\/56682","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,17]]},"article-number":"v8i15e56682"}}