{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T06:16:32Z","timestamp":1791612992219,"version":"4.3.4"},"reference-count":37,"publisher":"Public Library of Science (PLoS)","issue":"4","license":[{"start":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T00:00:00Z","timestamp":1776988800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP25K21528"],"award-info":[{"award-number":["JP25K21528"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100009427","name":"Telecommunications Advancement Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100009427","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>Humans often make gestures while speaking. These gestures have been extensively researched in psychology and cognitive science, revealing their functions and roles in communication. Despite being produced in three-dimensional (3D) space, gestures and gesture space have primarily been measured and analyzed using two-dimensional planes. Optical motion capture (MoCap) is commonly used for 3D gesture measurement. However, MoCap is expensive, requires a large space, and the attachment of retroreflective markers can interfere with the natural generation of gestures by the speaker. Recent advances in deep learning-based human pose estimation (HPE) offer promising alternatives by enabling 3D keypoint estimation from standard video cameras. This study investigates the accuracy of four HPEs\u2014two monocular and two stereo with triangulation methods\u2014in estimating upper-body keypoints commonly used in natural scenarios for gesture research. Ten participants were recorded performing gesture-rich speech by both a MoCap system and three video cameras. We compared 13 keypoints, including the wrists, elbows, shoulders, fingers, and face, with MoCap data, using Euclidean distance as an error metric. Statistical analyses revealed that stereo methods significantly outperformed monocular methods across all keypoints. The most accurate method achieved an average error of 49.4 mm with respect to MoCap, suggesting sufficient accuracy for practical gesture analysis. Additionally, we visualized 3D gesture spaces and found a 75.4% overlap between HPE and MoCap at a voxel size of 50 mm, indicating high spatial agreement. Our findings demonstrate that stereo HPE methods are viable, cost-effective alternatives to MoCap in gesture-related applications. Our ultimate objective is to create a toolbox that is accurate and easy to use for measuring human gestures in 3D space by exploiting the recent advances in HPE as an alternative to MoCap. Such a toolbox will be suitable for non-experts in machine learning, and these results lay the foundation for it.<\/jats:p>","DOI":"10.1371\/journal.pone.0347288","type":"journal-article","created":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T17:22:51Z","timestamp":1777051371000},"page":"e0347288","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":1,"title":["Comparison of deep learning-based three-dimensional human pose estimation methods with motion capture for gesture research"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4420-6361","authenticated-orcid":true,"given":"Naoto","family":"Ienaga","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5061-1657","authenticated-orcid":true,"given":"Kazuki","family":"Sekine","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,4,24]]},"reference":[{"issue":"1","key":"pone.0347288.ref001","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1075\/gest.19031.tel","article-title":"Handling language: How future language teachers adapt their gestures to their interlocutor","volume":"20","author":"M Tellier","year":"2021","journal-title":"Gesture"},{"key":"pone.0347288.ref002","doi-asserted-by":"crossref","unstructured":"Fern\u2019ndez-Baena A, Susin A, Lligadas X. Biomechanical Validation of Upper-Body and Lower-Body Joint Movements of Kinect Motion Capture Data for Rehabilitation Treatments. In: 2012 Fourth International Conference on Intelligent Networking and Collaborative Systems, 2012. 656\u201361. https:\/\/doi.org\/10.1109\/incos.2012.66","DOI":"10.1109\/iNCoS.2012.66"},{"key":"pone.0347288.ref003","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.cognition.2018.04.003","article-title":"Communicative intent modulates production and comprehension of actions and gestures: A Kinect study","volume":"180","author":"JP Trujillo","year":"2018","journal-title":"Cognition"},{"issue":"2","key":"pone.0347288.ref004","doi-asserted-by":"crossref","first-page":"769","DOI":"10.3758\/s13428-018-1086-8","article-title":"Toward the markerless and automatic analysis of kinematic features: A toolkit for gesture and movement research","volume":"51","author":"JP Trujillo","year":"2019","journal-title":"Behav Res Methods"},{"issue":"18","key":"pone.0347288.ref005","doi-asserted-by":"crossref","first-page":"5104","DOI":"10.3390\/s20185104","article-title":"Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect v2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study","volume":"20","author":"JA Albert","year":"2020","journal-title":"Sensors (Basel)"},{"key":"pone.0347288.ref006","volume-title":"Hand and mind: What gestures reveal about thought","author":"D McNeill","year":"1992"},{"issue":"4","key":"pone.0347288.ref007","doi-asserted-by":"crossref","first-page":"1783","DOI":"10.3758\/s13428-020-01350-2","article-title":"Speeding up the detection of non-iconic and iconic gestures (SPUDNIG): A toolkit for the automatic detection of hand movements and gestures in video data","volume":"52","author":"J Ripperda","year":"2020","journal-title":"Behav Res Methods"},{"issue":"1","key":"pone.0347288.ref008","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3603618","article-title":"Deep Learning-based Human Pose Estimation: A Survey","volume":"56","author":"C Zheng","year":"2023","journal-title":"ACM Comput Surv"},{"key":"pone.0347288.ref009","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIP.2023.3334954","article-title":"Dynamic Dense Graph Convolutional Network for Skeleton-Based Human Motion Prediction","volume":"33","author":"X Wang","year":"2024","journal-title":"IEEE Trans Image Process"},{"key":"pone.0347288.ref010","first-page":"345","volume-title":"Realistic human motion generation with cross-diffusion models. In: European Conference on Computer Vision","author":"Z Ren","year":"2024"},{"key":"pone.0347288.ref011","unstructured":"Gan Q, Ren Y, Zhang C, Ye Z, Xie P, Yin X, et al. Humandit: Pose-guided diffusion transformer for long-form human motion video generation. In: 2025. https:\/\/arxiv.org\/abs\/250204847"},{"issue":"1","key":"pone.0347288.ref012","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TPAMI.2019.2929257","article-title":"OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields","volume":"43","author":"Z Cao","year":"2021","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"4","key":"pone.0347288.ref013","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1008935","article-title":"Two-dimensional video-based analysis of human gait using pose estimation","volume":"17","author":"J Stenum","year":"2021","journal-title":"PLoS Comput Biol"},{"issue":"6","key":"pone.0347288.ref014","doi-asserted-by":"crossref","first-page":"7157","DOI":"10.1109\/TPAMI.2022.3222784","article-title":"AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time","volume":"45","author":"H-S Fang","year":"2023","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"9","key":"pone.0347288.ref015","doi-asserted-by":"crossref","first-page":"1281","DOI":"10.1038\/s41593-018-0209-y","article-title":"DeepLabCut: markerless pose estimation of user-defined body parts with deep learning","volume":"21","author":"A Mathis","year":"2018","journal-title":"Nat Neurosci"},{"issue":"1","key":"pone.0347288.ref016","doi-asserted-by":"crossref","first-page":"20673","DOI":"10.1038\/s41598-021-00212-x","article-title":"The accuracy of several pose estimation methods for 3D joint centre localisation","volume":"11","author":"L Needham","year":"2021","journal-title":"Sci Rep"},{"key":"pone.0347288.ref017","doi-asserted-by":"crossref","first-page":"50","DOI":"10.3389\/fspor.2020.00050","article-title":"Evaluation of 3D markerless motion capture accuracy using OpenPose with multiple video cameras","volume":"2","author":"N Nakano","year":"2020","journal-title":"Frontiers in Sports and Active Living"},{"key":"pone.0347288.ref018","doi-asserted-by":"crossref","unstructured":"Labuguen RT, Negrete SB, Kogami T, Ingco WEM, Shibata T. Performance Evaluation of Markerless 3D Skeleton Pose Estimates with Pop Dance Motion Sequence. In: 2020 Joint 9th International Conference on Informatics, Electronics & Vision (ICIEV) and 2020 4th International Conference on Imaging, Vision & Pattern Recognition (icIVPR), 2020. 1\u20137. https:\/\/doi.org\/10.1109\/icievicivpr48672.2020.9306581","DOI":"10.1109\/ICIEVicIVPR48672.2020.9306581"},{"issue":"3","key":"pone.0347288.ref019","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1007\/s10579-022-09586-4","article-title":"Semi-automation of gesture annotation by machine learning and human collaboration","volume":"56","author":"N Ienaga","year":"2022","journal-title":"Lang Resources & Evaluation"},{"key":"pone.0347288.ref020","doi-asserted-by":"crossref","unstructured":"Chang CY, Lange B, Zhang M, Koenig S, Requejo P, Somboon N, et al. Towards pervasive physical rehabilitation using Microsoft Kinect. In: 2012 6th international conference on pervasive computing technologies for healthcare (PervasiveHealth) and workshops. IEEE. 2012. 159\u201362.","DOI":"10.4108\/icst.pervasivehealth.2012.248714"},{"issue":"2","key":"pone.0347288.ref021","doi-asserted-by":"crossref","first-page":"155014771769608","DOI":"10.1177\/1550147717696083","article-title":"Dance motion capture and composition using multiple RGB and depth sensors","volume":"13","author":"Y Kim","year":"2017","journal-title":"International Journal of Distributed Sensor Networks"},{"key":"pone.0347288.ref022","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2021.3055793","article-title":"The Influence of Autofocus Lenses in the Camera Calibration Process","volume":"70","author":"C Ricolfe-Viala","year":"2021","journal-title":"IEEE Trans Instrum Meas"},{"key":"pone.0347288.ref023","unstructured":"Jiang T, Lu P, Zhang L, Ma N, Han R, Lyu C. RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose. In: 2023. https:\/\/arxiv.org\/abs\/2303.07399"},{"key":"pone.0347288.ref024","unstructured":"MMPose Contributors. OpenMMLab Pose Estimation Toolbox and Benchmark, 2020. https:\/\/github.com\/open-mmlab\/mmpose"},{"key":"pone.0347288.ref025","unstructured":"Qammaz A, Argyros AA. MocapNET: Ensemble of SNN Encoders for 3D Human Pose Estimation in RGB Images. In: 2019. 46."},{"issue":"11","key":"pone.0347288.ref026","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Z Zhang","year":"2000","journal-title":"IEEE Trans Pattern Anal Machine Intell"},{"key":"pone.0347288.ref027","unstructured":"Lugaresi C, Tang J, Nash H, McClanahan C, Uboweja E, Hays M. Mediapipe: A framework for building perception pipelines. 2019. https:\/\/arxiv.org\/abs\/1906.08172"},{"key":"pone.0347288.ref028","doi-asserted-by":"crossref","DOI":"10.1155\/2022\/6952999","article-title":"Development and verification of postural control assessment using deep-learning-based pose estimators: towards clinical applications","volume":"2022","author":"N Ienaga","year":"2022","journal-title":"Occupational Therapy International"},{"issue":"5","key":"pone.0347288.ref029","doi-asserted-by":"crossref","first-page":"698","DOI":"10.1109\/TPAMI.1987.4767965","article-title":"Least-squares fitting of two 3-d point sets","volume":"9","author":"KS Arun","year":"1987","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"pone.0347288.ref030","first-page":"59","article-title":"An introduction to the statistical free software HAD: Suggestions to improve teaching, learning and practice data analysis (Japanese)","volume":"1","author":"H Shimizu","year":"2016","journal-title":"Journal of Media, Information and Communication"},{"issue":"4","key":"pone.0347288.ref031","first-page":"148","article-title":"Hand length and hand breadth: a study of correlation statistics among human population","volume":"4","author":"S Dey","year":"2015","journal-title":"Int J Sci Res"},{"key":"pone.0347288.ref032","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.ergon.2015.10.004","article-title":"An anthropometric survey of Korean hand and hand shape types","volume":"53","author":"S Jee","year":"2016","journal-title":"International Journal of Industrial Ergonomics"},{"key":"pone.0347288.ref033","doi-asserted-by":"crossref","unstructured":"Kocabas M, Athanasiou N, Black MJ. VIBE: Video Inference for Human Body Pose and Shape Estimation. In: 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 5252\u201362. https:\/\/doi.org\/10.1109\/cvpr42600.2020.00530","DOI":"10.1109\/CVPR42600.2020.00530"},{"key":"pone.0347288.ref034","doi-asserted-by":"crossref","unstructured":"Li J, Xu C, Chen Z, Bian S, Yang L, Lu C. HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation. In: 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 3382\u201392. https:\/\/doi.org\/10.1109\/cvpr46437.2021.00339","DOI":"10.1109\/CVPR46437.2021.00339"},{"key":"pone.0347288.ref035","unstructured":"Seethapathi N, Wang S, Saluja R, Blohm G, Kording KP. Movement science needs different pose tracking algorithms. In: 2019. https:\/\/arxiv.org\/abs\/1907.10226"},{"key":"pone.0347288.ref036","doi-asserted-by":"crossref","first-page":"181","DOI":"10.3389\/fbioe.2020.00181","article-title":"3D Tracking of Human Motion Using Visual Skeletonization and Stereoscopic Vision","volume":"8","author":"M Zago","year":"2020","journal-title":"Front Bioeng Biotechnol"},{"issue":"19","key":"pone.0347288.ref037","doi-asserted-by":"crossref","first-page":"6530","DOI":"10.3390\/s21196530","article-title":"Pose2Sim: An End-to-End Workflow for 3D Markerless Sports Kinematics-Part 1: Robustness","volume":"21","author":"D Pagnon","year":"2021","journal-title":"Sensors (Basel)"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0347288","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T17:22:57Z","timestamp":1777051377000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0347288"}},"subtitle":[],"editor":[{"given":"Jyotindra","family":"Narayan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,4,24]]},"references-count":37,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4,24]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0347288","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,24]]}}}