{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T22:57:07Z","timestamp":1787007427640,"version":"build-2736575974"},"reference-count":50,"publisher":"Public Library of Science (PLoS)","issue":"8","license":[{"start":{"date-parts":[[2025,8,7]],"date-time":"2025-08-07T00:00:00Z","timestamp":1754524800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004504","name":"Lietuvos Mokslo Taryba","doi-asserted-by":"publisher","award":["S-PD-24-29"],"award-info":[{"award-number":["S-PD-24-29"]}],"id":[{"id":"10.13039\/501100004504","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>Accurate and reliable skeletal motion tracking is essential for rehabilitation monitoring, enabling objective assessment of patient progress and facilitating telerehabilitation applications. Traditional marker-based motion capture systems, while highly accurate, are costly and impractical for home rehabilitation, whereas marker-less methods often suffer from depth estimation errors and occlusions. Recent studies have explored various computer vision and deep learning approaches for human pose estimation, yet challenges remain in ensuring robust depth accuracy and tracking under occlusion conditions. This study proposes a three-dimensional human skeleton tracking system for upper limb activities that integrates temporal and spatial synchronization to improve depth estimation accuracy for rehabilitation exercises. The proposed system combines a 90\u00b0 secondary camera to compensate for the depth prediction inaccuracies inherent in single-camera systems, reducing error margins by up to 0.4 m. In addition, a linear regression-based depth error correction model is implemented to refine depth coordinates, further improving tracking precision. The Kalman filtering framework is employed to enhance temporal consistency, allowing real-time interpolation of missing joint positions. Experimental results demonstrate that the proposed method significantly reduces depth estimation errors of the elbow and wrist joint (<jats:italic>p<\/jats:italic> &lt; 0.001) compared to single camera setups, particularly in scenarios involving occlusions and non-frontal perspectives. This study provides a cost-effective and scalable solution for remote patient monitoring and motor function evaluation.<\/jats:p>","DOI":"10.1371\/journal.pone.0328969","type":"journal-article","created":{"date-parts":[[2025,8,7]],"date-time":"2025-08-07T17:34:25Z","timestamp":1754588065000},"page":"e0328969","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["Robust skeletal motion tracking using temporal and spatial synchronization of two video streams"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1588-6572","authenticated-orcid":true,"given":"Vytautas","family":"Abromavi\u010dius","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-6382-9016","authenticated-orcid":true,"given":"Ervinas","family":"Gisleris","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristina","family":"Daunoravi\u010dien\u0117","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jurgita","family":"\u017di\u017eien\u0117","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Art\u016bras","family":"Serackis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rytis","family":"Maskeli\u016bnas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,8,7]]},"reference":[{"key":"pone.0328969.ref001","article-title":"Opportunities for improving motor assessment and rehabilitation after stroke by leveraging video-based pose estimation","volume":"102","author":"KM Cherry-Allen","year":"2023","journal-title":"Am J Phys Med Rehabil."},{"key":"pone.0328969.ref002","doi-asserted-by":"crossref","unstructured":"Deshmukh SL, Fernandes F, Kulkarni S, Patil A, Jabade V, Madake J. Patient monitoring system. In: 2022 IEEE 2nd Mysore Sub Section International Conference (MysuruCon). 2022. p. 1\u20136. https:\/\/doi.org\/10.1109\/mysurucon55714.2022.9972563","DOI":"10.1109\/MysuruCon55714.2022.9972563"},{"key":"pone.0328969.ref003","doi-asserted-by":"crossref","unstructured":"Wang Y. Monocular 2D and 3D human pose estimation review. In: International Conference on Computer Vision, Application, and Algorithm (CVAA 2022). 2023. p. 45. https:\/\/doi.org\/10.1117\/12.2673629","DOI":"10.1117\/12.2673629"},{"issue":"2","key":"pone.0328969.ref004","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1109\/THMS.2022.3142108","article-title":"Multiview video-based 3-D pose estimation of patients in computer-assisted rehabilitation environment (CAREN)","volume":"52","author":"W Xu","year":"2022","journal-title":"IEEE Trans Hum-Mach Syst."},{"issue":"2","key":"pone.0328969.ref005","doi-asserted-by":"crossref","first-page":"61","DOI":"10.36548\/jiip.2022.2.001","article-title":"IoT based pose detection of patients in rehabilitation centre by PoseNet estimation control","volume":"4","author":"R Asokan","year":"2022","journal-title":"JIIP."},{"issue":"1","key":"pone.0328969.ref006","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1038\/s41597-022-01690-y","article-title":"From raw measurements to human pose - a dataset with low-cost and high-end inertial-magnetic sensor data","volume":"9","author":"M Palermo","year":"2022","journal-title":"Sci Data."},{"key":"pone.0328969.ref007","doi-asserted-by":"crossref","unstructured":"Sheng B, Cheng J, Tao J, Wang Y. Marker-less motion capture technology based on binocular stereo vision and deep learning. In: 2022 28th International Conference on Mechatronics and Machine Vision in Practice (M2VIP). 2022. p. 1\u20136.","DOI":"10.1109\/M2VIP55626.2022.10041084"},{"issue":"2","key":"pone.0328969.ref008","doi-asserted-by":"crossref","first-page":"1116","DOI":"10.3390\/app13021116","article-title":"Biomac3d: 2d-to-3d human pose analysis model for tele-rehabilitation based on pareto optimized deep-learning architecture","volume":"13","author":"R Maskeli\u016bnas","year":"2023","journal-title":"Applied sciences."},{"key":"pone.0328969.ref009","doi-asserted-by":"crossref","first-page":"115498","DOI":"10.1016\/j.eswa.2021.115498","article-title":"Real-time human pose estimation on a smart walker using convolutional neural networks","volume":"184","author":"M Palermo","year":"2021","journal-title":"Expert Syst Appl."},{"key":"pone.0328969.ref010","doi-asserted-by":"crossref","unstructured":"Niu Y, She J, Xu C. A survey on IMU-and-vision-based human pose estimation for rehabilitation. In: 2022 41st Chinese Control Conference (CCC). 2022. p. 6410\u20135.","DOI":"10.23919\/CCC55666.2022.9901525"},{"issue":"2","key":"pone.0328969.ref011","doi-asserted-by":"crossref","first-page":"339","DOI":"10.3390\/electronics12020339","article-title":"BiomacVR: A virtual reality-based system for precise human posture and motion analysis in rehabilitation exercises using depth sensors","volume":"12","author":"R Maskeliunas","year":"2023","journal-title":"Electronics."},{"issue":"5","key":"pone.0328969.ref012","first-page":"1079","article-title":"Estimating human poses using deep learning model","volume":"27","author":"F Muradli","year":"2023","journal-title":"Sakarya Univ J Sci."},{"issue":"1","key":"pone.0328969.ref013","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3603618","article-title":"Deep learning-based human pose estimation: a survey","volume":"56","author":"C Zheng","year":"2023","journal-title":"ACM Comput Surv."},{"key":"pone.0328969.ref014","article-title":"Deep learning-based pose estimation in providing feedback for physical movement: a review","author":"A Tharatipyakul","year":"2023","journal-title":"Preprints."},{"issue":"72","key":"pone.0328969.ref015","doi-asserted-by":"crossref","first-page":"75","DOI":"10.26906\/SUNZ.2023.2.075","article-title":"Human pose estimation system using deep learning algorithms","volume":"2","author":"D Vyshnivskyi","year":"2023","journal-title":"CNCS."},{"key":"pone.0328969.ref016","doi-asserted-by":"crossref","unstructured":"Martinelli G, Santoro L, Nardello M, Brunelli D, Fontanelli D, Conci N. UNPOSED: an ultra-wideband network for pose estimation with deep learning. In: 2023 IEEE International Workshop on Metrology for Industry 4.0 & IoT (MetroInd4.0&IoT). 2023. p. 299\u2013304.","DOI":"10.1109\/MetroInd4.0IoT57462.2023.10180019"},{"key":"pone.0328969.ref017","doi-asserted-by":"crossref","unstructured":"Shah M, Gandhi K, Pandhi BM, Padhiyar P, Degadwala S. Computer vision & deep learning-based real-time and pre-recorded human pose estimation. In: 2023 2nd International Conference on Applied Artificial Intelligence and Computing (ICAAIC). 2023. p. 313\u20139.","DOI":"10.1109\/ICAAIC56838.2023.10141279"},{"issue":"4","key":"pone.0328969.ref018","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s42979-023-01796-z","article-title":"Deep learning for head pose estimation: a survey","volume":"4","author":"A Asperti","year":"2023","journal-title":"SN Comput Sci."},{"key":"pone.0328969.ref019","doi-asserted-by":"crossref","unstructured":"XIA Y, Jiang X, Yan L. Research advanced in human pose estimation based on deep learning. In: Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022). 2023. 155.https:\/\/doi.org\/10.1117\/12.2667904","DOI":"10.1117\/12.2667904"},{"key":"pone.0328969.ref020","doi-asserted-by":"crossref","first-page":"1239","DOI":"10.54097\/hset.v39i.6734","article-title":"Human pose estimation based on HRNet and feature pyramids","volume":"39","author":"F Dong","year":"2023","journal-title":"HSET."},{"key":"pone.0328969.ref021","doi-asserted-by":"crossref","first-page":"11795727211022330","DOI":"10.1177\/11795727211022330","article-title":"The potential of computer vision-based marker-less human motion analysis for rehabilitation","volume":"10","author":"T Hellsten","year":"2021","journal-title":"Rehabil Process Outcome."},{"issue":"2","key":"pone.0328969.ref022","doi-asserted-by":"crossref","first-page":"906","DOI":"10.3390\/app15020906","article-title":"Enhanced human skeleton tracking for improved joint position and depth accuracy in rehabilitation exercises","volume":"15","author":"V Abromavi\u010dius","year":"2025","journal-title":"Appl Sci."},{"key":"pone.0328969.ref023","doi-asserted-by":"crossref","unstructured":"Zheng C, Zhu S, Mendieta M, Yang T, Chen C, Ding Z. 3D human pose estimation with spatial and temporal transformers. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV). 2021. p. 11656\u201365.","DOI":"10.1109\/ICCV48922.2021.01145"},{"key":"pone.0328969.ref024","doi-asserted-by":"crossref","unstructured":"Li J, Xu C, Chen Z, Bian S, Yang L, Lu C. Hybrik: a hybrid analytical-neural inverse kinematics solution for 3D human pose and shape estimation. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 2021. p. 3383\u201393.","DOI":"10.1109\/CVPR46437.2021.00339"},{"key":"pone.0328969.ref025","doi-asserted-by":"crossref","unstructured":"Lin K, Wang L, Liu Z. Mesh graphormer. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV). 2021. p. 12939\u201348.","DOI":"10.1109\/ICCV48922.2021.01270"},{"key":"pone.0328969.ref026","doi-asserted-by":"crossref","unstructured":"Choi H, Moon G, Lee KM. Pose2mesh: Graph convolutional network for 3D human pose and mesh recovery from a 2D human pose. In: Computer Vision\u2013ECCV 2020 : 16th European Conference, Glasgow, UK, August 23\u201328, 2020, Proceedings, Part VII. Springer; 2020. p. 769\u201387.","DOI":"10.1007\/978-3-030-58571-6_45"},{"key":"pone.0328969.ref027","article-title":"Liftformer: 3D human pose estimation using attention models","author":"A Llopart","year":"2020","journal-title":"arXiv preprint"},{"issue":"11","key":"pone.0328969.ref028","doi-asserted-by":"crossref","first-page":"4109","DOI":"10.3390\/s22114109","article-title":"Top-down system for multi-person 3D absolute pose estimation from monocular videos","volume":"22","author":"A El Kaid","year":"2022","journal-title":"Sensors (Basel)."},{"key":"pone.0328969.ref029","doi-asserted-by":"crossref","unstructured":"Sosa J, Hogg D. Self-supervised 3D human pose estimation from a single image. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 2023. p. 4788\u201397.","DOI":"10.1109\/CVPRW59228.2023.00507"},{"key":"pone.0328969.ref030","doi-asserted-by":"crossref","first-page":"103992","DOI":"10.1016\/j.cviu.2024.103992","article-title":"SlowFastFormer for 3D human pose estimation","volume":"243","author":"L Zhou","year":"2024","journal-title":"Comput Vis Image Understand."},{"key":"pone.0328969.ref031","doi-asserted-by":"crossref","unstructured":"Li W, Liu M, Liu H, Wang P, Cai J, Sebe N. Hourglass tokenizer for efficient transformer-based 3D human pose estimation. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 2024. p. 604\u201313.","DOI":"10.1109\/CVPR52733.2024.00064"},{"key":"pone.0328969.ref032","doi-asserted-by":"crossref","unstructured":"Mehraban S, Adeli V, Taati B. Motionagformer: enhancing 3D human pose estimation with a transformer-gcnformer network. In: Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision. 2024. p. 6920\u201330.","DOI":"10.1109\/WACV57701.2024.00677"},{"key":"pone.0328969.ref033","doi-asserted-by":"crossref","first-page":"103687","DOI":"10.1016\/j.compbiomed.2020.103687","article-title":"A review of computational approaches for evaluation of rehabilitation exercises","volume":"119","author":"Y Liao","year":"2020","journal-title":"Comput Biol Med."},{"issue":"11","key":"pone.0328969.ref034","doi-asserted-by":"crossref","first-page":"5121","DOI":"10.3390\/s23115121","article-title":"Deep learning for human activity recognition on 3D human skeleton: survey and comparative study","volume":"23","author":"H-C Nguyen","year":"2023","journal-title":"Sensors (Basel)."},{"key":"pone.0328969.ref035","doi-asserted-by":"crossref","unstructured":"Sun X, Liu B, Ye X, Xu R, Li H. Self-supervised monocular depth estimation from videos via pose-adaptive reconstruction. In: ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2025. p. 1\u20135. https:\/\/doi.org\/10.1109\/icassp49660.2025.10889476","DOI":"10.1109\/ICASSP49660.2025.10889476"},{"issue":"3","key":"pone.0328969.ref036","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pdig.0000467","article-title":"Clinical gait analysis using video-based pose estimation: multiple perspectives, clinical populations, and measuring change","volume":"3","author":"J Stenum","year":"2024","journal-title":"PLOS Digit Health."},{"issue":"4","key":"pone.0328969.ref037","doi-asserted-by":"crossref","first-page":"2700","DOI":"10.3390\/app13042700","article-title":"Human pose estimation using mediapipe pose and optimization method based on a humanoid model","volume":"13","author":"JW Kim","year":"2023","journal-title":"Appl Sci."},{"issue":"15","key":"pone.0328969.ref038","doi-asserted-by":"crossref","first-page":"3774","DOI":"10.1049\/iet-ipr.2020.0606","article-title":"EKFPnP: extended Kalman filter for camera pose estimation in a sequence of images","volume":"14","author":"MA Mehralian","year":"2020","journal-title":"IET Image Process."},{"issue":"1","key":"pone.0328969.ref039","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40648-020-00185-y","article-title":"Pose estimation by extended Kalman filter using noise covariance matrices based on sensor output","volume":"7","author":"A Saito","year":"2020","journal-title":"Robomech J."},{"issue":"2","key":"pone.0328969.ref040","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1111\/cgf.13386","article-title":"State of the art on 3D reconstruction with RGB-D cameras","volume":"37","author":"M Zollh\u00f6fer","year":"2018","journal-title":"Comput Graph Forum."},{"key":"pone.0328969.ref041","doi-asserted-by":"crossref","first-page":"109720","DOI":"10.1016\/j.ymssp.2022.109720","article-title":"A Gaussian process regression reduced order model for geometrically nonlinear structures","volume":"184","author":"K Park","year":"2023","journal-title":"Mech Syst Signal Process."},{"issue":"193","key":"pone.0328969.ref042","first-page":"1","article-title":"Gaussian process regression: optimality, robustness, and relationship with kernel ridge regression","volume":"23","author":"W Wang","year":"2022","journal-title":"J Mach Learn Res."},{"key":"pone.0328969.ref043","doi-asserted-by":"crossref","first-page":"111463","DOI":"10.1016\/j.knosys.2024.111463","article-title":"A lightweight multi-layer perceptron for efficient multivariate time series forecasting","volume":"288","author":"Z Wang","year":"2024","journal-title":"Knowl-Based Syst."},{"issue":"11","key":"pone.0328969.ref044","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Z Zhang","year":"2002","journal-title":"IEEE Trans Pattern Anal Mach Intell."},{"key":"pone.0328969.ref045","doi-asserted-by":"crossref","unstructured":"Olson E. AprilTag: a robust and flexible visual fiducial system. In: 2011 IEEE International Conference on Robotics and Automation. 2011. p. 3400\u20137.","DOI":"10.1109\/ICRA.2011.5979561"},{"issue":"6","key":"pone.0328969.ref046","doi-asserted-by":"crossref","first-page":"2694","DOI":"10.1109\/TIP.2017.2686001","article-title":"A precision analysis of camera distortion models","volume":"26","author":"Z Tang","year":"2017","journal-title":"IEEE Trans Image Process."},{"issue":"4","key":"pone.0328969.ref047","doi-asserted-by":"crossref","first-page":"2396","DOI":"10.1109\/TPAMI.2023.3330944","article-title":"Monocular depth estimation: a thorough review","volume":"46","author":"V Arampatzakis","year":"2024","journal-title":"IEEE Trans Pattern Anal Mach Intell."},{"key":"pone.0328969.ref048","doi-asserted-by":"crossref","unstructured":"Zeng A, Yang L, Ju X, Li J, Wang J, Xu Q. Smoothnet: A plug-and-play network for refining human poses in videos. In: European Conference on Computer Vision. 2022. p. 625\u201342.","DOI":"10.1007\/978-3-031-20065-6_36"},{"key":"pone.0328969.ref049","doi-asserted-by":"crossref","first-page":"109598","DOI":"10.1016\/j.sigpro.2024.109598","article-title":"FLK: a filter with learned kinematics for real-time 3D human pose estimation","volume":"224","author":"E Martini","year":"2024","journal-title":"Signal Process."},{"key":"pone.0328969.ref050","doi-asserted-by":"crossref","first-page":"868","DOI":"10.1109\/LSP.2023.3295756","article-title":"HiMoReNet: a hierarchical model for human motion refinement","volume":"30","author":"Z Wang","year":"2023","journal-title":"IEEE Signal Process Lett."}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0328969","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,7]],"date-time":"2025-08-07T17:34:36Z","timestamp":1754588076000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0328969"}},"subtitle":[],"editor":[{"given":"Jyotindra","family":"Narayan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,8,7]]},"references-count":50,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8,7]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0328969","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,7]]}}}