{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T11:37:16Z","timestamp":1784547436119,"version":"3.55.0"},"reference-count":37,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T00:00:00Z","timestamp":1762041600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012190","name":"Ministry of Science and Higher Education of the Russian Federation","doi-asserted-by":"publisher","award":["FEWG-2024-0001"],"award-info":[{"award-number":["FEWG-2024-0001"]}],"id":[{"id":"10.13039\/501100012190","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Physical therapy is a critical component of medical rehabilitation, aiding recovery from conditions such as stroke, spinal cord injuries, and musculoskeletal disorders. Effective rehabilitation requires precise monitoring of patient performance to ensure exercises are executed correctly and progress is accurately assessed. This paper presents a novel automated system for controlling the rehabilitation process and evaluating physical therapy exercise quality using computer vision and a customized Human Skeleton-based Balanced Time Warping algorithm. The proposed method quantitatively assesses the similarity between a physiotherapist and patient performance by analyzing skeletal motion data extracted from RGB-D video sequences without requiring pre-alignment or sensor-specific calibration. A motion-dependent, weighted Euclidean distance between 3D skeletal models is used to compute pose dissimilarity, while a modified DTW approach aligns temporal sequences and evaluates dynamic consistency. The total dissimilarity measure is a balanced combination of posture (DP) and dynamics (DT) components. Evaluated on a custom dataset of 136 video recordings from 23 participants performing exercises in sitting and standing positions under varying performance accuracy levels (\u201cgood,\u201d \u201cintermediate,\u201d and \u201cbad\u201d), the system demonstrates the strong clustering of accuracy levels. Proposed dissimilarity, together with a fixed reference element (physiotherapist), induces a natural non-strict order on the set of distances between patients and physiotherapists. A high value of Spearman\u2019s rank correlation coefficient between computed dissimilarity and execution accuracy (0.977) indicates that this method is suitable for assessing exercise performance accuracy and for adequately evaluating the patient\u2019s rehabilitation progress. The method enables objective, real-time feedback, reduces therapist workload, and supports remote monitoring, offering a scalable solution for personalized rehabilitation. Future work will involve clinical validation with post-stroke and cardiac patients.<\/jats:p>","DOI":"10.3390\/s25216696","type":"journal-article","created":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T18:21:46Z","timestamp":1762194106000},"page":"6696","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Automated Control of Rehabilitation Process in Physical Therapy Using a Novel Human Skeleton-Based Balanced Time Warping Algorithm"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0410-7705","authenticated-orcid":false,"given":"Oleg","family":"Seredin","sequence":"first","affiliation":[{"name":"Laboratory of Cognitive Technologies and Simulation Systems, Tula State University, Lenin Ave. 92, 300012 Tula, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3193-583X","authenticated-orcid":false,"given":"Andrey","family":"Kopylov","sequence":"additional","affiliation":[{"name":"Laboratory of Cognitive Technologies and Simulation Systems, Tula State University, Lenin Ave. 92, 300012 Tula, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6693-4098","authenticated-orcid":false,"given":"Egor","family":"Surkov","sequence":"additional","affiliation":[{"name":"Laboratory of Cognitive Technologies and Simulation Systems, Tula State University, Lenin Ave. 92, 300012 Tula, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7319-7068","authenticated-orcid":false,"given":"Nikita","family":"Mityugov","sequence":"additional","affiliation":[{"name":"Laboratory of Cognitive Technologies and Simulation Systems, Tula State University, Lenin Ave. 92, 300012 Tula, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexei","family":"Tokarev","sequence":"additional","affiliation":[{"name":"Medical Institute, Tula State University, Lenin Ave. 92, 300012 Tula, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9725-9582","authenticated-orcid":false,"given":"Parama","family":"Bagchi","sequence":"additional","affiliation":[{"name":"Department of CSE, RCC Institute of Information Technology, Beliaghata, Kolkata 700015, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1163-6413","authenticated-orcid":false,"given":"Debotosh","family":"Bhattacharjee","sequence":"additional","affiliation":[{"name":"Department of CSE, Jadavpur University, Kolkata 700032, India"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1093\/eurheartj\/ehac747","article-title":"Exercise-Based Cardiac Rehabilitation for Coronary Heart Disease: A Meta-Analysis","volume":"44","author":"Dibben","year":"2023","journal-title":"Eur. Heart J."},{"key":"ref_2","unstructured":"(2025, July 07). Physical Medicine and Rehabilitation: MedlinePlus Medical Encyclopedia, Available online: https:\/\/medlineplus.gov\/ency\/article\/007448.htm."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.artmed.2008.11.007","article-title":"Matching Incomplete Time Series with Dynamic Time Warping: An Algorithm and an Application to Post-Stroke Rehabilitation","volume":"45","author":"Tormene","year":"2009","journal-title":"Artif. Intell. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1080\/07370024.2015.1093419","article-title":"Automated Rehabilitation System: Movement Measurement and Feedback for Patients and Physiotherapists in the Rehabilitation Clinic","volume":"31","author":"Lam","year":"2016","journal-title":"Hum. Comput. Interact."},{"key":"ref_5","first-page":"189","article-title":"A Skeleton Features-Based Fall Detection Using Microsoft Kinect v2 with One Class-Classifier Outlier Removal","volume":"42","author":"Seredin","year":"2019","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.\u2014ISPRS Arch."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Surkov, E., Seredin, O., Kopylov, A., Kushnir, O., and Semenishchev, E. (2024, January 1\u20135). Standardizing Skeletal Models for Fall Detection. Proceedings of the ICPR 2024 International Workshops and Challenges, Kolkata, India.","DOI":"10.1007\/978-3-031-87663-9_29"},{"key":"ref_7","first-page":"19","article-title":"Skeletons Distance Evaluation for Automated Control of Exercises in Physical Therapy","volume":"XLVIII-2-W","author":"Bagchi","year":"2025","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"020029","DOI":"10.1063\/5.0111612","article-title":"Human Action Recognition from Depth Sensor via Skeletal Joint and Shape Trajectories with a Time-Series Graph Matching","volume":"2512","author":"Kumar","year":"2024","journal-title":"AIP Conf. Proc."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"286","DOI":"10.4218\/etrij.2020-0101","article-title":"Three-Dimensional Human Activity Recognition by Forming a Movement Polygon Using Posture Skeletal Data from Depth Sensor","volume":"44","author":"Vishwakarma","year":"2022","journal-title":"ETRI J."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"78","DOI":"10.4218\/etrij.2018-0577","article-title":"Human Activity Recognition with Analysis of Angles between Skeletal Joints Using a RGB-Depth Sensor","volume":"42","author":"Ince","year":"2020","journal-title":"ETRI J."},{"key":"ref_11","first-page":"3096858","article-title":"Enhanced Human Action Recognition Using Fusion of Skeletal Joint Dynamics and Structural Features","volume":"2020","author":"Muralikrishna","year":"2020","journal-title":"J. Robot."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.patrec.2021.02.013","article-title":"Action Recognition Using Kinematics Posture Feature on 3D Skeleton Joint Locations","volume":"145","author":"Ahad","year":"2021","journal-title":"Pattern Recognit. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4091","DOI":"10.1007\/s11263-024-02070-2","article-title":"Meet JEANIE: A Similarity Measure for 3D Skeleton Sequences via Temporal-Viewpoint Alignment","volume":"132","author":"Wang","year":"2024","journal-title":"Int. J. Comput. Vis."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Du, C., Graham, S., Depp, C., and Nguyen, T. (2021, January 15\u201319). Assessing Physical Rehabilitation Exercises Using Graph Convolutional Network with Self-Supervised Regularization. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, Orlando, FL, USA.","DOI":"10.1109\/EMBC46164.2021.9629569"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"36547","DOI":"10.1109\/ACCESS.2021.3063302","article-title":"A Body Part Embedding Model with Datasets for Measuring 2D Human Motion Similarity","volume":"9","author":"Park","year":"2021","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2215","DOI":"10.1007\/s10845-023-02152-x","article-title":"Fine-Grained Activity Classification in Assembly Based on Multi-Visual Modalities","volume":"35","author":"Chen","year":"2024","journal-title":"J. Intell. Manuf."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Anton, D., Goni, A., Illarramendi, A., Torres-Unda, J.J., and Seco, J. (2013, January 9\u201312). KiReS: A Kinect-Based Telerehabilitation System. Proceedings of the 2013 IEEE 15th International Conference on e-Health Networking, Applications and Services, Healthcom 2013, Lisbon, Portugal.","DOI":"10.1109\/HealthCom.2013.6720717"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.future.2021.04.003","article-title":"Kinect-Based Integrated Physiotherapy Mentor Application for Shoulder Damage","volume":"122","year":"2021","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.cmpb.2014.07.003","article-title":"Automated Evaluation of Physical Therapy Exercises Using Multi-Template Dynamic Time Warping on Wearable Sensor Signals","volume":"117","author":"Yurtman","year":"2014","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e0288279","DOI":"10.1371\/journal.pone.0288279","article-title":"Evaluation of Functional Tests Performance Using a Camera-Based and Machine Learning Approach","volume":"18","author":"Adolf","year":"2023","journal-title":"PLoS ONE"},{"key":"ref_21","first-page":"489","article-title":"A Kinect Based Intelligent E-Rehabilitation System in Physical Therapy","volume":"210","author":"Gal","year":"2015","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Esmaeeli, R., Valadan Zoej, M.J., Safdarinezhad, A., and Ghaderpour, E. (2024). Recognition and Scoring Physical Exercises via Temporal and Relative Analysis of Skeleton Nodes Extracted from the Kinect Sensor. Sensors, 24.","DOI":"10.3390\/s24206713"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"103687","DOI":"10.1016\/j.compbiomed.2020.103687","article-title":"A Review of Computational Approaches for Evaluation of Rehabilitation Exercises","volume":"119","author":"Liao","year":"2020","journal-title":"Comput. Biol. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"567502","DOI":"10.3389\/fdgth.2020.567502","article-title":"A Review of Virtual Coaching Systems in Healthcare: Closing the Loop With Real-Time Feedback","volume":"2","author":"Tsiouris","year":"2020","journal-title":"Front. Digit. Health"},{"key":"ref_25","unstructured":"Debnath, B., O\u2019Brien, M., Yamaguchi, M., and Behera, A. (2022). A Review of Computer Vision-Based Approaches for Physical Rehabilitation and Assessment, Springer."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Poggio, T., Torre, V., and Koch, C. (1987). Computational Vision and Regularization Theory. Readings in Computer Vision, Morgan Kaufmann.","DOI":"10.1016\/B978-0-08-051581-6.50061-1"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Jolion, J.-M., and Kropatsch, W.G. (1998). Optimization Techniques on Pixel Neighborhood Graphs for Image Processing. Graph-Based Representations in Pattern Recognition, Springer.","DOI":"10.1007\/978-3-7091-6487-7"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, J., Xue, J., Cao, R., Du, X., Mo, S., Ran, K., and Zhang, Z. (2024, January 17\u201318). FineRehab: A Multi-Modality and Multi-Task Dataset for Rehabilitation Analysis. Proceedings of the 2024 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, DC, USA.","DOI":"10.1109\/CVPRW63382.2024.00324"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Aguilar-Ortega, R., Berral-Soler, R., Jim\u00e9nez-Velasco, I., Romero-Ram\u00edrez, F.J., Garc\u00eda-Mar\u00edn, M., Zafra-Palma, J., Mu\u00f1oz-Salinas, R., Medina-Carnicer, R., and Mar\u00edn-Jim\u00e9nez, M.J. (2023). UCO Physical Rehabilitation: New Dataset and Study of Human Pose Estimation Methods on Physical Rehabilitation Exercises. Sensors, 23.","DOI":"10.3390\/s23218862"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Miron, A., Sadawi, N., Ismail, W., Hussain, H., and Grosan, C. (2021). IntelliRehabDS (IRDS)\u2014A Dataset of Physical Rehabilitation Movements. Data, 6.","DOI":"10.3390\/data6050046"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1038\/s41597-025-05099-1","article-title":"A Human Motion Data Capture Study The University of Liverpool Rehabilitation Exercise Dataset","volume":"12","author":"Reji","year":"2025","journal-title":"Sci. Data"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1436","DOI":"10.1109\/TNSRE.2019.2923060","article-title":"The KIMORE Dataset: KInematic Assessment of MOvement and Clinical Scores for Remote Monitoring of Physical REhabilitation","volume":"27","author":"Capecci","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"826","DOI":"10.1016\/j.patrec.2011.04.019","article-title":"The Dissimilarity Space: Bridging Structural and Statistical Pattern Recognition","volume":"33","author":"Duin","year":"2012","journal-title":"Pattern Recognit. Lett."},{"key":"ref_34","unstructured":"(2025, October 14). MLPClassifier\u2014Scikit-Learn 1.7.2 Documentation. Available online: https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.neural_network.MLPClassifier.html."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"441","DOI":"10.2307\/1422689","article-title":"The Proof and Measurement of Association between Two Things. By C. Spearman, 1904","volume":"100","author":"Spearman","year":"1987","journal-title":"Am. J. Psychol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1016\/S0167-8655(99)00085-9","article-title":"Relational Discriminant Analysis","volume":"20","author":"Duin","year":"1999","journal-title":"Pattern Recognit. Lett."},{"key":"ref_37","unstructured":"Mottl, V., Seredin, O., Dvoenko, S., Kulikowski, C., and Muchnik, I. (2002, January 11\u201315). Featureless Pattern Recognition in an Imaginary Hilbert Space. Proceedings of the 2002 International Conference on Pattern Recognition, 16th International Conference on Pattern Recognition, Quebec City, QC, Canada."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/21\/6696\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T18:28:03Z","timestamp":1762194483000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/21\/6696"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,2]]},"references-count":37,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["s25216696"],"URL":"https:\/\/doi.org\/10.3390\/s25216696","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,2]]}}}