{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,7]],"date-time":"2026-09-07T16:28:18Z","timestamp":1788798498412,"version":"build-2803163510"},"reference-count":33,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T00:00:00Z","timestamp":1689033600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["BK20210930"],"award-info":[{"award-number":["BK20210930"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010023","name":"Natural Science Research of Jiangsu higher education institutions of China","doi-asserted-by":"publisher","award":["21KJB510039"],"award-info":[{"award-number":["21KJB510039"]}],"id":[{"id":"10.13039\/501100010023","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurosci."],"abstract":"<jats:p>After regular rehabilitation training, paralysis sequelae can be significantly reduced in patients with limb movement disorders caused by stroke. Rehabilitation assessment is the basis for the formulation of rehabilitation training programs and the objective standard for evaluating the effectiveness of training. However, the quantitative rehabilitation assessment is still in the experimental stage and has not been put into clinical practice. In this work, we propose improved spatial-temporal graph convolutional networks based on precise posture measurement for upper limb rehabilitation assessment. Two Azure Kinect are used to enlarge the angle range of the visual field. The rigid body model of the upper limb with multiple degrees of freedom is established. And the inverse kinematics is optimized based on the hybrid particle swarm optimization algorithm. The self-attention mechanism map is calculated to analyze the role of each upper limb joint in rehabilitation assessment, to improve the spatial-temporal graph convolution neural network model. Long short-term memory is built to explore the sequence dependence in spatial-temporal feature vectors. An exercise protocol for detecting the distal reachable workspace and proximal self-care ability of the upper limb is designed, and a virtual environment is built. The experimental results indicate that the proposed posture measurement method can reduce position jumps caused by occlusion, improve measurement accuracy and stability, and increase Signal Noise Ratio. By comparing with other models, our rehabilitation assessment model achieved the lowest mean absolute deviation, root mean square error, and mean absolute percentage error. The proposed method can effectively quantitatively evaluate the upper limb motor function of stroke patients.<\/jats:p>","DOI":"10.3389\/fnins.2023.1219556","type":"journal-article","created":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T16:57:07Z","timestamp":1689094627000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Improved spatial\u2013temporal graph convolutional networks for upper limb rehabilitation assessment based on precise posture measurement"],"prefix":"10.3389","volume":"17","author":[{"given":"Jing","family":"Bai","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhixian","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuanming","family":"Lu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiulan","family":"Wen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,7,11]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"956381","DOI":"10.3389\/fresc.2022.956381","article-title":"Enhancing motion tracking accuracy of a low-cost 3D video sensor using a biomechanical model, sensor fusion, and deep learning","volume":"3","author":"Agami","year":"2022","journal-title":"Front. Rehabil. Sci."},{"key":"ref2","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1109\/TAI.2021.3076974","article-title":"Graph convolutional neural network for human action recognition: a comprehensive survey","volume":"2","author":"Ahmad","year":"2021","journal-title":"IEEE Trans. Artificial Intell."},{"key":"ref3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TCDS.2021.3098350","article-title":"Machine learning in robot assisted upper limb rehabilitation: a focused review","author":"Ai","year":"2021","journal-title":"IEEE Trans. Cog. Dev. Syst."},{"key":"ref4","doi-asserted-by":"publisher","first-page":"9667","DOI":"10.1109\/ACCESS.2019.2891606","article-title":"Development of a novel home based multi-scene upper limb rehabilitation training and evaluation system for post-stroke patients","volume":"7","author":"Bai","year":"2019","journal-title":"IEEE Access."},{"key":"ref5","doi-asserted-by":"publisher","first-page":"60122","DOI":"10.1155\/2021\/4360122","article-title":"A review on the use of Microsoft Kinect for gait abnormality and postural disorder assessment","volume":"2021","author":"Bawa","year":"2021","journal-title":"Hindawi Limited"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1186\/s12984-019-0609-6","article-title":"Detecting compensatory movements of stroke survivors using pressure distribution data and machine learning algorithms","volume":"16","author":"Cai","year":"2019","journal-title":"J. Neuroeng. Rehabil."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"410","DOI":"10.1109\/TNSRE.2022.3150392","article-title":"Graph convolutional networks for assessment of physical rehabilitation exercises","volume":"30","author":"Deb","year":"2022","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"758","DOI":"10.1109\/JBHI.2018.2837380","article-title":"A novel multi-standard compliant hand function assessment method using an infrared imaging device","volume":"23","author":"Fang","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref9","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1177\/17474930211065917","article-title":"World stroke organization (WSO): global stroke fact sheet 2022","volume":"17","author":"Feigin","year":"2022","journal-title":"Int. J. Stroke"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"922","DOI":"10.1609\/aaai.v33i01.3301922","article-title":"Attention based spatial-temporal graph convolutional networks for traffic flow forecasting","volume":"33","author":"Guo","year":"2019","journal-title":"Proc. AAAI Con. Artificial Intell."},{"key":"ref11","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1007\/s40846-019-00491-w","article-title":"Support vector machine-based classifier for the assessment of finger movement of stroke patients undergoing rehabilitation","volume":"40","author":"Hamaguchi","year":"2020","journal-title":"J. Med. Biol. Eng."},{"key":"ref12","doi-asserted-by":"publisher","first-page":"545","DOI":"10.1002\/mus.24894","article-title":"Reachable workspace and performance of upper limb (PUL) in Duchenne muscular dystrophy","volume":"53","author":"Han","year":"2016","journal-title":"Muscle Nerve"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"1014","DOI":"10.1109\/JSYST.2014.2327792","article-title":"Movement analysis of rehabilitation exercises: distance metrics for measuring patient progress","volume":"10","author":"Houmanfar","year":"2016","journal-title":"IEEE Syst. J."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1504\/IJSNET.2019.10020697","article-title":"Computational model for the recognition of lower limb movement using wearable gyroscope sensor","volume":"30","author":"Hussain","year":"2019","journal-title":"Int. J. Sensor Networks."},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1609.02907","article-title":"Semi-supervised classification with graph convolutional networks","author":"Kipf","year":"2017","journal-title":"ICLR"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"s20","DOI":"10.1016\/j.clinbiomech.2005.09.009","article-title":"Bilateral and unilateral shoulder girdle kinematics during humeral elevation","volume":"21","author":"Klop\u010dar","year":"2006","journal-title":"Clin. Biomech."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1109\/TNSRE.2017.2755667","article-title":"Automated evaluation of upper-limb motor function impairment using Fugl-Meyer assessment","volume":"26","author":"Lee","year":"2018","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref18","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1080\/01969722.2020.1827798","article-title":"An automatic rehabilitation assessment system for hand function based on leap motion and ensemble learning","volume":"52","author":"Li","year":"2021","journal-title":"Cybern. Syst."},{"key":"ref19","doi-asserted-by":"publisher","first-page":"2032","DOI":"10.1109\/TNSRE.2022.3192479","article-title":"Quantitative assessment of hand motor function for post-stroke rehabilitation based on HAGCN and multimodality fusion","volume":"30","author":"Li","year":"2022","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1109\/TNSRE.2020.2966249","article-title":"A deep learning framework for assessing physical rehabilitation exercises","volume":"28","author":"Liao","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref21","doi-asserted-by":"publisher","first-page":"2437","DOI":"10.1109\/TVCG.2015.2510000","article-title":"Kinect posture reconstruction based on a local mixture of Gaussian process models","volume":"22","author":"Liu","year":"2016","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref22","doi-asserted-by":"publisher","first-page":"769509","DOI":"10.3389\/fpsyg.2021.769509","article-title":"Multimodal art pose recognition and interaction with human intelligence enhancement","volume":"12","author":"Ma","year":"2021","journal-title":"Front. Psychol."},{"key":"ref23","doi-asserted-by":"publisher","first-page":"1784","DOI":"10.1109\/JBHI.2018.2872834","article-title":"Kinematic and kinetic validation of an improved depth camera motion assessment system using rigid bodies","volume":"23","author":"Matthew","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref24","doi-asserted-by":"publisher","first-page":"3285","DOI":"10.1109\/JBHI.2020.2989722","article-title":"Reachable workspace and proximal function measures for quantifying upper limb motion","volume":"24","author":"Matthew","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"113518","DOI":"10.1016\/j.expneurol.2020.113518","article-title":"Emerging neuroprotective strategies for the treatment of ischemic stroke: an overview of clinical and preclinical studies","volume":"335","author":"Paul","year":"2021","journal-title":"Exp. Neurol."},{"key":"ref26","first-page":"273","article-title":"Research on human-computer interaction following motion based on Kinect somatosensory sensor","author":"Qiao","year":"2022"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1109\/TNSRE.2022.3219085","article-title":"AI-driven stroke rehabilitation systems and assessment: a systematic review","volume":"31","author":"Rahman","year":"2023","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref28","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1109\/TIP.2019.2925285","article-title":"A comparative review of recent Kinect-based action recognition algorithms","volume":"29","author":"Wang","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1265","DOI":"10.1109\/TCSVT.2015.2462011","article-title":"Handling occlusion and large displacement through improved RGB-D scene flow estimation","volume":"26","author":"Wang","year":"2016","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref30","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/j.medengphy.2019.10.003","article-title":"Assessment of physical rehabilitation movements through dimensionality reduction and statistical modeling","volume":"74","author":"Williams","year":"2019","journal-title":"Med. Eng. Phys."},{"key":"ref31","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v32i1.12328","article-title":"Spatial temporal graph convolutional networks for skeleton-based action recognition","author":"Yan","year":"2018"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"704","DOI":"10.3390\/s16050704","article-title":"Kinect-based virtual game for the elderly that detects incorrect body postures in real time","volume":"16","author":"Zelai","year":"2016","journal-title":"Sensors"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"15356","DOI":"10.1016\/j.eswa.2011.06.029","article-title":"Randomization in particle swarm optimization for global search ability","volume":"38","author":"Zhou","year":"2011","journal-title":"Expert Syst. Appl."}],"container-title":["Frontiers in Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2023.1219556\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T16:57:13Z","timestamp":1689094633000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2023.1219556\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,11]]},"references-count":33,"alternative-id":["10.3389\/fnins.2023.1219556"],"URL":"https:\/\/doi.org\/10.3389\/fnins.2023.1219556","relation":{},"ISSN":["1662-453X"],"issn-type":[{"value":"1662-453X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,11]]},"article-number":"1219556"}}