{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T15:47:53Z","timestamp":1753890473771,"version":"3.41.2"},"reference-count":47,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,6,20]],"date-time":"2023-06-20T00:00:00Z","timestamp":1687219200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Bioeng. Biotechnol."],"abstract":"<jats:p><jats:bold>Introduction:<\/jats:bold> Balance impairment is an important indicator to a variety of diseases. Early detection of balance impairment enables doctors to provide timely treatments to patients, thus reduce their fall risk and prevent related disease progression. Currently, balance abilities are usually assessed by balance scales, which depend heavily on the subjective judgement of assessors.<\/jats:p><jats:p><jats:bold>Methods:<\/jats:bold> To address this issue, we specifically designed a method combining 3D skeleton data and deep convolutional neural network (DCNN) for automated balance abilities assessment during walking. A 3D skeleton dataset with three standardized balance ability levels were collected and used to establish the proposed method. To obtain better performance, different skeleton-node selections and different DCNN hyperparameters setting were compared. Leave-one-subject-out-cross-validation was used in training and validation of the networks.<\/jats:p><jats:p><jats:bold>Results and Discussion:<\/jats:bold> Results showed that the proposed deep learning method was able to achieve 93.33% accuracy, 94.44% precision and 94.46% F1 score, which outperformed four other commonly used machine learning methods and CNN-based methods. We also found that data from body trunk and lower limbs are the most important while data from upper limbs may reduce model accuracy. To further validate the performance of the proposed method, we migrated and applied a state-of-the-art posture classification method to the walking balance ability assessment task. Results showed that the proposed DCNN model improved the accuracy of walking balance ability assessment. Layer-wise Relevance Propagation (LRP) was used to interpret the output of the proposed DCNN model. Our results suggest that DCNN classifier is a fast and accurate method for balance assessment during walking.<\/jats:p>","DOI":"10.3389\/fbioe.2023.1191868","type":"journal-article","created":{"date-parts":[[2023,6,20]],"date-time":"2023-06-20T08:15:51Z","timestamp":1687248951000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Combining 3D skeleton data and deep convolutional neural network for balance assessment during walking"],"prefix":"10.3389","volume":"11","author":[{"given":"Xiangyuan","family":"Ma","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Buhui","family":"Zeng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanghui","family":"Xing","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,6,20]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1117\/12.2052588","article-title":"Gait recognition based on kinect sensor","author":"Ahmed","year":"2014"},{"article-title":"iNNvestigate neural networks","year":"2018","author":"Alber","key":"B2"},{"key":"B3","doi-asserted-by":"publisher","first-page":"9575","DOI":"10.1109\/jsen.2019.2928777","article-title":"Deep learning for monitoring of human gait: A review","volume":"19","author":"Alharthi","year":"2019","journal-title":"IEEE Sensors J."},{"key":"B4","doi-asserted-by":"publisher","first-page":"e0181142","DOI":"10.1371\/journal.pone.0181142","article-title":"What is relevant in a text document? An interpretable machine learning approach","volume":"12","author":"Arras","year":"2017","journal-title":"PloS one"},{"key":"B5","doi-asserted-by":"publisher","first-page":"e0130140","DOI":"10.1371\/journal.pone.0130140","article-title":"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation","volume":"10","author":"Bach","year":"2015","journal-title":"PloS one"},{"article-title":"An empirical evaluation of generic convolutional and recurrent networks for sequence modeling","year":"2018","author":"Bai","key":"B6"},{"volume-title":"Delving deeper into convolutional networks for learning video representations","year":"2015","author":"Ballas","key":"B7"},{"key":"B8","doi-asserted-by":"publisher","first-page":"162708","DOI":"10.1109\/access.2019.2952065","article-title":"Artificial neural network based gait recognition using kinect sensor","volume":"7","author":"Bari","year":"2019","journal-title":"IEEE Access"},{"key":"B9","doi-asserted-by":"publisher","first-page":"4561","DOI":"10.1111\/jocn.15484","article-title":"Exploring the use of ageing simulation to enable nurses to gain insight into what it is like to be an older person","volume":"29","author":"Bowden","year":"2020","journal-title":"J. Clin. Nurs."},{"key":"B10","first-page":"16","article-title":"Skeleton image representation for 3d action recognition based on tree structure and reference joints","author":"Caetano","year":"2019"},{"key":"B11","first-page":"183","article-title":"Skeleton-based action recognition with shift graph convolutional network","author":"Cheng","year":"2020"},{"key":"B12","doi-asserted-by":"crossref","first-page":"103","DOI":"10.3115\/v1\/W14-4012","article-title":"On the properties of neural machine translation: Encoder-decoder approaches","volume-title":"Proceedings of SSST-8, eighth workshop on syntax, semantics and structure in statistical translation","author":"Cho","year":"2014"},{"key":"B13","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1016\/j.gaitpost.2017.09.010","article-title":"Kinect-based assessment of lower limb kinematics and dynamic postural control during the star excursion balance test","volume":"58","author":"Eltoukhy","year":"2017","journal-title":"Gait posture"},{"key":"B14","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1109\/ASRU.2013.6707742","article-title":"Hybrid speech recognition with deep bidirectional LSTM","volume-title":"IEEE workshop on automatic speech recognition and understanding","author":"Graves","year":"2013"},{"key":"B15","doi-asserted-by":"publisher","first-page":"2391","DOI":"10.1038\/s41598-019-38748-8","article-title":"Explaining the unique nature of individual gait patterns with deep learning","volume":"9","author":"Horst","year":"2019","journal-title":"Sci. Rep."},{"key":"B16","doi-asserted-by":"publisher","first-page":"CD004963","DOI":"10.1002\/14651858.CD004963.pub3","article-title":"Exercise for improving balance in older people","author":"Howe","year":"2011","journal-title":"Cochrane database Syst. Rev."},{"key":"B17","doi-asserted-by":"publisher","first-page":"139881","DOI":"10.1109\/access.2020.3013029","article-title":"Pathological gait classification using kinect v2 and gated recurrent neural networks","volume":"8","author":"Jun","year":"2020","journal-title":"IEEE Access"},{"key":"B18","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1002\/ima.22316","article-title":"Comparative analysis of Alzheimer's disease classification by CDR level using CNN, feature selection, and machine\u2010learning techniques","volume":"29","author":"Khagi","year":"2019","journal-title":"Int. J. Imaging Syst. Technol."},{"key":"B19","doi-asserted-by":"publisher","first-page":"133","DOI":"10.5194\/isprsarchives-5-w12-133-2011","article-title":"Accuracy analysis of kinect depth data","volume":"38","author":"Khoshelham","year":"2012","journal-title":"Int. Archives Photogrammetry, Remote Sens. Spatial Inf. Sci."},{"article-title":"Learning how to explain neural networks: Patternnet and pattern attribution","year":"2017","author":"Kindermans","key":"B20"},{"key":"B21","doi-asserted-by":"publisher","first-page":"4028","DOI":"10.3390\/app10114028","article-title":"Development of classification algorithms for the detection of postures using non-marker-based motion capture systems","volume":"10","author":"Klishkovskaia","year":"2020","journal-title":"Appl. Sci."},{"key":"B22","first-page":"1629","article-title":"Understanding and comparing deep neural networks for age and gender classification","author":"Lapuschkin","year":"2017"},{"key":"B23","doi-asserted-by":"publisher","first-page":"227","DOI":"10.3928\/19404921-20170831-04","article-title":"Effect of an age simulation suit on younger adults' gait performance compared to older adults' normal gait","volume":"10","author":"Lauenroth","year":"2017","journal-title":"Res. gerontological Nurs."},{"key":"B24","first-page":"542","article-title":"Abnormal gait recognition using 3D joint information of multiple Kinects system and RNN-LSTM","author":"Lee","year":"2019"},{"key":"B26","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1049\/iet-cvi.2016.0385","article-title":"Human motion segmentation using collaborative representations of 3D skeletal sequences","volume":"12","author":"Li","year":"2018","journal-title":"IET Comput. Vis."},{"key":"B27","doi-asserted-by":"publisher","first-page":"107293","DOI":"10.1016\/j.patcog.2020.107293","article-title":"Learning shape and motion representations for view invariant skeleton-based action recognition","volume":"103","author":"Li","year":"2020","journal-title":"Pattern Recognit."},{"article-title":"A survey on text classification: from shallow to deep learning","year":"2021","author":"Li","key":"B25"},{"key":"B28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ISMSIT.2019.8932908","article-title":"Flower classification with deep CNN and machine learning algorithms","volume-title":"2019 3rd international symposium on multidisciplinary Studies and innovative technologies (ISMSIT)","author":"Mete","year":"2019"},{"key":"B29","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/j.patcog.2016.11.008","article-title":"Explaining nonlinear classification decisions with deep Taylor decomposition","volume":"65","author":"Montavon","year":"2017","journal-title":"Pattern Recognit."},{"key":"B30","doi-asserted-by":"publisher","first-page":"745","DOI":"10.1007\/s40596-020-01233-w","article-title":"An immersive simulation to build empathy for geriatric patients with co-occurring physical and mental illness","volume":"44","author":"Saiva","year":"2020","journal-title":"Acad. Psychiatry"},{"key":"B31","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1016\/j.jbiomech.2013.11.031","article-title":"Accuracy and repeatability of joint angles measured using a single camera markerless motion capture system","volume":"47","author":"Schmitz","year":"2014","journal-title":"J. biomechanics"},{"key":"B32","doi-asserted-by":"publisher","first-page":"706","DOI":"10.1038\/s41586-019-1923-7","article-title":"Improved protein structure prediction using potentials from deep learning","volume":"577","author":"Senior","year":"2020","journal-title":"Nature"},{"key":"B33","doi-asserted-by":"publisher","first-page":"9532","DOI":"10.1109\/tip.2020.3028207","article-title":"Skeleton-based action recognition with multi-stream adaptive graph convolutional networks","volume":"29","author":"Shi","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"B34","first-page":"395","article-title":"Postural sway biofeedback: Its effect on reestablishing stance stability in hemiplegic patients","volume":"69","author":"Shumway-Cook","year":"1988","journal-title":"Archives Phys. Med. Rehabilitation"},{"key":"B35","unstructured":"SmoothGrad: Removing noise by adding noise\n            SmilkovD.\n            ThoratN.\n            KimB.\n            Vi\u00e9gasF.\n            WattenbergM.\n          2017"},{"article-title":"Striving for simplicity: The all convolutional net","year":"2015","author":"Springenberg","key":"B36"},{"key":"B37","doi-asserted-by":"publisher","first-page":"e10124","DOI":"10.7717\/peerj.10124","article-title":"Convolutional neural network in upper limb functional motion analysis after stroke","volume":"8","author":"Szcz\u0119sna","year":"2020","journal-title":"PeerJ"},{"key":"B38","first-page":"1","article-title":"Going deeper with convolutions","author":"Szegedy","year":"2015"},{"key":"B39","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1080\/0361073x.2020.1766299","article-title":"How an age simulation suit affects motor and cognitive performance and self-perception in younger adults","volume":"46","author":"Vieweg","year":"2020","journal-title":"Exp. Aging Res."},{"key":"B40","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1136\/bmjstel-2021-000867","article-title":"Age suit simulation replicates in healthy young adults the functional challenges to balance experienced by older adults: An observational study","volume":"7","author":"Watkins","year":"2021","journal-title":"BMJ Simul. Technol. Enhanc. Learn."},{"key":"B41","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1016\/j.gaitpost.2005.04.004","article-title":"Automated feature assessment in instrumented gait analysis","volume":"23","author":"Wolf","year":"2006","journal-title":"Gait Posture"},{"key":"B42","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1049\/cit2.12014","article-title":"Deep learning\u2010based action recognition with 3D skeleton: A survey","volume":"6","author":"Xing","year":"2021","journal-title":"Caai Trans. Intell. Technol."},{"key":"B43","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.patrec.2018.05.018","article-title":"A review of convolutional-neural-network-based action recognition","volume":"118","author":"Yao","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"B44","doi-asserted-by":"publisher","first-page":"2244","DOI":"10.1177\/14759217211053546","article-title":"Crack detection of concrete structures using deep convolutional neural networks optimized by enhanced chicken swarm algorithm","volume":"21","author":"Yu","year":"","journal-title":"Struct. Health Monit."},{"key":"B45","doi-asserted-by":"publisher","first-page":"105246","DOI":"10.1016\/j.jobe.2022.105246","article-title":"Vision-based concrete crack detection using a hybrid framework considering noise effect","volume":"61","author":"Yu","year":"","journal-title":"J. Build. Eng."},{"key":"B46","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1177\/1475921718804132","article-title":"A novel deep learning-based method for damage identification of smart building structures","volume":"18","author":"Yu","year":"2019","journal-title":"Struct. Health Monit."},{"key":"B47","first-page":"4694","article-title":"Beyond short snippets: Deep networks for video classification","author":"Yue-Hei Ng","year":"2015"}],"container-title":["Frontiers in Bioengineering and Biotechnology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fbioe.2023.1191868\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,20]],"date-time":"2023-06-20T08:15:57Z","timestamp":1687248957000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fbioe.2023.1191868\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,20]]},"references-count":47,"alternative-id":["10.3389\/fbioe.2023.1191868"],"URL":"https:\/\/doi.org\/10.3389\/fbioe.2023.1191868","relation":{},"ISSN":["2296-4185"],"issn-type":[{"type":"electronic","value":"2296-4185"}],"subject":[],"published":{"date-parts":[[2023,6,20]]},"article-number":"1191868"}}