{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T10:31:05Z","timestamp":1783161065883,"version":"3.54.6"},"reference-count":45,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T00:00:00Z","timestamp":1627430400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Using optical motion capture and wearable sensors is a common way to analyze impaired movement in individuals with neurological and musculoskeletal disorders. However, using optical motion sensors and wearable sensors is expensive and often requires highly trained professionals to identify specific impairments. In this work, we proposed a graph convolutional neural network that mimics the intuition of physical therapists to identify patient-specific impairments based on video of a patient. In addition, two modeling approaches are compared: a graph convolutional network applied solely on skeleton input data and a graph convolutional network accompanied with a 1-dimensional convolutional neural network (1D-CNN). Experiments on the dataset showed that the proposed method not only improves the correlation of the predicted gait measure with the ground truth value (speed = 0.791, gait deviation index (GDI) = 0.792) but also enables faster training with fewer parameters. In conclusion, the proposed method shows that the possibility of using video-based data to treat neurological and musculoskeletal disorders with acceptable accuracy instead of depending on the expensive and labor-intensive optical motion capture systems.<\/jats:p>","DOI":"10.3390\/fi13080194","type":"journal-article","created":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T05:23:48Z","timestamp":1627449828000},"page":"194","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Movement Analysis for Neurological and Musculoskeletal Disorders Using Graph Convolutional Neural Network"],"prefix":"10.3390","volume":"13","author":[{"given":"Ibsa K.","family":"Jalata","sequence":"first","affiliation":[{"name":"Computer Vision and Image Understanding Lab, Department of Computer Science and Computer Engineering, University of Arkansas, Fayetteville, AR 72701, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thanh-Dat","family":"Truong","sequence":"additional","affiliation":[{"name":"Computer Vision and Image Understanding Lab, Department of Computer Science and Computer Engineering, University of Arkansas, Fayetteville, AR 72701, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jessica L.","family":"Allen","sequence":"additional","affiliation":[{"name":"Department of Chemical and Biomedical Engineering, West Virginia University, Morgantown, WV 26506, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9845-8199","authenticated-orcid":false,"given":"Han-Seok","family":"Seo","sequence":"additional","affiliation":[{"name":"Department of Food Science, University of Arkansas, Fayetteville, AR 72701, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Khoa","family":"Luu","sequence":"additional","affiliation":[{"name":"Computer Vision and Image Understanding Lab, Department of Computer Science and Computer Engineering, University of Arkansas, Fayetteville, AR 72701, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1055\/s-0030-1268506","article-title":"Walking Ability and All-Cause Mortality in Older Women","volume":"32","author":"Mutikainen","year":"2011","journal-title":"Int. J. Sports Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1111\/jgs.12170","article-title":"Mobility Activity and Its Value as a Prognostic Indicator of Survival in Hospitalized Older Adults","volume":"61","author":"Ostir","year":"2013","journal-title":"J. Am. Geriatr."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1016\/j.arcmed.2012.11.009","article-title":"Early-onset Alzheimer\u2019s disease: Nonamnestic subtypes and type 2 AD","volume":"43","author":"Mendez","year":"2012","journal-title":"Arch. Med. Res."},{"key":"ref_4","unstructured":"Lee, L., and Grimson, W.E.L. (2002, January 21). Gait analysis for recognition and classification. Proceedings of the Fifth IEEE International Conference on Automatic Face Gesture Recognition, Washington, DC, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.gaitpost.2003.09.011","article-title":"Measurement and management of errors in quantitative gait data","volume":"20","author":"Schwartz","year":"2002","journal-title":"Gait Posture"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"461","DOI":"10.2522\/ptj.20140253","article-title":"Role of Body-Worn Movement Monitor Technology for Balance and Gait Rehabilitation","volume":"95","author":"Horak","year":"2015","journal-title":"Phys. Ther."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Al-Amri, M., Nicholas, K., Button, K., Sparkes, V., Sheeran, L., and Davies, J. (2018). Inertial Measurement Units for Clinical Movement Analysis. Natl. Libr. Med., 18.","DOI":"10.3390\/s18030719"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Chambers, C., Prosser, L., Johnson, M.J., and Kording, K.P. (2019). Computer vision to automatically assess infant neuromotor risk. IEEE Trans. Neural. Syst. Rehabil. Eng., 28.","DOI":"10.1101\/756262"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Stenum, J., Cristina, R., and Ryan, T. (2020). Roemmich. Two-dimensional video-based analysis of human gait using pose estimation. PLoS Comput. Biol., 17.","DOI":"10.1101\/2020.07.24.218776"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Huang, Z., Liu, Y., Fang, Y., and Horn, B.K. (2018, January 21\u201324). Video-based fall detection for seniors with human pose estimation. Proceedings of the 4th International Conference on Universal Village (UV), Boston, MA, USA.","DOI":"10.1109\/UV.2018.8642130"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4054","DOI":"10.1038\/s41467-020-17807-z","article-title":"Deep neural networks enable quantitative movement analysis using single-camera videos","volume":"11","author":"Kidzinski","year":"2020","journal-title":"Nat. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Cao, Z., Simon, T., Wei, S.E., and Sheikh, Y. (2017, January 21\u201326). Realtime multi-person 2D pose estimation using part affinity fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.143"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Li, S., Li, W., Cook, C., Zhu, C., and Gao, Y. (2018, January 18\u201323). Independently recurrent neural network (indrnn): Building a longer and deeper RNN. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00572"},{"key":"ref_14","unstructured":"Wang, J., Liu, Z., Wu, Y., and Yuan, J. (2012, January 16\u201321). Mining actionlet ensemble for action recognition with depth cameras. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Evangelidis, G., Singh, G., and Horaud, R. (2014, January 24\u201328). Skeletal quads: Human action recognition using joint quadruples. Proceedings of the 2014 22nd International Conference on Pattern Recognition, Stockholm, Sweden.","DOI":"10.1109\/ICPR.2014.772"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Luo, J., Wang, W., and Qi, H. (2013, January 1\u20138). Group sparsity and geometry constrained dictionary learning for action recognition from depth maps. Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, NSW, Australia.","DOI":"10.1109\/ICCV.2013.227"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Vemulapalli, R., Arrate, F., and Chellappa, R. (2014, January 23\u201328). Human action recognition by representing 3d skeletons as points in a lie group. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.82"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Rahmani, H., and Mian, A. (2015, January 7\u201312). Learning a non-linear knowledge transfer model for crossview action recognition. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298860"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Hu, J.F., Zheng, W.S., Lai, J., and Zhang, J. (2015, January 7\u201312). Jointly learning heterogeneous features for RGB-D activity recognition. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299172"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ohn-Bar, E., and Trivedi, M. (2013, January 23\u201328). Joint angles similarities and HOG2 for action recognition. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA.","DOI":"10.1109\/CVPRW.2013.76"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Rahmani, H., Mahmood, A., Huynh, D.Q., and Mian, A. (2014, January 24\u201326). Real time action recognition using histograms of depth gradients and random decision forests. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Steamboat Springs, CO, USA.","DOI":"10.1109\/WACV.2014.6836044"},{"key":"ref_22","unstructured":"Du, Y., Wang, W., and Wang, L. (2015, January 7\u201312). Hierarchical recurrent neural network for skeleton based action recognition. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Song, S., Lan, C., Xing, J., Zeng, W., and Liu, J. (2017, January 4\u20139). An end-to-end spatio-temporal attention model for human action recognition from skeleton data. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11212"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zamir, A.R., Wu, T.L., Sun, L., Shen, W.B., Shi, B.E., Malik, J., and Savarese, S. (2017, January 21\u201326). Feedback networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.196"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2017","journal-title":"Nature"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhao, R., Ali, H., and Van der Smagt, P. (2017, January 24\u201328). Two-stream RNN\/CNN for action recognition in 3D videos. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206288"},{"key":"ref_27","unstructured":"Gori, M., Monfardini, G., and Scarselli, F. (August, January 31). A new model for learning in graph domains. Proceedings of the 2005 IEEE International Joint Conference on Neural Networks, Montreal, QC, Canada."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1109\/TNN.2008.2005605","article-title":"The graph neural network model","volume":"20","author":"Scarselli","year":"2009","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Gallicchio, C., and Micheli, A. (2010, January 18\u201323). Graph echo state networks. Proceedings of the 2010 International Joint Conference on Neural Networks (IJCNN), Barcelona, Spain.","DOI":"10.1109\/IJCNN.2010.5596796"},{"key":"ref_30","unstructured":"Li, Y., Tarlow, D., Brockschmidt, M., and Zemel, R. (2015, January 7\u20139). Gated graph sequence neural networks. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_31","unstructured":"Dai, H., Kozareva, Z., Dai, B., Smola, A., and Song, L. (2018, January 10\u201315). Learning steadystates of iterative algorithms over graphs. Proceedings of the International Conference on Machine Learning, Stockholmsm\u00e4ssan, Stockholm, Sweden."},{"key":"ref_32","unstructured":"Henaff, M., Bruna, J., and LeCun, Y. (2015, January 6\u201311). Deep convolutional networks on graph-structured data. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_33","unstructured":"Defferrard, M., Bresson, X., and Vandergheynst, P. (2016, January 5). Convolutional neural networks on graphs with fast localized spectral filtering. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_34","unstructured":"Kipf, T.N., and Welling, M. (2017, January 24\u201326). Semi-supervised classification with graph convolutional networks. Proceedings of the 5th International Conference on Learning Representations, Toulon, France."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1109\/TSP.2018.2879624","article-title":"Cayleynets: Graph convolutional neural networks with complex rational spectral filters","volume":"67","author":"Levie","year":"2017","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_36","unstructured":"Atwood, J., and Towsley, D. (2016, January 5). Diffusion-convolutional neural networks. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_37","unstructured":"Niepert, M., Ahmed, M., and Kutzkov, K. (2016, January 19\u201324). Learning convolutional neural networks for graphs. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_38","unstructured":"Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., and Dahl, G.E. (2017, January 6\u201311). Neural message passing for quantum chemistry. Proceedings of the 34th International Conference on Machine Learning, Sydney, NSW, Australia."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yu, B., Yin, H., and Zhu, Z. (2018). Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting. arXiv.","DOI":"10.24963\/ijcai.2018\/505"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"50","DOI":"10.3389\/fspor.2020.00050","article-title":"Evaluation of 3D Markerless Motion Capture Accuracy Using OpenPose with Multiple Video Cameras","volume":"2","author":"Nakano","year":"2020","journal-title":"Front. Sport. Act. Living"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1950003","DOI":"10.1142\/S0129065719500035","article-title":"How Far can Neural Correlations Reduce Uncertainty? Comparison of Information Transmission Rates for Markov and Bernoulli Processes","volume":"29","author":"Pregowska","year":"2019","journal-title":"Int. J. Neural. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"151","DOI":"10.3389\/fnsys.2015.00151","article-title":"Philosophy of the spike: Rate-based versus spike-based theories of the brain","volume":"9","author":"Brette","year":"2015","journal-title":"Front. Syst. Neurosci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"3507","DOI":"10.3390\/e15093507","article-title":"The measurement of information transmitted by a neural population: Promises and challenges","volume":"15","author":"Crumiller","year":"2013","journal-title":"Entropy"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"756","DOI":"10.1016\/j.neucom.2016.08.034","article-title":"Temporal code versus rate code for binary Information Sources","volume":"216","author":"Pregowska","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"9227","DOI":"10.1523\/JNEUROSCI.5995-08.2009","article-title":"Quality time: Representation of a multidimensional sensory domain through temporal coding","volume":"29","author":"Lorenzo","year":"2009","journal-title":"J. Neurosci."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/13\/8\/194\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:35:58Z","timestamp":1760164558000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/13\/8\/194"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,28]]},"references-count":45,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["fi13080194"],"URL":"https:\/\/doi.org\/10.3390\/fi13080194","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,28]]}}}