{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T20:59:28Z","timestamp":1776113968848,"version":"3.50.1"},"reference-count":26,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2022,7,14]],"date-time":"2022-07-14T00:00:00Z","timestamp":1657756800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science, KAKENHI","doi-asserted-by":"publisher","award":["21K09098"],"award-info":[{"award-number":["21K09098"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"name":"G-7 Scholarship Foundation","award":["21K09098"],"award-info":[{"award-number":["21K09098"]}]},{"name":"Taiju Life Social Welfare Foundation","award":["21K09098"],"award-info":[{"award-number":["21K09098"]}]},{"name":"Osaka Gas Group Welfare Foundation","award":["21K09098"],"award-info":[{"award-number":["21K09098"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>To quantitatively assess pathological gait, we developed a novel smartphone application for full-body human motion tracking in real time from markerless video-based images using a smartphone monocular camera and deep learning. As training data for deep learning, the original three-dimensional (3D) dataset comprising more than 1 million captured images from the 3D motion of 90 humanoid characters and the two-dimensional dataset of COCO 2017 were prepared. The 3D heatmap offset data consisting of 28 \u00d7 28 \u00d7 28 blocks with three red\u2013green\u2013blue colors at the 24 key points of the entire body motion were learned using the convolutional neural network, modified ResNet34. At each key point, the hottest spot deviating from the center of the cell was learned using the tanh function. Our new iOS application could detect the relative tri-axial coordinates of the 24 whole-body key points centered on the navel in real time without any markers for motion capture. By using the relative coordinates, the 3D angles of the neck, lumbar, bilateral hip, knee, and ankle joints were estimated. Any human motion could be quantitatively and easily assessed using a new smartphone application named Three-Dimensional Pose Tracker for Gait Test (TDPT-GT) without any body markers or multipoint cameras.<\/jats:p>","DOI":"10.3390\/s22145282","type":"journal-article","created":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T01:57:11Z","timestamp":1657850231000},"page":"5282","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Development of Smartphone Application for Markerless Three-Dimensional Motion Capture Based on Deep Learning Model"],"prefix":"10.3390","volume":"22","author":[{"given":"Yukihiko","family":"Aoyagi","sequence":"first","affiliation":[{"name":"Digital Standard Co., Ltd., Osaka 536-0013, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7158-5569","authenticated-orcid":false,"given":"Shigeki","family":"Yamada","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Shiga University of Medical Science, Otsu 520-2192, Japan"},{"name":"Department of Neurosurgery, Nagoya City University Graduate School of Medical Science, Nagoya 467-8601, Japan"},{"name":"Normal Pressure Hydrocephalus Center, Rakuwakai Otowa Hospital, Kyoto 607-8062, Japan"},{"name":"Interfaculty Initiative in Information Studies\/Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan"}]},{"given":"Shigeo","family":"Ueda","sequence":"additional","affiliation":[{"name":"Shin-Aikai Spine Center, Katano Hospital, Katano 576-0043, Japan"}]},{"given":"Chifumi","family":"Iseki","sequence":"additional","affiliation":[{"name":"Division of Neurology and Clinical Neuroscience, Department of Internal Medicine III, Yamagata University School of Medicine, Yamagata 990-9585, Japan"}]},{"given":"Toshiyuki","family":"Kondo","sequence":"additional","affiliation":[{"name":"Division of Neurology and Clinical Neuroscience, Department of Internal Medicine III, Yamagata University School of Medicine, Yamagata 990-9585, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0072-5741","authenticated-orcid":false,"given":"Keisuke","family":"Mori","sequence":"additional","affiliation":[{"name":"School of Medicine, Shiga University of Medical Science, Otsu 520-2192, Japan"}]},{"given":"Yoshiyuki","family":"Kobayashi","sequence":"additional","affiliation":[{"name":"Human Augmentation Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Kashiwa II Campus, University of Tokyo, Kashiwa 277-0882, Japan"}]},{"given":"Tadanori","family":"Fukami","sequence":"additional","affiliation":[{"name":"Department of Informatics and Electronics, Faculty of Engineering, Yamagata University, Yamagata 992-8510, Japan"}]},{"given":"Minoru","family":"Hoshimaru","sequence":"additional","affiliation":[{"name":"Shin-Aikai Spine Center, Katano Hospital, Katano 576-0043, Japan"}]},{"given":"Masatsune","family":"Ishikawa","sequence":"additional","affiliation":[{"name":"Normal Pressure Hydrocephalus Center, Rakuwakai Otowa Hospital, Kyoto 607-8062, Japan"},{"name":"Rakuwa Villa Ilios, Rakuwakai Healthcare System, Kyoto 604-8402, Japan"}]},{"given":"Yasuyuki","family":"Ohta","sequence":"additional","affiliation":[{"name":"Division of Neurology and Clinical Neuroscience, Department of Internal Medicine III, Yamagata University School of Medicine, Yamagata 990-9585, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ishikawa, M., Yamada, S., and Yamamoto, K. (2019). Agreement study on gait assessment using a video-assisted rating method in patients with idiopathic normal-pressure hydrocephalus. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0224202"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"S1","DOI":"10.1227\/01.NEU.0000168188.25559.0E","article-title":"Development of guidelines for idiopathic normal-pressure hydrocephalus: Introduction","volume":"57","author":"Marmarou","year":"2005","journal-title":"Neurosurgery"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"63","DOI":"10.2176\/nmc.st.2020-0292","article-title":"Guidelines for Management of Idiopathic Normal Pressure Hydrocephalus (Third Edition): Endorsed by the Japanese Society of Normal Pressure Hydrocephalus","volume":"61","author":"Nakajima","year":"2021","journal-title":"Neurol. Med. Chir."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"942","DOI":"10.1177\/0269215518760126","article-title":"A systematic review of the diagnostic utility of simple tests of change after trial removal of cerebrospinal fluid in adults with normal pressure hydrocephalus","volume":"32","author":"Scully","year":"2018","journal-title":"Clin. Rehabil."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1678","DOI":"10.1016\/S1388-2457(00)00362-X","article-title":"Gait analysis in idiopathic normal pressure hydrocephalus\u2014which parameters respond to the CSF tap test?","volume":"111","author":"Stolze","year":"2000","journal-title":"Clin. Neurophysiol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1186\/1475-925X-10-99","article-title":"Quantification and recognition of parkinsonian gait from monocular video imaging using kernel-based principal component analysis","volume":"10","author":"Chen","year":"2011","journal-title":"Biomed. Eng. Online"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1016\/j.humov.2007.05.003","article-title":"Gait dynamics, fractals and falls: Finding meaning in the stride-to-stride fluctuations of human walking","volume":"26","author":"Hausdorff","year":"2007","journal-title":"Hum. Mov. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/j.gaitpost.2018.01.008","article-title":"Relationship between gait parameters and MR imaging in idiopathic normal pressure hydrocephalus patients after shunt surgery","volume":"61","author":"Kitade","year":"2018","journal-title":"Gait Posture"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"724","DOI":"10.23736\/S1973-9087.18.04949-3","article-title":"Computerized gait analysis with inertial sensor in the management of idiopathic normal pressure hydrocephalus","volume":"54","author":"Panciani","year":"2018","journal-title":"Eur. J. Phys. Rehabil. Med."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wang, L., Li, Y., Xiong, F., and Zhang, W. (2021). Gait recognition using optical motion capture: A decision fusion based method. Sensors, 21.","DOI":"10.3390\/s21103496"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1097\/PHM.0b013e31815b6461","article-title":"Objective assessment of gait in normal-pressure hydrocephalus","volume":"87","author":"Williams","year":"2008","journal-title":"Am. J. Phys. Med. Rehabil."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"23","DOI":"10.14336\/AD.2018.0426","article-title":"Quantitative evaluation of gait disturbance on an instrumented timed up-and-go test","volume":"10","author":"Yamada","year":"2019","journal-title":"Aging Dis."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"653964","DOI":"10.3389\/fnagi.2021.653964","article-title":"Gait assessment using three-dimensional acceleration of the trunk in idiopathic normal pressure hydrocephalus","volume":"13","author":"Yamada","year":"2021","journal-title":"Front. Aging Neurosci."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Diaz-San Martin, G., Reyes-Gonzalez, L., Sainz-Ruiz, S., Rodriguez-Cobo, L., and Lopez-Higuera, J.M. (2021). Automatic Ankle Angle Detection by Integrated RGB and Depth Camera System. Sensors (Basel), 21.","DOI":"10.3390\/s21051909"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Springer, S., and Yogev Seligmann, G. (2016). Validity of the kinect for gait assessment: A focused review. Sensors, 16.","DOI":"10.3390\/s16020194"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"109929","DOI":"10.1016\/j.jbiomech.2020.109929","article-title":"Reliability of a markerless motion capture system to measure the trunk, hip and knee angle during walking on a flatland and a treadmill","volume":"109","author":"Tamura","year":"2020","journal-title":"J. Biomech."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Baak, A., Muller, M., Bharaj, G., Seidel, H., and Theobalt, C. (2013, January 16\u201317). A data-driven approach for real-time full body pose reconstruction from a depth camera. Proceedings of the IEEE 13th International Conference on Computer Vision, London, UK.","DOI":"10.1007\/978-1-4471-4640-7_5"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Buker, L.C., Zuber, F., Hein, A., and Fudickar, S. (2021). HRDepthNet: Depth Image-Based Marker-Less Tracking of Body Joints. Sensors, 21.","DOI":"10.3390\/s21041356"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2100412","DOI":"10.1109\/JTEHM.2021.3069353","article-title":"Development of a smart hallway for marker-less human foot tracking and stride analysis","volume":"9","author":"Gutta","year":"2021","journal-title":"IEEE J. Transl. Eng. Health Med."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TPAMI.2019.2929257","article-title":"OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields","volume":"43","author":"Cao","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Fernandez-Gonzalez, P., Koutsou, A., Cuesta-Gomez, A., Carratala-Tejada, M., Miangolarra-Page, J.C., and Molina-Rueda, F. (2020). Reliability of Kinovea\u00ae Software and Agreement with a Three-Dimensional Motion System for Gait Analysis in Healthy Subjects. Sensors, 20.","DOI":"10.3390\/s20113154"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Martinez, J., Hossain, R., Romero, J., and Little, J.J. (2017). A simple yet effective baseline for 3D human pose estimation. Int. Conf. Comput. Vis. (ICCV), 2640\u20132649.","DOI":"10.1109\/ICCV.2017.288"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"50","DOI":"10.3389\/fspor.2020.00050","article-title":"Evaluation of 3D Markerless Motion Capture Accuracy Using OpenPose With Multiple Video Cameras","volume":"2","author":"Nakano","year":"2020","journal-title":"Front. Sports Act. Living"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"181","DOI":"10.3389\/fbioe.2020.00181","article-title":"3D Tracking of Human Motion Using Visual Skeletonization and Stereoscopic Vision","volume":"8","author":"Zago","year":"2020","journal-title":"Front. Bioeng. Biotechnol."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_26","first-page":"536","article-title":"Integral Human Pose Regression","volume":"11210","author":"Sun","year":"2018","journal-title":"Eur. Conf. Comput. Vis. (ECCV)"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5282\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:50:39Z","timestamp":1760140239000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5282"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,14]]},"references-count":26,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22145282"],"URL":"https:\/\/doi.org\/10.3390\/s22145282","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,14]]}}}