{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T16:27:20Z","timestamp":1772555240301,"version":"3.50.1"},"reference-count":59,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2021,9,24]],"date-time":"2021-09-24T00:00:00Z","timestamp":1632441600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Institute of Information &amp; Communications Technology Planning &amp; Evaluation","award":["2019-0-00050"],"award-info":[{"award-number":["2019-0-00050"]}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2020R1C1C1010666"],"award-info":[{"award-number":["2020R1C1C1010666"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Generally, people do various things while walking. For example, people frequently walk while looking at their smartphones. Sometimes we walk differently than usual; for example, when walking on ice or snow, we tend to waddle. Understanding walking patterns could provide users with contextual information tailored to the current situation. To formulate this as a machine-learning problem, we defined 18 different everyday walking styles. Noting that walking strategies significantly affect the spatiotemporal features of hand motions, e.g., the speed and intensity of the swinging arm, we propose a smartwatch-based wearable system that can recognize these predefined walking styles. We developed a wearable system, suitable for use with a commercial smartwatch, that can capture hand motions in the form of multivariate timeseries (MTS) signals. Then, we employed a set of machine learning algorithms, including feature-based and recent deep learning algorithms, to learn the MTS data in a supervised fashion. Experimental results demonstrated that, with recent deep learning algorithms, the proposed approach successfully recognized a variety of walking patterns, using the smartwatch measurements. We analyzed the results with recent attention-based recurrent neural networks to understand the relative contributions of the MTS signals in the classification process.<\/jats:p>","DOI":"10.3390\/s21196393","type":"journal-article","created":{"date-parts":[[2021,9,27]],"date-time":"2021-09-27T22:16:38Z","timestamp":1632780998000},"page":"6393","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Recognition of Fine-Grained Walking Patterns Using a Smartwatch with Deep Attentive Neural Networks"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0917-1781","authenticated-orcid":false,"given":"Hyejoo","family":"Kim","sequence":"first","affiliation":[{"name":"Machine Learning Systems Lab., College of Sports Science, Sungkyunkwan University, Suwon 16419, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8066-6563","authenticated-orcid":false,"given":"Hyeon-Joo","family":"Kim","sequence":"additional","affiliation":[{"name":"Machine Learning Systems Lab., College of Sports Science, Sungkyunkwan University, Suwon 16419, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6440-1493","authenticated-orcid":false,"given":"Jinyoon","family":"Park","sequence":"additional","affiliation":[{"name":"Machine Learning Systems Lab., College of Sports Science, Sungkyunkwan University, Suwon 16419, Korea"},{"name":"The Department of Sport Interaction Science (SIS), Sungkyunkwan University, Suwon 16419, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6942-9399","authenticated-orcid":false,"given":"Jeh-Kwang","family":"Ryu","sequence":"additional","affiliation":[{"name":"Department of Physical Education, College of Education, Dongguk University, Seoul 04620, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7292-5166","authenticated-orcid":false,"given":"Seung-Chan","family":"Kim","sequence":"additional","affiliation":[{"name":"Machine Learning Systems Lab., College of Sports Science, Sungkyunkwan University, Suwon 16419, Korea"},{"name":"The Department of Sport Interaction Science (SIS), Sungkyunkwan University, Suwon 16419, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kwon, M.C., Ju, M., and Choi, S. (2017, January 4\u20137). Classification of various daily behaviors using deep learning and smart watch. Proceedings of the 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN), Milan, Italy.","DOI":"10.1109\/ICUFN.2017.7993888"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/0020294018813692","article-title":"Human activity recognition from smart watch sensor data using a hybrid of principal component analysis and random forest algorithm","volume":"52","author":"Balli","year":"2019","journal-title":"Meas. Control"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e001004","DOI":"10.1136\/bmjsem-2020-001004","article-title":"Predicting lying, sitting, walking and running using Apple Watch and Fitbit data","volume":"7","author":"Fuller","year":"2021","journal-title":"BMJ Open Sport Exerc. Med."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Weiss, G.M., Timko, J.L., Gallagher, C.M., Yoneda, K., and Schreiber, A.J. (2016, January 24\u201327). Smartwatch-based activity recognition: A machine learning approach. Proceedings of the 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Las Vegas, NV, USA.","DOI":"10.1109\/BHI.2016.7455925"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Laput, G., and Harrison, C. (2019, January 4\u20139). Sensing Fine-grained hand activity with smartwatches. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, Scotland.","DOI":"10.1145\/3290605.3300568"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, N., Ambikairajah, E., Lovell, N.H., and Celler, B.G. (2007, January 22\u201326). Accelerometry based classification of walking patterns using time-frequency analysis. Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Lyon, France.","DOI":"10.1109\/IEMBS.2007.4353438"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1109\/TITB.2009.2036165","article-title":"Wearable assistant for Parkinson\u2019s disease patients with the freezing of gait symptom","volume":"14","author":"Bachlin","year":"2009","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1145\/1964897.1964918","article-title":"Activity recognition using cell phone accelerometers","volume":"12","author":"Kwapisz","year":"2011","journal-title":"ACM SIGKDD Explor. Newsl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.neucom.2018.03.067","article-title":"Time series feature extraction on basis of scalable hypothesis tests (tsfresh\u2013a python package)","volume":"307","author":"Christ","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_10","unstructured":"Alsheikh, M.A., Selim, A., Niyato, D., Doyle, L., Lin, S., and Tan, H.P. (2016, January 12\u201317). Deep activity recognition models with triaxial accelerometers. Proceedings of the Workshops at the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1016\/S0003-9993(97)90034-4","article-title":"Increased gait unsteadiness in community-dwelling elderly fallers","volume":"78","author":"Hausdorff","year":"1997","journal-title":"Arch. Phys. Med. Rehabil."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Saito, K., Zecca, M., Sessa, S., Lin, Z., Bartolomeo, L., Cosentino, S., Petersen, K., Ishii, H., Ikai, T., and Takanishi, A. (2012, January 7\u20139). Assessment of walking quality by using inertial measurement units. Proceedings of the 2012 First International Conference on Innovative Engineering Systems, Alexandria, Egypt.","DOI":"10.1109\/ICIES.2012.6530837"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ferscha, A., and Mattern, F. (2004). Activity recognition from user-annotated acceleration data. Pervasive Computing, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/b96922"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2034","DOI":"10.3390\/s18072034","article-title":"A robust step detection algorithm and walking distance estimation based on daily wrist activity recognition using a smart band","volume":"18","author":"Bui","year":"2018","journal-title":"Sensors"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2884","DOI":"10.1109\/TBME.2012.2212245","article-title":"Walking pattern classification and walking distance estimation algorithms using gait phase information","volume":"59","author":"Wang","year":"2012","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2506","DOI":"10.1109\/TBME.2010.2049357","article-title":"Can triaxial accelerometry accurately recognize inclined walking terrains?","volume":"57","author":"Wang","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Alemayoh, T., Lee, J., and Okamoto, S. (2021). New Sensor data structuring for deeper feature extraction in human activity recognition. Sensors, 21.","DOI":"10.3390\/s21082814"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hanai, Y., Nishimura, J., and Kuroda, T. (2009, January 4\u20137). Haar-like filtering for human activity recognition using 3d accelerometer. Proceedings of the 2009 IEEE 13th digital Signal Processing Workshop and 5th IEEE signal Processing Education Workshop, Marco Island, FL, USA.","DOI":"10.1109\/DSP.2009.4786008"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Lockhart, J.W., and Weiss, G.M. (2014, January 24\u201326). The benefits of personalized smartphone-based activity recognition models. Proceedings of the SIAM international Conference On Data Mining, Philadelphia, PA, USA.","DOI":"10.1137\/1.9781611973440.71"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ord\u00f3\u00f1ez, F.J., and Roggen, D. (2016). Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16.","DOI":"10.3390\/s16010115"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Mekruksavanich, S., Hnoohom, N., and Jitpattanakul, A. (2018, January 25\u201328). Smartwatch-based sitting detection with human activity recognition for office workers syndrome. Proceedings of the 2018 International ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI-NCON), Chiang Rai, Thailand.","DOI":"10.1109\/ECTI-NCON.2018.8378302"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Mekruksavanich, S., and Jitpattanakul, A. (2020, January 25\u201328). Smartwatch-based human activity recognition using hybrid lstm network. Proceedings of the 2020 IEEE Sensors, Rotterdam, The Netherlands.","DOI":"10.1109\/SENSORS47125.2020.9278630"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 16\u201320). Deep high-resolution representation learning for human pose estimation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00584"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1214\/aos\/1013699998","article-title":"The control of the false discovery rate in multiple testing under dependency","volume":"29","author":"Benjamini","year":"2001","journal-title":"Ann. Stat."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_26","first-page":"18","article-title":"Classification and regression by randomForest","volume":"2","author":"Liaw","year":"2002","journal-title":"R News"},{"key":"ref_27","unstructured":"Bishop, C.M. (2006). Pattern Recognition and Machine Learning, Springer."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.patrec.2005.08.011","article-title":"Random forests for land cover classification","volume":"27","author":"Gislason","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_29","unstructured":"Kang, G., and Kim, S.-C. (2020). DeepEcho: Echoacoustic recognition of materials using returning echoes with deep neural networks. IEEE Trans. Emerg. Top. Comput., in press."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 11\u201318). Learning spatiotemporal features with 3d convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Las Condes, Chile.","DOI":"10.1109\/ICCV.2015.510"},{"key":"ref_31","unstructured":"LeCun, Y., and Bengio, Y. (1995). Convolutional networks for images, speech, and time series. The Handbook of Brain Theory and Neural Networks, MIT Press."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014, January 25\u201329). Convolutional neural networks for sentence classification. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_33","first-page":"10739","article-title":"1D convolutional neural networks and applications: A survey","volume":"151","author":"Kiranyaz","year":"2020","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"e1700578","DOI":"10.1126\/sciadv.1700578","article-title":"Convolutional neural network for earthquake detection and location","volume":"4","author":"Perol","year":"2018","journal-title":"Sci. Adv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1109\/LES.2020.2996758","article-title":"Embedded identification of surface based on multirate sensor fusion with deep neural network","volume":"13","author":"Ryu","year":"2020","journal-title":"IEEE Embed. Syst. Lett."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Han, B.-K., Ryu, J.-K., and Kim, S.-C. (2019). Context-aware winter sports based on multivariate sequence learning. Sensors, 19.","DOI":"10.3390\/s19153296"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kiranyaz, S., Ince, T., Hamila, R., and Gabbouj, M. (2015, January 25\u201329). Convolutional neural networks for patient-specific ECG classification. Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.","DOI":"10.1109\/EMBC.2015.7318926"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Ryu, S., and Kim, S.-C. (2020). Knocking and listening: Learning mechanical impulse response for understanding surface characteristics. Sensors, 20.","DOI":"10.3390\/s20020369"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_40","unstructured":"Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"76263","DOI":"10.1109\/ACCESS.2018.2880882","article-title":"DeepSnake: Sequence learning of joint torques using a gated recurrent neural network","volume":"6","author":"Han","year":"2018","journal-title":"IEEE Access"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1461","DOI":"10.1007\/s00607-021-00928-8","article-title":"Multi-input CNN-GRU based human activity recognition using wearable sensors","volume":"103","author":"Dua","year":"2021","journal-title":"Computing"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Graves, A., Mohamed, A.-R., and Hinton, G. (2013, January 26\u201331). Speech recognition with deep recurrent neural networks. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Vancuver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"ref_45","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Luong, T., Pham, H., and Manning, C. (2015). Effective approaches to attention-based neural machine translation. arXiv.","DOI":"10.18653\/v1\/D15-1166"},{"key":"ref_47","unstructured":"Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015, January 6\u201311). Show, attend and tell: Neural image caption generation with visual attention. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Ran, X., Shan, Z., Fang, Y., and Lin, C. (2019). An LSTM-based method with attention mechanism for travel time prediction. Sensors, 19.","DOI":"10.3390\/s19040861"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Zeng, M., Gao, H., Yu, T., Mengshoel, O.J., Langseth, H., Lane, I., and Liu, X. (2018, January 8\u201312). Understanding and improving recurrent networks for human activity recognition by continuous attention. Proceedings of the 2018 ACM International Symposium on Wearable Computers, Singapore.","DOI":"10.1145\/3267242.3267286"},{"key":"ref_50","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_51","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Haque, M.N., Mahbub, M., Tarek, M.H., Lota, L.N., and Ali, A.A. (2019, January 11\u201313). Nurse care activity recognition: A GRU-based approach with attention mechanism. Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and In Proceedings of the 2019 ACM International Symposium on Wearable Computers, London, UK.","DOI":"10.1145\/3341162.3344848"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Ma, H., Li, W., Zhang, X., Gao, S., and Lu, S. (2019, January 10\u201316). AttnSense: Multi-level attention mechanism for multimodal human activity recognition. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, Macao, China.","DOI":"10.24963\/ijcai.2019\/431"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"107728","DOI":"10.1016\/j.asoc.2021.107728","article-title":"DanHAR: Dual attention network for multimodal human activity recognition using wearable sensors","volume":"111","author":"Gao","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Reiss, A., and Stricker, D. (2012, January 12\u201316). Introducing a new benchmarked dataset for activity monitoring. Proceedings of the 2012 16th International Symposium On Wearable Computers, Heidelberg, Germany.","DOI":"10.1109\/ISWC.2012.13"},{"key":"ref_56","unstructured":"Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2013, January 24\u201326). A public domain dataset for human activity recognition using smartphones. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"754","DOI":"10.1016\/j.neucom.2015.07.085","article-title":"Transition-aware human activity recognition using smartphones","volume":"171","author":"Oneto","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.eswa.2016.10.065","article-title":"Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN","volume":"72","author":"Chen","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"2673","DOI":"10.1109\/78.650093","article-title":"Bidirectional recurrent neural networks","volume":"45","author":"Schuster","year":"1997","journal-title":"IEEE Trans. Signal Process."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/19\/6393\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:04:36Z","timestamp":1760166276000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/19\/6393"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,24]]},"references-count":59,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["s21196393"],"URL":"https:\/\/doi.org\/10.3390\/s21196393","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,24]]}}}