{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T12:33:00Z","timestamp":1789993980100,"version":"4.0.1"},"reference-count":38,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2019,4,16]],"date-time":"2019-04-16T00:00:00Z","timestamp":1555372800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["ES 434\/8-1"],"award-info":[{"award-number":["ES 434\/8-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014419","name":"EIT health","doi-asserted-by":"publisher","award":["HOOP 2.0"],"award-info":[{"award-number":["HOOP 2.0"]}],"id":[{"id":"10.13039\/100014419","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Activity monitoring using wearables is becoming ubiquitous, although accurate cycle level analysis, such as step-counting and gait analysis, are limited by a lack of realistic and labeled datasets. The effort required to obtain and annotate such datasets is massive, therefore we propose a smart annotation pipeline which reduces the number of events needing manual adjustment to 14%. For scenarios dominated by walking, this annotation effort is as low as 8%. The pipeline consists of three smart annotation approaches, namely edge detection of the pressure data, local cyclicity estimation, and iteratively trained hierarchical hidden Markov models. Using this pipeline, we have collected and labeled a dataset with over 150,000 labeled cycles, each with 2 phases, from 80 subjects, which we have made publicly available. The dataset consists of 12 different task-driven activities, 10 of which are cyclic. These activities include not only straight and steady-state motions, but also transitions, different ranges of bouts, and changing directions. Each participant wore 5 synchronized inertial measurement units (IMUs) on the wrists, shoes, and in a pocket, as well as pressure insoles and video. We believe that this dataset and smart annotation pipeline are a good basis for creating a benchmark dataset for validation of other semi- and unsupervised algorithms.<\/jats:p>","DOI":"10.3390\/s19081820","type":"journal-article","created":{"date-parts":[[2019,4,17]],"date-time":"2019-04-17T03:02:01Z","timestamp":1555470121000},"page":"1820","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Hidden Markov Model-Based Smart Annotation for Benchmark Cyclic Activity Recognition Database Using Wearables"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9397-5944","authenticated-orcid":false,"given":"Christine F.","family":"Martindale","sequence":"first","affiliation":[{"name":"Machine Learning and Data Analytics Lab, Computer Science Department, 91052 Erlangen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3711-1110","authenticated-orcid":false,"given":"Sebastijan","family":"Sprager","sequence":"additional","affiliation":[{"name":"Faculty of Computer and Information Science, University of Ljubljana, 1000 Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0417-0336","authenticated-orcid":false,"given":"Bjoern M.","family":"Eskofier","sequence":"additional","affiliation":[{"name":"Machine Learning and Data Analytics Lab, Computer Science Department, 91052 Erlangen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,4,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1303","DOI":"10.1007\/s40279-016-0663-1","article-title":"Step Counting: A Review of Measurement Considerations and Health-Related Applications","volume":"47","author":"Bassett","year":"2017","journal-title":"Sports Med."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Toph\u00f8j, K.H., Petersen, M.G., S\u00e6bye, C., Baad-Hansen, T., and Wagner, S. (2018). Validity and Reliability Evaluation of Four Commercial Activity Trackers\u2019 Step Counting Performance. Telemed. e-Health.","DOI":"10.1089\/tmj.2017.0264"},{"key":"ref_3","first-page":"503","article-title":"Current State of Commercial Wearable Technology in Physical Activity Monitoring 2015-2017","volume":"11","author":"Bunn","year":"2018","journal-title":"Int. J. Exerc. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1315","DOI":"10.1249\/MSS.0000000000001569","article-title":"Video-Recorded Validation of Wearable Step Counters under Free-living Conditions","volume":"50","author":"Toth","year":"2018","journal-title":"Med. Sci. Sports Exerc."},{"key":"ref_5","unstructured":"De la Torre, F., Hodgins, J., Montano, J., Valcarcel, S., Forcada, R., and Macey, J. (2009). Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database, Carnegie Mellon University. CMU-RI-TR-08-22."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Leutheuser, H., Schuldhaus, D., and Eskofier, B.M. (2013). Hierarchical, Multi-Sensor Based Classification of Daily Life Activities: Comparison with State-of-the-Art Algorithms Using a Benchmark Dataset. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0075196"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., and Bajcsy, R. (2013, January 15\u201317). Berkeley MHAD: A comprehensive Multimodal Human Action Database. Proceedings of the 2013 IEEE Workshop on Applications of Computer Vision (WACV), Tampa, FL, USA.","DOI":"10.1109\/WACV.2013.6474999"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kluge, F., Ga\u00dfner, H., Hannink, J., Pasluosta, C., Klucken, J., and Eskofier, B. (2017). Towards Mobile Gait Analysis: Concurrent Validity and Test-Retest Reliability of an Inertial Measurement System for the Assessment of Spatio-Temporal Gait Parameters. Sensors, 17.","DOI":"10.3390\/s17071522"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.gaitpost.2016.09.023","article-title":"Evaluation of the performance of accelerometer-based gait event detection algorithms in different real-world scenarios using the MAREA gait database","volume":"51","author":"Khandelwal","year":"2017","journal-title":"Gait Posture"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Trung, N.T., Makihara, Y., Nagahara, H., Mukaigawa, Y., and Yagi, Y. (April, January 29). Performance evaluation of gait recognition using the largest inertial sensor-based gait database. Proceedings of the 2012 5th IAPR International Conference on Biometrics (ICB), New Delhi, India.","DOI":"10.1109\/ICB.2012.6199833"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Casilari, E., Santoyo-Ram\u00f3n, J.A., and Cano-Garc\u00eda, J.M. (2017). Analysis of Public Datasets for Wearable Fall Detection Systems. Sensors, 17.","DOI":"10.3390\/s17071513"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Whitehouse, S., Yordanova, K., Paiement, A., and Mirmehdi, M. (2016, January 24\u201325). Recognition of unscripted kitchen activities and eating behaviour for health monitoring. Proceedings of the 2nd IET International Conference on Technologies for Active and Assisted Living (TechAAL 2016), London, UK.","DOI":"10.1049\/ic.2016.0050"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yordanova, K., Kruger, F., and Kirste, T. (2018, January 19\u201323). Providing Semantic Annotation for the CMU Grand Challenge Dataset. Proceedings of the 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Athens, Greece.","DOI":"10.1109\/PERCOMW.2018.8480380"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Diete, A., Sztyler, T., and Stuckenschmidt, H. (2018). Exploring Semi-Supervised Methods for Labeling Support in Multimodal Datasets. Sensors, 18.","DOI":"10.3390\/s18082639"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Miu, T., Missier, P., and Plotz, T. (2015, January 26\u201328). Bootstrapping Personalised Human Activity Recognition Models Using Online Active Learning. Proceedings of the 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, Liverpool, UK.","DOI":"10.1109\/CIT\/IUCC\/DASC\/PICOM.2015.170"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Martindale, C.F., Roth, N., Hannink, J., Sprager, S., and Eskofier, B.M. (2018, January 19\u201323). Smart Annotation Tool for Multi-sensor Gait-based Daily Activity Data. Proceedings of the 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Athens, Greece.","DOI":"10.1109\/PERCOMW.2018.8480193"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"\u0160prager, S., and Juri\u010d, M. (2018). Robust Stride Segmentation of Inertial Signals Based on Local Cyclicity Estimation. Sensors, 18.","DOI":"10.3390\/s18041091"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ye, J. (2018, January 19\u201323). SLearn: Shared learning human activity labels across multiple datasets. Proceedings of the 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom), Athens, Greece.","DOI":"10.1109\/PERCOM.2018.8444594"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A Survey on Transfer Learning","volume":"22","author":"Pan","year":"2010","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Pius Owoh, N., Mahinderjit Singh, M., and Zaaba, Z.F. (2018). Automatic Annotation of Unlabeled Data from Smartphone-Based Motion and Location Sensors. Sensors, 18.","DOI":"10.3390\/s18072134"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.pmcj.2017.01.003","article-title":"Information gain-based metric for recognizing transitions in human activities","volume":"38","author":"Sadri","year":"2017","journal-title":"Pervasive Mobile Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Liu, R., Chen, T., and Huang, L. (2010, January 11\u201314). Research on human activity recognition based on active learning. Proceedings of the 2010 International Conference on Machine Learning and Cybernetics, Qingdao, China.","DOI":"10.1109\/ICMLC.2010.5581050"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.pmcj.2016.08.017","article-title":"Active learning enabled activity recognition","volume":"38","author":"Hossain","year":"2017","journal-title":"Pervasive Mobile Comput."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Maekawa, T., Nakai, D., Ohara, K., and Namioka, Y. (2016, January 12\u201316). Toward practical factory activity recognition. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing\u2014UbiComp \u201916, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971721"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1109\/THMS.2015.2489688","article-title":"Toward Personalized Activity Recognition Systems With a Semipopulation Approach","volume":"46","author":"Hong","year":"2016","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Diete, A., Sztyler, T., and Stuckenschmidt, H. (2017, January 13\u201317). A smart data annotation tool for multi-sensor activity recognition. Proceedings of the 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Kona, HI, USA.","DOI":"10.1109\/PERCOMW.2017.7917542"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Alemdar, H., van Kasteren, T.L.M., and Ersoy, C. (2011). Using Active Learning to Allow Activity Recognition on a Large Scale. Ambient Intelligence, Springer.","DOI":"10.1007\/978-3-642-25167-2_12"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sztyler, T., and Stuckenschmidt, H. (2016, January 14\u201319). On-body Localization of Wearable Devices: An Investigation of Position-Aware Activity Recognition. Proceedings of the 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), Sydney, Australia.","DOI":"10.1109\/PERCOM.2016.7456521"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Brajdic, A., and Harle, R. (2013, January 8\u201312). Walk detection and step counting on unconstrained smartphones. Proceedings of the 2013 ACM International Joint Conference on Pervasive and ubiquitous computing, Zurich, Switzerland.","DOI":"10.1145\/2493432.2493449"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1864","DOI":"10.1109\/TCYB.2014.2361287","article-title":"Accelerometer-Based Gait Recognition by Sparse Representation of Signature Points With Clusters","volume":"45","author":"Zhang","year":"2015","journal-title":"IEEE Trans. Cybern."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Martindale, C., Hoenig, F., Strohrmann, C., and Eskofier, B. (2017). Smart Annotation of Cyclic Data Using Hierarchical Hidden Markov Models. Sensors, 17.","DOI":"10.3390\/s17102328"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Leutheuser, H., Doelfel, S., Schuldhaus, D., Reinfelder, S., and Eskofier, B.M. (2014, January 16\u201319). Performance Comparison of Two Step Segmentation Algorithms Using Different Step Activities. Proceedings of the 2014 11th International Conference on Wearable and Implantable Body Sensor Networks, Zurich, Switzerland.","DOI":"10.1109\/BSN.2014.37"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"6419","DOI":"10.3390\/s150306419","article-title":"Stride Segmentation during Free Walk Movements Using Multi-Dimensional Subsequence Dynamic Time Warping on Inertial Sensor Data","volume":"15","author":"Barth","year":"2015","journal-title":"Sensors"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Groh, B.H., Fleckenstein, M., and Eskofier, B.M. (2016, January 14\u201317). Wearable trick classification in freestyle snowboarding. Proceedings of the 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN), San Francisco, CA, USA.","DOI":"10.1109\/BSN.2016.7516238"},{"key":"ref_35","unstructured":"(2017, November 01). Moticon Science. Available online: http:\/\/www.moticon.de\/science\/."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Martindale, C., Roth, N., Ga\u00dfner, H., Jensen, D., Kohl, Z., and Eskofier, B. (2018, January 18\u201321). Mobile Gait Analysis Using Personalised Hidden Markov Models for Hereditary Spastic Paraplegia Patients. Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8513508"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Taborri, J., Palermo, E., Rossi, S., and Cappa, P. (2016). Gait Partitioning Methods: A Systematic Review. Sensors, 16.","DOI":"10.3390\/s16010066"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Steidl, S., Riedhammer, K., and Bocklet, T. (2011, January 27\u201331). Java Visual Speech Components for Rapid Application Development of GUI Based Speech Processing Applications. Proceedings of the 12th Annual Conference of the International Speech Communication Association (INTERSPEECH 2011), Florence, Italy.","DOI":"10.21437\/Interspeech.2011-821"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/8\/1820\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:45:55Z","timestamp":1760186755000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/8\/1820"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,16]]},"references-count":38,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2019,4]]}},"alternative-id":["s19081820"],"URL":"https:\/\/doi.org\/10.3390\/s19081820","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,4,16]]}}}