{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T15:23:18Z","timestamp":1778599398769,"version":"3.51.4"},"reference-count":48,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,2,25]],"date-time":"2022-02-25T00:00:00Z","timestamp":1645747200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Activity recognition based on inertial sensors is an essential task in mobile and ubiquitous computing. To date, the best performing approaches in this task are based on deep learning models. Although the performance of the approaches has been increasingly improving, a number of issues still remain. Specifically, in this paper we focus on the issue of the dependence of today\u2019s state-of-the-art approaches to complex ad hoc deep learning convolutional neural networks (CNNs), recurrent neural networks (RNNs), or a combination of both, which require specialized knowledge and considerable effort for their construction and optimal tuning. To address this issue, in this paper we propose an approach that automatically transforms the inertial sensors time-series data into images that represent in pixel form patterns found over time, allowing even a simple CNN to outperform complex ad hoc deep learning models that combine RNNs and CNNs for activity recognition. We conducted an extensive evaluation considering seven benchmark datasets that are among the most relevant in activity recognition. Our results demonstrate that our approach is able to outperform the state of the art in all cases, based on image representations that are generated through a process that is easy to implement, modify, and extend further, without the need of developing complex deep learning models.<\/jats:p>","DOI":"10.3390\/s22051840","type":"journal-article","created":{"date-parts":[[2022,2,27]],"date-time":"2022-02-27T20:48:33Z","timestamp":1645994913000},"page":"1840","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Improving Wearable-Based Activity Recognition Using Image Representations"],"prefix":"10.3390","volume":"22","author":[{"given":"Alejandro","family":"Sanchez Guinea","sequence":"first","affiliation":[{"name":"Department of Computer Science, Technical University of Darmstadt, 64289 Darmstadt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9472-5034","authenticated-orcid":false,"given":"Mehran","family":"Sarabchian","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Technical University of Darmstadt, 64289 Darmstadt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Max","family":"M\u00fchlh\u00e4user","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Technical University of Darmstadt, 64289 Darmstadt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1038\/scientificamerican0991-94","article-title":"The Computer for the 21 st Century","volume":"265","author":"Weiser","year":"1991","journal-title":"Sci. Am."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1109\/MPRV.2003.1186726","article-title":"Preserving privacy in environments with location-based applications","volume":"2","author":"Myles","year":"2003","journal-title":"IEEE Pervasive Comput."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Guinea, A.S., Sarabchian, M., and M\u00fchlh\u00e4user, M. (2021, January 22\u201326). Image-based Activity Recognition from IMU Data. Proceedings of the 2021 IEEE international conference on pervasive computing and communications workshops (PerCom workshops), Kassel, Germany.","DOI":"10.1109\/PerComWorkshops51409.2021.9430990"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Sanchez Guinea, A., Heinrich, S., and M\u00fchlh\u00e4user, M. (2021, January 21\u201326). VIDENS: Vision-based User Identification from Inertial Sensors. Proceedings of the 2021 International Symposium on Wearable Computers, Virtual Event.","DOI":"10.1145\/3460421.3480426"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5455","DOI":"10.1007\/s10462-020-09825-6","article-title":"A survey of the recent architectures of deep convolutional neural networks","volume":"53","author":"Khan","year":"2020","journal-title":"Artif. Intell. Rev."},{"key":"ref_6","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ord\u00f3\u00f1ez, F.J., and Roggen, D. (2016). Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16.","DOI":"10.3390\/s16010115"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ha, S., Yun, J.M., and Choi, S. (2015, January 9\u201312). Multi-modal convolutional neural networks for activity recognition. Proceedings of the 2015 IEEE International conference on systems, man, and cybernetics, Hong Kong, China.","DOI":"10.1109\/SMC.2015.525"},{"key":"ref_9","unstructured":"Yang, J.B., Nguyen, M.N., San, P.P., Li, X.L., and Krishnaswamy, S. (2015, January 25\u201331). Deep convolutional neural networks on multichannel time series for human activity recognition. Proceedings of the 24th International Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_10","unstructured":"Hammerla, N.Y., Halloran, S., and Pl\u00f6tz, T. (2016, January 9\u201315). Deep, convolutional, and recurrent models for human activity recognition using wearables. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ha, S., and Choi, S. (2016, January 24\u201329). Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors. Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada.","DOI":"10.1109\/IJCNN.2016.7727224"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1145\/3381012","article-title":"METIER: A Deep Multi-Task Learning Based Activity and User Recognition Model Using Wearable Sensors","volume":"4","author":"Chen","year":"2020","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3328919","article-title":"Au-Id: Automatic User Identification and Authentication Through the Motions Captured from Sequential Human Activities Using RFID","volume":"3","author":"Huang","year":"2019","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1145\/3397330","article-title":"Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables","volume":"4","author":"Sheng","year":"2020","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_15","unstructured":"Wang, Z., and Oates, T. (2015, January 25\u201326). Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. Proceedings of the Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA."},{"key":"ref_16","unstructured":"Wang, Z., and Oates, T. (2015, January 25\u201331). Imaging time-series to improve classification and imputation. Proceedings of the 24th International Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_17","first-page":"106960Y","article-title":"Classification of time-series images using deep convolutional neural networks","volume":"Volume 10696","author":"Hatami","year":"2018","journal-title":"Proceedings of the Tenth International Conference on Machine Vision (ICMV 2017)"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yang, C.L., Yang, C.Y., Chen, Z.X., and Lo, N.W. (2019, January 14\u201316). Multivariate Time Series Data Transformation for Convolutional Neural Network. Proceedings of the 2019 IEEE\/SICE International Symposium on System Integration (SII), Paris, France.","DOI":"10.1109\/SII.2019.8700425"},{"key":"ref_19","unstructured":"Alsheikh, M.A., Selim, A., Niyato, D., Doyle, L., Lin, S., and Tan, H.P. (2016, January 12\u201317). Deep activity recognition models with triaxial accelerometers. Proceedings of the Workshops at the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA."},{"key":"ref_20","first-page":"1","article-title":"Deep learning-based model architecture for time-frequency images analysis","volume":"9","author":"Alaskar","year":"2018","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"8344","DOI":"10.1038\/s41598-020-65264-x","article-title":"Automated detection of the head-twitch response using wavelet scalograms and a deep convolutional neural network","volume":"10","author":"Halberstadt","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"20398","DOI":"10.1109\/JSEN.2021.3095176","article-title":"AdaptNet: Human Activity Recognition via Bilateral Domain Adaptation Using Semi-Supervised Deep Translation Networks","volume":"21","author":"An","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Jiang, W., and Yin, Z. (2015, January 26\u201330). Human activity recognition using wearable sensors by deep convolutional neural networks. Proceedings of the 23rd ACM International Conference on Multimedia, Brisbane, Australia.","DOI":"10.1145\/2733373.2806333"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1016\/j.asoc.2017.09.027","article-title":"Real-time human activity recognition from accelerometer data using Convolutional Neural Networks","volume":"62","author":"Ignatov","year":"2018","journal-title":"Appl. Soft Comput."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1109\/TCSI.2018.2848647","article-title":"Sensornet: A scalable and low-power deep convolutional neural network for multimodal data classification","volume":"66","author":"Jafari","year":"2018","journal-title":"IEEE Trans. Circuits Syst. I Regul. Pap."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wang, R. (2016). Edge detection using convolutional neural network. International Symposium on Neural Networks, Springer.","DOI":"10.1007\/978-3-319-40663-3_2"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"6317","DOI":"10.1109\/JSEN.2019.2911204","article-title":"Robust Single Accelerometer-Based Activity Recognition Using Modified Recurrence Plot","volume":"19","author":"Lu","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Haresamudram, H., Anderson, D.V., and Pl\u00f6tz, T. (2019, January 9\u201313). On the role of features in human activity recognition. Proceedings of the 23rd International Symposium on Wearable Computers, London, UK.","DOI":"10.1145\/3341163.3347727"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1145\/1964897.1964918","article-title":"Activity recognition using cell phone accelerometers","volume":"12","author":"Kwapisz","year":"2011","journal-title":"ACM SigKDD Explor. Newsl."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3214269","article-title":"DeActive: Scaling activity recognition with active deep learning","volume":"2","author":"Hossain","year":"2018","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_31","unstructured":"Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2013). A Public Domain Dataset for Human Activity Recognition Using Smartphones, Esann."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Murad, A., and Pyun, J.Y. (2017). Deep recurrent neural networks for human activity recognition. Sensors, 17.","DOI":"10.3390\/s17112556"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2033","DOI":"10.1016\/j.patrec.2012.12.014","article-title":"The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition","volume":"34","author":"Chavarriaga","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1109\/TITB.2009.2036165","article-title":"Wearable assistant for Parkinson\u2019s disease patients with the freezing of gait symptom","volume":"14","author":"Bachlin","year":"2009","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/MPRV.2008.40","article-title":"Wearable activity tracking in car manufacturing","volume":"7","author":"Stiefmeier","year":"2008","journal-title":"IEEE Pervasive Comput."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zeng, M., Gao, H., Yu, T., Mengshoel, O.J., Langseth, H., Lane, I., and Liu, X. (2018, January 8\u201312). Understanding and improving recurrent networks for human activity recognition by continuous attention. Proceedings of the 2018 ACM International Symposium on Wearable Computers, Singapore.","DOI":"10.1145\/3267242.3267286"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zeng, M., Nguyen, L.T., Yu, B., Mengshoel, O.J., Zhu, J., Wu, P., and Zhang, J. (2014, January 6\u20137). Convolutional neural networks for human activity recognition using mobile sensors. Proceedings of the 6th International Conference on Mobile Computing, Applications and Services, Austin, TX, USA.","DOI":"10.4108\/icst.mobicase.2014.257786"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"75213","DOI":"10.1109\/ACCESS.2019.2920969","article-title":"Human Activity Recognition Based on Motion Sensor Using U-Net","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_40","first-page":"5614","article-title":"A novel distribution-embedded neural network for sensor-based activity recognition","volume":"2019","author":"Qian","year":"2019","journal-title":"IJCAI"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.eswa.2016.04.032","article-title":"Human activity recognition with smartphone sensors using deep learning neural networks","volume":"59","author":"Ronao","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"4334","DOI":"10.1109\/TII.2018.2789925","article-title":"Distilling the knowledge from handcrafted features for human activity recognition","volume":"14","author":"Chen","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Xi, R., Hou, M., Fu, M., Qu, H., and Liu, D. (2018, January 8\u201313). Deep dilated convolution on multimodality time series for human activity recognition. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489540"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"56855","DOI":"10.1109\/ACCESS.2020.2982225","article-title":"LSTM-CNN Architecture for Human Activity Recognition","volume":"8","author":"Xia","year":"2020","journal-title":"IEEE Access"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3432208","article-title":"GIobalFusion: A Global Attentional Deep Learning Framework for Multisensor Information Fusion","volume":"4","author":"Liu","year":"2020","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3090076","article-title":"Ensembles of deep lstm learners for activity recognition using wearables","volume":"1","author":"Guan","year":"2017","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Ma, H., Li, W., Zhang, X., Gao, S., and Lu, S. AttnSense: Multi-level attention mechanism for multimodal human activity recognition. Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, 10\u201316 August 2019.","DOI":"10.24963\/ijcai.2019\/431"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Zhang, Y., Zhang, Z., Bao, J., and Song, Y. (2018). Human activity recognition based on time series analysis using U-Net. arXiv.","DOI":"10.1109\/ACCESS.2019.2920969"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/5\/1840\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:27:37Z","timestamp":1760135257000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/5\/1840"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,25]]},"references-count":48,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["s22051840"],"URL":"https:\/\/doi.org\/10.3390\/s22051840","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,25]]}}}