{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T16:24:57Z","timestamp":1771518297480,"version":"3.50.1"},"reference-count":53,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2018,11,13]],"date-time":"2018-11-13T00:00:00Z","timestamp":1542067200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010418","name":"Institute for Information and communications Technology Promotion","doi-asserted-by":"publisher","award":["2017-0-00655"],"award-info":[{"award-number":["2017-0-00655"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Ministry of Science and ICT","award":["IITP-2017-0-01629"],"award-info":[{"award-number":["IITP-2017-0-01629"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The most significant barrier to success in human activity recognition is extracting and selecting the right features. In traditional methods, the features are chosen by humans, which requires the user to have expert knowledge or to do a large amount of empirical study. Newly developed deep learning technology can automatically extract and select features. Among the various deep learning methods, convolutional neural networks (CNNs) have the advantages of local dependency and scale invariance and are suitable for temporal data such as accelerometer (ACC) signals. In this paper, we propose an efficient human activity recognition method, namely Iss2Image (Inertial sensor signal to Image), a novel encoding technique for transforming an inertial sensor signal into an image with minimum distortion and a CNN model for image-based activity classification. Iss2Image converts real number values from the X, Y, and Z axes into three color channels to precisely infer correlations among successive sensor signal values in three different dimensions. We experimentally evaluated our method using several well-known datasets and our own dataset collected from a smartphone and smartwatch. The proposed method shows higher accuracy than other state-of-the-art approaches on the tested datasets.<\/jats:p>","DOI":"10.3390\/s18113910","type":"journal-article","created":{"date-parts":[[2018,11,14]],"date-time":"2018-11-14T10:58:22Z","timestamp":1542193102000},"page":"3910","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":64,"title":["Iss2Image: A Novel Signal-Encoding Technique for CNN-Based Human Activity Recognition"],"prefix":"10.3390","volume":"18","author":[{"given":"Taeho","family":"Hur","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Kyung Hee University, (Global Campus), 1732, Deogyeong-daero, Giheung-gu, Yongin-si, Gyeonggi-do 17104, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3675-2258","authenticated-orcid":false,"given":"Jaehun","family":"Bang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Kyung Hee University, (Global Campus), 1732, Deogyeong-daero, Giheung-gu, Yongin-si, Gyeonggi-do 17104, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9172-2935","authenticated-orcid":false,"given":"Thien","family":"Huynh-The","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Kyung Hee University, (Global Campus), 1732, Deogyeong-daero, Giheung-gu, Yongin-si, Gyeonggi-do 17104, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jongwon","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Kyung Hee University, (Global Campus), 1732, Deogyeong-daero, Giheung-gu, Yongin-si, Gyeonggi-do 17104, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jee-In","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Smart ICT Convergence, Konkuk University, 120 Neungdong-ro, Gwangjin-gu, Seoul 05029, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sungyoung","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Kyung Hee University, (Global Campus), 1732, Deogyeong-daero, Giheung-gu, Yongin-si, Gyeonggi-do 17104, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,11,13]]},"reference":[{"key":"ref_1","unstructured":"Pl\u00f6tz, T., Hammerla, N.Y., and Olivier, P. (2011, January 16\u201322). Feature learning for activity recognition in ubiquitous computing. Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Spain."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"522","DOI":"10.1016\/j.procs.2016.09.070","article-title":"Integrating features for accelerometer-based activity recognition","volume":"98","author":"Atasoy","year":"2016","journal-title":"Procedia Comput. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Heaton, J. (April, January 30). An empirical analysis of feature engineering for predictive modeling. Proceedings of the SoutheastCon 2016, Norfolk, VA, USA.","DOI":"10.1109\/SECON.2016.7506650"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zebin, T., Scully, P.J., and Ozanyan, K.B. (November, January 30). Human activity recognition with inertial sensors using a deep learning approach. Proceedings of the 2016 IEEE SENSORS, Orlando, FL, USA.","DOI":"10.1109\/ICSENS.2016.7808590"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1016\/j.pmcj.2017.07.001","article-title":"Learning multi-level features for sensor-based human action recognition","volume":"40","author":"Xu","year":"2017","journal-title":"Pervasive Mob. Comput."},{"key":"ref_6","unstructured":"Zhang, C., and Chen, T. (2003). From low level features to high level semantics. Handbook of Video Databases: Design and Applications, CRC Press."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ha, S., and Choi, S. (2016, January 24\u201329). Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors. Proceedings of the 2016 International Joint Conference on Neural Networks, Vancouver, BC, Canada.","DOI":"10.1109\/IJCNN.2016.7727224"},{"key":"ref_8","first-page":"160","article-title":"An effective deep autoencoder approach for online smartphone-based human activity recognition","volume":"17","author":"Almaslukh","year":"2017","journal-title":"Int. J. Comput. Sci. Netw. Secur."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhang, L., Wu, X., and Luo, D. (2015, January 10\u201314). Real-time activity recognition on smartphones using deep neural networks. Proceedings of the Ubiquitous Intelligence and Computing and 2015 IEEE 12th International Conference on Autonomic and Trusted Computing and 2015 IEEE 15th International Conference on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom), Beijing, China.","DOI":"10.1109\/UIC-ATC-ScalCom-CBDCom-IoP.2015.224"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/s10015-017-0422-x","article-title":"Deep recurrent neural network for mobile human activity recognition with high throughput","volume":"23","author":"Inoue","year":"2018","journal-title":"Artif. Life Rob."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yao, S., Hu, S., Zhao, Y., Zhang, A., and Abdelzaher, T. (2017, January 3\u20137). Deepsense: A unified deep learning framework for time-series mobile sensing data processing. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.","DOI":"10.1145\/3038912.3052577"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1016\/j.patcog.2017.10.013","article-title":"Recent advances in convolutional neural networks","volume":"77","author":"Gu","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zeng, M., Nguyen, L.T., Yu, B., Mengshoel, O.J., Zhu, J., Wu, P., and Zhang, J. (2014, January 6\u20137). Convolutional neural networks for human activity recognition using mobile sensors. Proceedings of the 6th International Conference on Mobile Computing, Applications and Services, Austin, TX, USA.","DOI":"10.4108\/icst.mobicase.2014.257786"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chen, Y., and Xue, Y. (2015, January 9\u201312). A deep learning approach to human activity recognition based on single accelerometer. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China.","DOI":"10.1109\/SMC.2015.263"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Radu, V., Lane, N.D., Bhattacharya, S., Mascolo, C., Marina, M.K., and Kawsar, F. (2016, January 12\u201316). Towards multimodal deep learning for activity recognition on mobile devices. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, Heidelberg, Germany.","DOI":"10.1145\/2968219.2971461"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lane, N.D., and Georgiev, P. (2015, January 12\u201313). Can deep learning revolutionize mobile sensing?. Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, Snata Fe, NM, USA.","DOI":"10.1145\/2699343.2699349"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bhattacharya, S., and Lane, N.D. (2016, January 14\u201318). From smart to deep: Robust activity recognition on smartwatches using deep learning. Proceedings of the 2016 IEEE International Conference on Pervasive Computing and Communication Workshops, Sydney, Australia.","DOI":"10.1109\/PERCOMW.2016.7457169"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ronao, C.A., and Cho, S.B. (2015, January 9\u201312). Deep convolutional neural networks for human activity recognition with smartphone sensors. Proceedings of the Conference on Neural Information Processing, Istanbul, Turkey.","DOI":"10.1007\/978-3-319-26561-2_6"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.eswa.2016.04.032","article-title":"Human activity recognition with smartphone sensors using deep learning neural networks","volume":"59","author":"Ronao","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.procs.2018.04.025","article-title":"Classification of Recurrence Plots\u2019 Distance Matrices with a Convolutional Neural Network for Activity Recognition","volume":"130","author":"Uddin","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, R., and Li, C. (2015). Motion sequence recognition with multi-sensors using deep convolutional neural network. Intelligent Data Analysis and Applications, Springer.","DOI":"10.1007\/978-3-319-21206-7_2"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jiang, W., and Yin, Z. (2015, January 26\u201327). Human activity recognition using wearable sensors by deep convolutional neural networks. Proceedings of the 23rd ACM international conference on Multimedia, Kyoto, Japan.","DOI":"10.1145\/2733373.2806333"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Dehzangi, O., Taherisadr, M., and ChangalVala, R. (2017). IMU-Based Gait Recognition Using Convolutional Neural Networks and Multi-Sensor Fusion. Sensors, 17.","DOI":"10.3390\/s17122735"},{"key":"ref_24","unstructured":"Alsheikh, M.A., Selim, A., Niyato, D., Doyle, L., Lin, S., and Tan, H.P. (2016, January 12). Deep Activity Recognition Models with Triaxial Accelerometers. Proceedings of the Artificial Intelligence Applied to Assistive Technologies and Smart Environments, Phoenix, AZ, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ravi, D., Wong, C., Lo, B., and Yang, G.Z. (2016, January 14\u201317). Deep learning for human activity recognition: A resource efficient implementation on low-power devices. Proceedings of the IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks, San Francisco, CA, USA.","DOI":"10.1109\/BSN.2016.7516235"},{"key":"ref_26","first-page":"576","article-title":"Deconvolutive short-time Fourier transform spectrogram","volume":"16","author":"Lu","year":"2019","journal-title":"IEEE Signal Process Lett."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ha, S., Yun, J.M., and Choi, S. (2015, January 9\u201312). Multi-modal convolutional neural networks for activity recognition. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China.","DOI":"10.1109\/SMC.2015.525"},{"key":"ref_28","unstructured":"Yang, J., Nguyen, M.N., San, P.P., Li, X., and Krishnaswamy, S. (August, January 28). Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Baldominos, A., Saez, Y., and Isasi, P. (2018). Evolutionary Design of Convolutional Neural Networks for Human Activity Recognition in Sensor-Rich Environments. Sensors, 18.","DOI":"10.3390\/s18041288"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ord\u00f3\u00f1ez, F.J., and Roggen, D. (2016). Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16.","DOI":"10.3390\/s16010115"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cho, H., and Yoon, S.M. (2018). Divide and Conquer-Based 1D CNN Human Activity Recognition Using Test Data Sharpening. Sensors, 18.","DOI":"10.3390\/s18041055"},{"key":"ref_32","unstructured":"Hammerla, N.Y., Halloran, S., and Ploetz, T. (2016, January 9\u201316). Deep, convolutional, and recurrent models for human activity recognition using wearables. Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Murad, A., and Pyun, J.Y. (2017). Deep recurrent neural networks for human activity recognition. Sensors, 17.","DOI":"10.3390\/s17112556"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Saez, Y., Baldominos, A., and Isasi, P. (2017). A comparison study of classifier algorithms for cross-person physical activity recognition. Sensors, 17.","DOI":"10.3390\/s17010066"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, F., Shirahama, K., Nisar, M.A., K\u00f6ping, L., and Grzegorzek, M. (2018). Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors. Sensors, 18.","DOI":"10.3390\/s18020679"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Vavoulas, G., Chatzaki, C., Malliotakis, T., Pediaditis, M., and Tsiknakis, M. (2016, January 21\u201322). The MobiAct Dataset: Recognition of Activities of Daily Living using Smartphones. Proceedings of the 2nd International Conference on Information and Communication Technologies for Ageing Well and e-Health, Rome, Italy.","DOI":"10.5220\/0005792401430151"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Leutheuser, H., Schuldhaus, D., and Eskofier, B.M. (2013). Hierarchical, multi-sensor based classification of daily life activities: Comparison with state-of-the-art algorithms using a benchmark dataset. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0075196"},{"key":"ref_38","unstructured":"Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2013, January 24\u201326). A Public Domain Dataset for Human Activity Recognition Using Smartphones. Proceedings of the 21th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.procs.2014.07.009","article-title":"A study on human activity recognition using accelerometer data from smartphones","volume":"34","author":"Bayat","year":"2014","journal-title":"Procedia Comput. Sci."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1109\/JBHI.2016.2633287","article-title":"A deep learning approach to on-node sensor data analytics for mobile or wearable devices","volume":"21","author":"Ravi","year":"2017","journal-title":"IEEE J. Biomed. Health Inf."},{"key":"ref_41","unstructured":"Penatti, O.A., and Santos, M.F. (arXiv, 2017). Human activity recognition from mobile inertial sensors using recurrence plots, arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zheng, X., Wang, M., and Ordieres-Mer\u00e9, J. (2018). Comparison of Data Preprocessing Approaches for Applying Deep Learning to Human Activity Recognition in the Context of Industry 4.0. Sensors, 18.","DOI":"10.3390\/s18072146"},{"key":"ref_43","unstructured":"Krizhevsky, A., and Hinton, G. (2009). Learning Multiple Layers of Features from Tiny Images, University of Toronto. Technical Report."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_47","unstructured":"Lin, M., Chen, Q., and Yan, S. (arXiv, 2013). Network in network, arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Ahmer, M., Shah, M.Z.A., Shah, S.M.Z.S., Shah, S.M.S., Chowdhry, B.S., Shah, A., and Bhatti, K.H. (2017). Using Non-Linear Support Vector Machines for Detection of Activities of Daily Living. Indian J. Sci. Technol., 10.","DOI":"10.17485\/ijst\/2017\/v10i36\/119182"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Bao, L., and Intille, S.S. (2004, January 21\u201323). Activity recognition from user-annotated acceleration data. Proceedings of the International Conference on Pervasive Computing, Vienna, Austria.","DOI":"10.1007\/978-3-540-24646-6_1"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"871","DOI":"10.1109\/TBME.2008.2006190","article-title":"A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data","volume":"56","author":"Preece","year":"2009","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"5262","DOI":"10.1109\/ACCESS.2017.2684913","article-title":"Improving activity recognition accuracy in ambient-assisted living systems by automated feature engineering","volume":"5","author":"Zdravevski","year":"2017","journal-title":"IEEE Access"},{"key":"ref_52","first-page":"1295","article-title":"Energy Efficient Smartphone-Based Activity Recognition using Fixed-Point Arithmetic","volume":"19","author":"Anguita","year":"2013","journal-title":"J. Univ. Comput. Sci."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.jpdc.2017.05.007","article-title":"GCHAR: An efficient Group-based Context\u2014Aware human activity recognition on smartphone","volume":"118","author":"Cao","year":"2018","journal-title":"J. Parallel Distrib. Comput."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/11\/3910\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:29:29Z","timestamp":1760196569000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/11\/3910"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,11,13]]},"references-count":53,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2018,11]]}},"alternative-id":["s18113910"],"URL":"https:\/\/doi.org\/10.3390\/s18113910","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,11,13]]}}}