{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T23:19:50Z","timestamp":1777936790330,"version":"3.51.4"},"reference-count":55,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,12,29]],"date-time":"2022-12-29T00:00:00Z","timestamp":1672272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Trade, Industry &amp; Energy of the Republic of Korea","award":["B0080119002854"],"award-info":[{"award-number":["B0080119002854"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Human activity recognition (HAR) has emerged as a significant area of research due to its numerous possible applications, including ambient assisted living, healthcare, abnormal behaviour detection, etc. Recently, HAR using WiFi channel state information (CSI) has become a predominant and unique approach in indoor environments compared to others (i.e., sensor and vision) due to its privacy-preserving qualities, thereby eliminating the need to carry additional devices and providing flexibility of capture motions in both line-of-sight (LOS) and non-line-of-sight (NLOS) settings. Existing deep learning (DL)-based HAR approaches usually extract either temporal or spatial features and lack adequate means to integrate and utilize the two simultaneously, making it challenging to recognize different activities accurately. Motivated by this, we propose a novel DL-based model named spatio-temporal convolution with nested long short-term memory (STC-NLSTMNet), with the ability to extract spatial and temporal features concurrently and automatically recognize human activity with very high accuracy. The proposed STC-NLSTMNet model is mainly comprised of depthwise separable convolution (DS-Conv) blocks, feature attention module (FAM) and NLSTM. The DS-Conv blocks extract the spatial features from the CSI signal and add feature attention modules (FAM) to draw attention to the most essential features. These robust features are fed into NLSTM as inputs to explore the hidden intrinsic temporal features in CSI signals. The proposed STC-NLSTMNet model is evaluated using two publicly available datasets: Multi-environment and StanWiFi. The experimental results revealed that the STC-NLSTMNet model achieved activity recognition accuracies of 98.20% and 99.88% on Multi-environment and StanWiFi datasets, respectively. Its activity recognition performance is also compared with other existing approaches and our proposed STC-NLSTMNet model significantly improves the activity recognition accuracies by 4% and 1.88%, respectively, compared to the best existing method.<\/jats:p>","DOI":"10.3390\/s23010356","type":"journal-article","created":{"date-parts":[[2022,12,30]],"date-time":"2022-12-30T03:19:46Z","timestamp":1672370386000},"page":"356","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["STC-NLSTMNet: An Improved Human Activity Recognition Method Using Convolutional Neural Network with NLSTM from WiFi CSI"],"prefix":"10.3390","volume":"23","author":[{"given":"Md Shafiqul","family":"Islam","sequence":"first","affiliation":[{"name":"Department of Electronics Engineering, Kwangwoon University, Seoul 01897, Republic of Korea"}]},{"given":"Mir Kanon Ara","family":"Jannat","sequence":"additional","affiliation":[{"name":"Department of Electronics Engineering, Kwangwoon University, Seoul 01897, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7423-2096","authenticated-orcid":false,"given":"Mohammad Nahid","family":"Hossain","sequence":"additional","affiliation":[{"name":"Department of Electronics Engineering, Kwangwoon University, Seoul 01897, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6517-2044","authenticated-orcid":false,"given":"Woo-Su","family":"Kim","sequence":"additional","affiliation":[{"name":"Graduate School of Knowledge-Based Technology and Energy, Tech University of Korea, Siheung 15073, Republic of Korea"}]},{"given":"Soo-Wook","family":"Lee","sequence":"additional","affiliation":[{"name":"Kwangwoon Academy, Kwangwoon University, Seoul 01897, Republic of Korea"}]},{"given":"Sung-Hyun","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Electronics Engineering, Kwangwoon University, Seoul 01897, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kalimuthu, S., Perumal, T., Yaakob, R., Marlisah, E., and Babangida, L. (2021, January 4\u20135). Human Activity Recognition based on smart home environment and their applications, challenges. Proceedings of the 2021 International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), Greater Noida, India.","DOI":"10.1109\/ICACITE51222.2021.9404753"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ahad, M.A. (2018, January 8\u201312). Activity recognition for health-care and related works. Proceedings of the 2018 ACM International Joint Conference and International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, Singapore.","DOI":"10.1145\/3267305.3277813"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1038\/s41597-022-01573-2","article-title":"OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors","volume":"9","author":"Bocus","year":"2022","journal-title":"Sci. Data"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1145\/1922649.1922653","article-title":"Human activity analysis: A review","volume":"43","author":"Aggarwal","year":"2011","journal-title":"ACM Comput. Surv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Uddin, M.H., Ara, J.M., Rahman, M.H., and Yang, S.H. (2021, January 17\u201319). A Study of Real-Time Physical Activity Recognition from Motion Sensors via Smartphone Using Deep Neural Network. Proceedings of the 2021 5th International Conference on Electrical Information and Communication Technology (EICT), Khulna, Bangladesh.","DOI":"10.1109\/EICT54103.2021.9733607"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, X., He, Y., and Jing, X. (2019). A survey of deep learning-based human activity recognition in radar. Remote Sens., 11.","DOI":"10.3390\/rs11091068"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shafiqul, I.M., Jannat, M.K., Kim, J.W., Lee, S.W., and Yang, S.H. (2022). HHI-AttentionNet: An Enhanced Human-Human Interaction Recognition Method Based on a Lightweight Deep Learning Model with Attention Network from CSI. Sensors, 22.","DOI":"10.3390\/s22166018"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"166624","DOI":"10.1109\/ACCESS.2021.3134794","article-title":"CSI-IANet: An Inception Attention Network for Human-Human Interaction Recognition Based on CSI Signal","volume":"9","author":"Kabir","year":"2021","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Su, J., Liao, Z., Sheng, Z., Liu, A.X., Singh, D., and Lee, H.N. (2022). Human activity recognition using self-powered sensors based on multilayer bi-directional long short-term memory networks. IEEE Sens. J., Early Access.","DOI":"10.1109\/JSEN.2022.3195274"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"153287","DOI":"10.1109\/ACCESS.2019.2948102","article-title":"Wi-motion: A robust human activity recognition using WiFi signals","volume":"7","author":"Li","year":"2019","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Youssef, M., Mah, M., and Agrawala, A. (2007, January 25\u201329). Challenges: Device-free passive localization for wireless environments. Proceedings of the 13th Annual ACM international Conference on Mobile Computing and Networking, New Orleans, LA, USA.","DOI":"10.1145\/1287853.1287880"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"10639","DOI":"10.1109\/JIOT.2019.2940368","article-title":"Recurrent neural networks for accurate RSSI indoor localization","volume":"6","author":"Hoang","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"69","DOI":"10.5121\/ijwmn.2011.3306","article-title":"Why the Accuracy of the Received Signal Strengths as a Positioning Technique was not accurate?","volume":"3","author":"Mohamed","year":"2011","journal-title":"Int. J. Wirel. Mob. Netw."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2907","DOI":"10.1109\/TMC.2016.2517630","article-title":"We can hear you with Wi-Fi!","volume":"15","author":"Wang","year":"2016","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ren, Y., Wang, Z., Wang, Y., Tan, S., Chen, Y., and Yang, J. (2021, January 15\u201317). 3D Human Pose Estimation Using WiFi Signals. Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, Coimbra, Portugal.","DOI":"10.1145\/3485730.3492871"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"10483","DOI":"10.1109\/JIOT.2020.2997053","article-title":"WiReader: Adaptive air handwriting recognition based on commercial WiFi signal","volume":"7","author":"Guo","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Han, C., Wu, K., Wang, Y., and Ni, L.M. (May, January 27). WiFall: Device-free fall detection by wireless networks. Proceedings of the IEEE INFOCOM 2014\u2014IEEE Conference on Computer Communications, Toronto, ON, Canada.","DOI":"10.1109\/INFOCOM.2014.6847948"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Fard Moshiri, P., Shahbazian, R., Nabati, M., and Ghorashi, S.A. (2021). A CSI-Based Human Activity Recognition Using Deep Learning. Sensors, 21.","DOI":"10.3390\/s21217225"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2714","DOI":"10.1109\/TMC.2018.2878233","article-title":"WiFi CSI based passive human activity recognition using attention-based BLSTM","volume":"18","author":"Chen","year":"2018","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"107066","DOI":"10.1016\/j.asoc.2020.107066","article-title":"Device-free single-user activity recognition using diversified deep ensemble learning","volume":"102","author":"Cui","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1353","DOI":"10.1109\/TMC.2017.2761744","article-title":"HARKE: Human Activity Recognition from Kinetic Energy Harvesting Data in Wearable Devices","volume":"17","author":"Khalifa","year":"2018","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zubair, M., Song, K., and Yoon, C. (2016, January 26\u201328). Human activity recognition using wearable accelerometer sensors. Proceedings of the 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), Seoul, Republic of Korea.","DOI":"10.1109\/ICCE-Asia.2016.7804737"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ugulino, W., Cardador, D., Vega, K., Velloso, E., Milidi\u00fa, R., and Fuks, H. (2012). Wearable computing: Accelerometers\u2019 data classification of body postures and movements. Brazilian Symposium on Artificial Intelligence, Springer.","DOI":"10.1007\/978-3-642-34459-6_6"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Badawi, A.A., Al-Kabbany, A., and Shaban, H. (2018, January 3\u20136). Multimodal human activity recognition from wearable inertial sensors using machine learning. Proceedings of the 2018 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES), Sarawak, Malaysia.","DOI":"10.1109\/IECBES.2018.8626737"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Vemulapalli, R., Arrate, F., and Chellappa, R. (2014, January 23\u201328). Human action recognition by representing 3d skeletons as points in a lie group. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.82"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.sigpro.2017.08.016","article-title":"Skeleton embedded motion body partition for human action recognition using depth sequences","volume":"143","author":"Ji","year":"2018","journal-title":"Signal Process."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Xia, L., Chen, C.C., and Aggarwal, J.K. (2012, January 16\u201321). View invariant human action recognition using histograms of 3D joints. Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.","DOI":"10.1109\/CVPRW.2012.6239233"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"62984","DOI":"10.1109\/ACCESS.2021.3074088","article-title":"Unsupervised Doppler Radar Based Activity Recognition for e-Healthcare","volume":"9","author":"Karayaneva","year":"2021","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1002311","DOI":"10.1109\/TGRS.2022.3162833","article-title":"Radar-Based Human Activity Recognition Combining Range\u2013Time\u2013Doppler Maps and Range-Distributed-Convolutional Neural Networks","volume":"60","author":"Kim","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Franceschini, S., Ambrosanio, M., Vitale, S., Baselice, F., Gifuni, A., Grassini, G., and Pascazio, V. (2020, January 21\u201325). Hand gesture recognition via radar sensors and convolutional neural networks. Proceedings of the 2020 IEEE Radar Conference, Florence, Italy.","DOI":"10.1109\/RadarConf2043947.2020.9266565"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ghosh, A., Sanyal, A., Chakraborty, A., Sharma, P.K., Saha, M., Nandi, S., and Saha, S. (2017, January 4\u20138). On automatizing recognition of multiple human activities using ultrasonic sensor grid. Proceedings of the 2017 9th International Conference on Communication Systems and Networks (COMSNETS), Bengaluru, India.","DOI":"10.1109\/COMSNETS.2017.7945440"},{"key":"ref_32","unstructured":"Hori, T., and Nishida, Y. (2005, January 25\u201328). Ultrasonic sensors for the elderly and caregivers in a nursing home. Proceedings of the ICEIS, Miami, FL, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1109\/JSEN.2015.2454000","article-title":"Fusion of different height pyroelectric infrared sensors for person identification","volume":"16","author":"Xiong","year":"2016","journal-title":"IEEE Sens. J."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Sigg, S., Blanke, U., and Tr\u00f6ster, G. (2014, January 24\u201328). The telepathic phone: Frictionless activity recognition from WiFi-RSSI. Proceedings of the 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom), Budapest, Hungary.","DOI":"10.1109\/PerCom.2014.6813955"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1109\/JIOT.2015.2511805","article-title":"PAWS: Passive Human Activity Recognition Based on WiFi Ambient Signals","volume":"3","author":"Gu","year":"2016","journal-title":"IEEE Internet Things J."},{"key":"ref_36","unstructured":"Gu, Y., Quan, L., and Ren, F. (2014, January 28\u201330). WiFi-assisted human activity recognition. Proceedings of the 2014 IEEE Asia Pacific Conference on Wireless and Mobile, Bali, Indonesia."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Sigg, S., Shi, S., Buesching, F., Ji, Y., and Wolf, L. (2013, January 2\u20134). Leveraging RF-channel fluctuation for activity recognition: Active and passive systems, continuous and RSSI-based signal features. Proceedings of the International Conference on Advances in Mobile Computing & Multimedia, Vienna, Austria.","DOI":"10.1145\/2536853.2536873"},{"key":"ref_38","unstructured":"Wang, F., Panev, S., Dai, Z., Han, J., and Huang, D. (2019). Can WiFi estimate person pose?. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"6654752","DOI":"10.1155\/2021\/6654752","article-title":"A framework for human activity recognition based on WiFi CSI signal enhancement","volume":"2021","author":"Yang","year":"2021","journal-title":"Int. J. Antennas Propag."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Damodaran, N., and Sch\u00e4fer, J. (2019, January 19\u201323). Device free human activity recognition using WiFi channel state information. Proceedings of the 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation, Leicester, UK.","DOI":"10.1109\/SmartWorld-UIC-ATC-SCALCOM-IOP-SCI.2019.00205"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1109\/MCOM.2017.1700082","article-title":"A survey on behavior recognition using WiFi channel state information","volume":"55","author":"Yousefi","year":"2017","journal-title":"IEEE Commun. Mag."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.neunet.2021.11.011","article-title":"CSITime: Privacy-preserving human activity recognition using WiFi channel state information","volume":"146","author":"Yadav","year":"2022","journal-title":"Neural Netw."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1118","DOI":"10.1109\/JSAC.2017.2679658","article-title":"Device-Free Human Activity Recognition Using Commercial WiFi Devices","volume":"35","author":"Wang","year":"2017","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1109\/JSEN.2019.2938245","article-title":"WiAct: A passive WiFi-based human activity recognition system","volume":"20","author":"Yan","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Muaaz, M., Chelli, A., and P\u00e4tzold, M. (2021). Wi-Fi-based human activity recognition using convolutional neural network. Innovative and Intelligent Technology-Based Services for Smart Environments\u2013Smart Sensing and Artificial Intelligence, CRC Press.","DOI":"10.1201\/9781003181545-11"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"4628","DOI":"10.1109\/JIOT.2020.3026732","article-title":"Data augmentation and dense-LSTM for human activity recognition using WiFi signal","volume":"8","author":"Zhang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"012139","DOI":"10.1088\/1742-6596\/1883\/1\/012139","article-title":"LSTM-CNN network for human activity recognition using WiFi CSI data","volume":"1883","author":"Shang","year":"2021","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"11290","DOI":"10.1109\/JIOT.2020.2997237","article-title":"Differential channel-state-information-based human activity recognition in IoT networks","volume":"7","author":"Khan","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Alsaify, B.A., Almazari, M.M., Alazrai, R., Alouneh, S., and Daoud, M.I. (2022). A CSI-Based Multi-Environment Human Activity Recognition Framework. Appl. Sci., 12.","DOI":"10.3390\/app12020930"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"106534","DOI":"10.1016\/j.dib.2020.106534","article-title":"A dataset for Wi-Fi-based human activity recognition in line-of-sight and non-line-of-sight indoor environments","volume":"33","author":"Baha","year":"2020","journal-title":"Data Brief"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_52","unstructured":"Moniz, J.R.A., and David, K. (2017, January 15\u201317). Nested LSTMs. Proceedings of the Asian Conference on Machine Learning, PMLR, Seoul, Republic of Korea."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Islam, M., Shafiqul, K.T., and Sung-Hyun, Y. (2022). Epileptic-Net: An Improved Epileptic Seizure Detection System Using Dense Convolutional Block with Attention Network from EEG. Sensors, 22.","DOI":"10.3390\/s22030728"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Salehinejad, H., and Valaee, S. (2022, January 23\u201327). LiteHAR: Lightweight Human Activity Recognition from WIFI Signals with Random Convolution Kernels. Proceedings of the ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.","DOI":"10.1109\/ICASSP43922.2022.9746803"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Alsaify, B.A., Almazari, M.M., Alazrai, R., and Daoud, M.I. (2021, January 24\u201326). Exploiting Wi-Fi Signals for Human Activity Recognition. Proceedings of the 2021 12th International Conference on Information and Communication Systems (ICICS), Valencia, Spain.","DOI":"10.1109\/ICICS52457.2021.9464613"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/1\/356\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:55:13Z","timestamp":1760147713000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/1\/356"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,29]]},"references-count":55,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["s23010356"],"URL":"https:\/\/doi.org\/10.3390\/s23010356","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,29]]}}}