{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T13:20:47Z","timestamp":1784294447369,"version":"3.55.0"},"reference-count":61,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T00:00:00Z","timestamp":1709078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>The Recurrent Neural Networks (RNNs) are an essential class of supervised learning algorithms. Complex tasks like speech recognition, machine translation, sentiment classification, weather prediction, etc., are now performed by well-trained RNNs. Local or cloud-based GPU machines are used to train them. However, inference is now shifting to miniature, mobile, IoT devices and even micro-controllers. Due to their colossal memory and computing requirements, mapping RNNs directly onto resource-constrained platforms is arcane and challenging. The efficacy of edge-intelligent RNNs (EI-RNNs) must satisfy both performance and memory-fitting requirements at the same time without compromising one for the other. This study\u2019s aim was to provide an empirical evaluation and optimization of historic as well as recent RNN architectures for high-performance and low-memory footprint goals. We focused on Human Activity Recognition (HAR) tasks based on wearable sensor data for embedded healthcare applications. We evaluated and optimized six different recurrent units, namely Vanilla RNNs, Long Short-Term Memory (LSTM) units, Gated Recurrent Units (GRUs), Fast Gated Recurrent Neural Networks (FGRNNs), Fast Recurrent Neural Networks (FRNNs), and Unitary Gated Recurrent Neural Networks (UGRNNs) on eight publicly available time-series HAR datasets. We used the hold-out and cross-validation protocols for training the RNNs. We used low-rank parameterization, iterative hard thresholding, and spare retraining compression for RNNs. We found that efficient training (i.e., dataset handling and preprocessing procedures, hyperparameter tuning, and so on, and suitable compression methods (like low-rank parameterization and iterative pruning) are critical in optimizing RNNs for performance and memory efficiency. We implemented the inference of the optimized models on Raspberry Pi.<\/jats:p>","DOI":"10.3390\/a17030104","type":"journal-article","created":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T03:37:57Z","timestamp":1709091477000},"page":"104","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["A Systematic Evaluation of Recurrent Neural Network Models for Edge Intelligence and Human Activity Recognition Applications"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7475-9322","authenticated-orcid":false,"given":"Varsha S.","family":"Lalapura","sequence":"first","affiliation":[{"name":"Department of Electronics and Communication Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Bengaluru 560035, Karnataka, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1987-1071","authenticated-orcid":false,"given":"Veerender Reddy","family":"Bhimavarapu","sequence":"additional","affiliation":[{"name":"Department of Electronics and Communication Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Bengaluru 560035, Karnataka, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6736-2573","authenticated-orcid":false,"given":"J.","family":"Amudha","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Amrita School of Computing, Amrita Vishwa Vidyapeetham, Bengaluru 560035, Karnataka, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-4497-6564","authenticated-orcid":false,"given":"Hariram Selvamurugan","family":"Satheesh","sequence":"additional","affiliation":[{"name":"ABB Global Industries and Services Private Limited, Bengaluru 560048, Karnataka, India"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kolen, J., and Kremer, S. (2010). A Field Guide to Dynamical Recurrent Network, IEEE.","DOI":"10.1109\/9780470544037"},{"key":"ref_2","unstructured":"Martens, J., and Sutskever, I. (July, January 28). Learning recurrent neural networks with hessian-free optimization. Proceedings of the 28th International Conference on Machine Learning, Washington, DC, USA."},{"key":"ref_3","unstructured":"Collins, J., Sohl-Dickstein, J., and Sussillo, D. (2016). Capacity and trainability in recurrent neural networks. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3448974","article-title":"Recurrent neural networks for edge intelligence: A survey","volume":"54","author":"Lalapura","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Amudha, J., Thakur, M.S., Shrivastava, A., Gupta, S., Gupta, D., and Sharma, K. (2022, January 19\u201320). Wild OCR: Deep Learning Architecture for Text Recognition in Images. Proceedings of the International Conference on Computing and Communication Networks, Manchester, UK.","DOI":"10.1007\/978-981-19-0604-6_46"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Vanishree, K., George, A., Gunisetty, S., Subramanian, S., Kashyap, S., and Purnaprajna, M. (2020, January 6\u20137). CoIn: Accelerated CNN Co-Inference through data partitioning on heterogeneous devices. Proceedings of the 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India.","DOI":"10.1109\/ICACCS48705.2020.9074444"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/j.procs.2020.04.065","article-title":"Towards identifying most important leads for ECG classification. A Data driven approach employing Deep Learning","volume":"171","author":"Sujadevi","year":"2020","journal-title":"Procedia Comput. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"e16","DOI":"10.23915\/distill.00016","article-title":"Visualizing memorization in RNNs","volume":"4","author":"Madsen","year":"2019","journal-title":"Distill"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning long-term dependencies with gradient descent is difficult","volume":"5","author":"Bengio","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Gupta, D., Khanna, A., Bhattacharyya, S., Hassanien, A.E., Anand, S., and Jaiswal, A. (2023, January 17\u201318). Ensemble Deep Learning Models for Vehicle Classification in Motorized Traffic Analysis. Proceedings of the International Conference on Innovative Computing and Communications, Delhi, India.","DOI":"10.1007\/978-981-19-2821-5"},{"key":"ref_11","unstructured":"Gupta, D., Khanna, A., Bhattacharyya, S., Hassanien, A.E., Anand, S., and Jaiswal, A. (2023, January 17\u201318). Data-Driven Volatile Cryptocurrency Price Forecasting via Variational Mode Decomposition and BiLSTM. Proceedings of the International Conference on Innovative Computing and Communications, Delhi, India."},{"key":"ref_12","unstructured":"Pascanu, R., Mikolov, T., and Bengio, Y. (2013, January 17\u201319). On the difficulty of training recurrent neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Atlanta, GA, USA."},{"key":"ref_13","unstructured":"Lin, J. (2021). Efficient Algorithms and Systems for Tiny Deep Learning. [Ph.D. Thesis, Massachusetts Institute of Technology]."},{"key":"ref_14","unstructured":"Han, S., Kang, J., Mao, H., Hu, Y., Li, X., Li, Y., Xie, D., Luo, H., Yao, S., and Wang, Y. (2017, January 22\u201324). Ese: Efficient speech recognition engine with sparse lstm on fpga. Proceedings of the 2017 ACM\/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Graves, A. (2013). Generating sequences with recurrent neural networks. arXiv.","DOI":"10.1007\/978-3-642-24797-2_3"},{"key":"ref_17","unstructured":"Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Bahdanau, D., and Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv.","DOI":"10.3115\/v1\/W14-4012"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_20","unstructured":"Arjovsky, M., Shah, A., and Bengio, Y. (2016, January 20\u201322). Unitary evolution recurrent neural networks. Proceedings of the International Conference on Machine Learning, PMLR, New York, NY, USA."},{"key":"ref_21","first-page":"800","article-title":"TensorFlow lite micro: Embedded machine learning for tinyml systems","volume":"3","author":"David","year":"2021","journal-title":"Proc. Mach. Learn. Syst."},{"key":"ref_22","unstructured":"(2018). Microsoft-v2019, Microsoft Corporation. ELL: Embedded Learning Library."},{"key":"ref_23","first-page":"517","article-title":"Micronets: Neural network architectures for deploying tinyml applications on commodity microcontrollers","volume":"3","author":"Banbury","year":"2021","journal-title":"Proc. Mach. Learn. Syst."},{"key":"ref_24","first-page":"1","article-title":"A survey on deep learning for human activity recognition","volume":"54","author":"Gu","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_25","unstructured":"Jordao, A., Nazare, A.C., Sena, J., and Schwartz, W.R. (2018). Human activity recognition based on wearable sensor data: A standardization of the state-of-the-art. arXiv."},{"key":"ref_26","unstructured":"Demrozi, F., Turetta, C., and Pravadelli, G. (2021). B-HAR: An open-source baseline framework for in depth study of human activity recognition datasets and workflows. arXiv."},{"key":"ref_27","unstructured":"Olah, C. (2022, May 22). Understanding LSTM Networks. Available online: http:\/\/colah.github.io\/posts\/2015-08-Understanding-LSTMs\/."},{"key":"ref_28","unstructured":"Kusupati, A., Singh, M., Bhatia, K., Kumar, A., Jain, P., and Varma, M. (2018). Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_29","unstructured":"Han, S., Mao, H., and Dally, W.J. (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1109\/72.572092","article-title":"An iterative pruning algorithm for feedforward neural networks","volume":"8","author":"Castellano","year":"1997","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1109\/72.248452","article-title":"Pruning algorithms-a survey","volume":"4","author":"Reed","year":"1993","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_32","unstructured":"Guo, Y., Yao, A., and Chen, Y. (2016). Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Gao, C., Neil, D., Ceolini, E., Liu, S.C., and Delbruck, T. (2018, January 25\u201327). DeltaRNN: A power-efficient recurrent neural network accelerator. Proceedings of the 2018 ACM\/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA.","DOI":"10.1145\/3174243.3174261"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Yao, S., Zhao, Y., Zhang, A., Su, L., and Abdelzaher, T. (2017, January 6\u20138). Deepiot: Compressing deep neural network structures for sensing systems with a compressor-critic framework. Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems, Delft, The Netherlands.","DOI":"10.1145\/3131672.3131675"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, S., Li, Z., Ding, C., Yuan, B., Qiu, Q., Wang, Y., and Liang, Y. (2018, January 6\u20138). C-LSTM: Enabling efficient LSTM using structured compression techniques on FPGAs. Proceedings of the 2018 ACM\/SIGDA International Symposium on Field-Programmable Gate Arrays, Delft, The Netherlands.","DOI":"10.1145\/3174243.3174253"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3005348","article-title":"Structured pruning of deep convolutional neural networks","volume":"13","author":"Anwar","year":"2017","journal-title":"ACM J. Emerg. Technol. Comput. Syst. (JETC)"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/j.neunet.2019.11.018","article-title":"Structured pruning of recurrent neural networks through neuron selection","volume":"123","author":"Wen","year":"2020","journal-title":"Neural Netw."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Thakker, U., Beu, J., Gope, D., Dasika, G., and Mattina, M. (2019, January 17). Run-time efficient RNN compression for inference on edge devices. Proceedings of the 2019 2nd Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2), Washington, DC, USA.","DOI":"10.1109\/EMC249363.2019.00013"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2057","DOI":"10.1109\/TNNLS.2021.3105818","article-title":"DRRNets: Dynamic Recurrent Routing via Low-Rank Regularization in Recurrent Neural Networks","volume":"34","author":"Shan","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Li, J., Kumar, K., and Gong, Y. (2017). Extended Low-Rank Plus Diagonal Adaptation for Deep and Recurrent Neural Networks, IEEE Press.","DOI":"10.1109\/ICASSP.2017.7953116"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Lu, Z., Sindhwani, V., and Sainath, T.N. (2016, January 20\u201325). Learning compact recurrent neural networks. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.","DOI":"10.1109\/ICASSP.2016.7472821"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Prabhavalkar, R., Alsharif, O., Bruguier, A., and McGraw, L. (2016, January 20\u201325). On the compression of recurrent neural networks with an application to LVCSR acoustic modeling for embedded speech recognition. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.","DOI":"10.1109\/ICASSP.2016.7472823"},{"key":"ref_43","unstructured":"Vanhoucke, V., Senior, A., and Mao, M.Z. (2024, February 10). Improving the Speed of Neural Networks on CPUs. Available online: https:\/\/research.google\/pubs\/improving-the-speed-of-neural-networks-on-cpus\/."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Ramakrishnan, R., Dev, A.K., Darshik, A., Chinchwadkar, R., and Purnaprajna, M. (2021, January 20\u201324). Demystifying Compression Techniques in CNNs: CPU, GPU and FPGA cross-platform analysis. Proceedings of the 2021 34th International Conference on VLSI Design and 2021 20th International Conference on Embedded Systems (VLSID), Virtual.","DOI":"10.1109\/VLSID51830.2021.00046"},{"key":"ref_45","unstructured":"Warden, P., and Situnayake, D. (2024, February 10). TinyML. Available online: https:\/\/www.oreilly.com\/library\/view\/tinyml\/9781492052036\/."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"4403","DOI":"10.1109\/JIOT.2020.2976702","article-title":"FANN-on-MCU: An open-source toolkit for energy-efficient neural network inference at the edge of the Internet of Things","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_47","unstructured":"Biewald, L. (2022, June 30). Experiment Tracking with Weights and Biases. Available online: https:\/\/www.wandb.com."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2499621","article-title":"A tutorial on human activity recognition using body-worn inertial sensors","volume":"46","author":"Bulling","year":"2014","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Altun, K., and Barshan, B. (2010, January 22). Human activity recognition using inertial\/magnetic sensor units. Proceedings of the International Workshop on Human Behavior Understanding, Istanbul, Turkey.","DOI":"10.1007\/978-3-642-14715-9_5"},{"key":"ref_50","unstructured":"Anguita, D., Ghio, A., Oneto, L., Parra Perez, X., and Reyes Ortiz, J.L. (2013, January 24\u201326). A public domain dataset for human activity recognition using smartphones. Proceedings of the 21th International European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"2033","DOI":"10.1016\/j.patrec.2012.12.014","article-title":"The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition","volume":"34","author":"Chavarriaga","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Reiss, A., and Stricker, D. (2012, January 6\u20139). Creating and benchmarking a new dataset for physical activity monitoring. Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments, Crete, Greece.","DOI":"10.1145\/2413097.2413148"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Banos, O., Garcia, R., Holgado-Terriza, J.A., Damas, M., Pomares, H., Rojas, I., Saez, A., and Villalonga, C. (2014, January 2\u20135). mHealthDroid: A novel framework for agile development of mobile health applications. Proceedings of the International Workshop on Ambient Assisted Living, Belfast, UK.","DOI":"10.1007\/978-3-319-13105-4_14"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Zhang, M., and Sawchuk, A.A. (2012, January 5\u20138). USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Pittsburgh, PA, USA.","DOI":"10.1145\/2370216.2370438"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1109\/MRA.2015.2448279","article-title":"Wearable inertial sensors: Applications, challenges, and public test benches","volume":"22","author":"Bruno","year":"2015","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Lockhart, J.W., Weiss, G.M., Xue, J.C., Gallagher, S.T., Grosner, A.B., and Pulickal, T.T. (2011, January 21). Design considerations for the WISDM smart phone-based sensor mining architecture. Proceedings of the Fifth International Workshop on Knowledge Discovery from Sensor Data, San Diego, CA, USA.","DOI":"10.1145\/2003653.2003656"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1145\/2347736.2347755","article-title":"A few useful things to know about machine learning","volume":"55","author":"Domingos","year":"2012","journal-title":"Commun. ACM"},{"key":"ref_58","unstructured":"Dennis, D.K., Gaurkar, Y., Gopinath, S., Goyal, S., Gupta, C., Jain, M., Jaiswal, S., Kumar, A., Kusupati, A., and Lovett, C. (2022, June 30). EdgeML: Machine Learning for Resource-Constrained Edge Devices. Available online: https:\/\/github.com\/Microsoft\/EdgeML."},{"key":"ref_59","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http:\/\/www.deeplearningbook.org."},{"key":"ref_60","unstructured":"LeCun, Y.A., Bottou, L., Orr, G.B., and M\u00fcller, K.R. (2012). Neural Networks: Tricks of the Trade, Springer."},{"key":"ref_61","unstructured":"Zaremba, W., Sutskever, I., and Vinyals, O. (2014). Recurrent neural network regularization. arXiv."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/3\/104\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:06:10Z","timestamp":1760105170000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/3\/104"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,28]]},"references-count":61,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["a17030104"],"URL":"https:\/\/doi.org\/10.3390\/a17030104","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,28]]}}}