{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T15:25:03Z","timestamp":1773156303970,"version":"3.50.1"},"reference-count":45,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2020,9,1]],"date-time":"2020-09-01T00:00:00Z","timestamp":1598918400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The recognition of human activities is usually considered to be a simple procedure. Problems occur in complex scenes involving high speeds. Activity prediction using Artificial Intelligence (AI) by numerical analysis has attracted the attention of several researchers. Human activities are an important challenge in various fields. There are many great applications in this area, including smart homes, assistive robotics, human\u2013computer interactions, and improvements in protection in several areas such as security, transport, education, and medicine through the control of falling or aiding in medication consumption for elderly people. The advanced enhancement and success of deep learning techniques in various computer vision applications encourage the use of these methods in video processing. The human presentation is an important challenge in the analysis of human behavior through activity. A person in a video sequence can be described by their motion, skeleton, and\/or spatial characteristics. In this paper, we present a novel approach to human activity recognition from videos using the Recurrent Neural Network (RNN) for activity classification and the Convolutional Neural Network (CNN) with a new structure of the human skeleton to carry out feature presentation. The aims of this work are to improve the human presentation through the collection of different features and the exploitation of the new RNN structure for activities. The performance of the proposed approach is evaluated by the RGB-D sensor dataset CAD-60. The experimental results show the performance of the proposed approach through the average error rate obtained (4.5%).<\/jats:p>","DOI":"10.3390\/s20174944","type":"journal-article","created":{"date-parts":[[2020,9,1]],"date-time":"2020-09-01T08:53:43Z","timestamp":1598950423000},"page":"4944","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Prediction of Human Activities Based on a New Structure of Skeleton Features and Deep Learning Model"],"prefix":"10.3390","volume":"20","author":[{"given":"Neziha","family":"Jaouedi","sequence":"first","affiliation":[{"name":"Sciences and Technologies of Image and Telecommunications (SETIT) Laboratory, Sfax 3029, Tunisia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9872-3172","authenticated-orcid":false,"given":"Francisco J.","family":"Perales","sequence":"additional","affiliation":[{"name":"Mathematics and Computer Science Department, Universitat de les Illes Balears (UIB), E-07122 Palma, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6137-9558","authenticated-orcid":false,"given":"Jos\u00e9 Maria","family":"Buades","sequence":"additional","affiliation":[{"name":"Mathematics and Computer Science Department, Universitat de les Illes Balears (UIB), E-07122 Palma, Spain"}]},{"given":"Noureddine","family":"Boujnah","sequence":"additional","affiliation":[{"name":"Telecommunication Software and Systems Group (TSSG), Waterford Institute of Technology, X91 P20H Waterford, Ireland"}]},{"given":"Med Salim","family":"Bouhlel","sequence":"additional","affiliation":[{"name":"Sciences and Technologies of Image and Telecommunications (SETIT) Laboratory, Sfax 3029, Tunisia"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Rodr\u00edguez-Moreno, I., Mart\u00ednez-Otzeta, J.M., Sierra, B., Rodriguez, I., and Jauregi, E. (2019). Video Activity Recognition: State-of-the-Art. Sensors, 19.","DOI":"10.3390\/s19143160"},{"key":"ref_2","unstructured":"Wren, C.R., Azarbayejani, A.J., Darrell, T.J., and Pentland, A.P. (1996). Integration Issues in Large Commercial Media Delivery Systems, SPIE."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Elgammal, A., Harwood, D., and Davis, L. (2000). Non-parametric model for background subtraction. Computer Vision\u2014ECCV 2000, Springer.","DOI":"10.1007\/3-540-45053-X_48"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Barnich, O., and Van Droogenbroeck, M. (2009, January 19\u201324). ViBE: A powerful random technique to estimate the background in video sequences. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Taipei, Taiwan.","DOI":"10.1109\/ICASSP.2009.4959741"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1007\/BF01215814","article-title":"Segmentation and tracking of piglets in images","volume":"8","author":"McFarlane","year":"1995","journal-title":"Mach. Vis. Appl."},{"key":"ref_6","unstructured":"Lucas, B.D., and Kanadee, T. (1981, January 24\u201328). Aniterative image registration technique with an application tostereovision. Proceedings of the Imaging Understanding Workshop, Pittsburgh, PA, USA."},{"key":"ref_7","first-page":"185","article-title":"Determining optical flow","volume":"Volume 17","author":"Horn","year":"1981","journal-title":"Techniques and Applications of Image Understanding"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Lowe, D.G. (1999, January 20\u201327). Object recognition from local scale-invariant features. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece.","DOI":"10.1109\/ICCV.1999.790410"},{"key":"ref_9","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of oriented gradients for human detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Comaniciu, D., and Meer, P. (1999, January 20\u201327). Mean shift analysis and applications. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece.","DOI":"10.1109\/ICCV.1999.790416"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"52277","DOI":"10.1109\/ACCESS.2018.2869790","article-title":"Human Action Monitoring for Healthcare based on Deep Learning","volume":"6","author":"Gao","year":"2018","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"7027","DOI":"10.1007\/s00500-018-3364-x","article-title":"Human activity learning for assistive robotics using a classifier ensemble","volume":"22","author":"Adama","year":"2018","journal-title":"Soft Comp."},{"key":"ref_13","unstructured":"Albu, V. (2016). Measuring Customer Behavior with Deep Convolutional Neural Networks, BRAIN. Broad Research in Artificial Intelligence and Neuroscience."},{"key":"ref_14","first-page":"47","article-title":"Human action recognition using support vector machines and 3D convolutional neural networks","volume":"3","author":"Majd","year":"2017","journal-title":"Intern. J. Adv. Intel. Inf."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Murad, A., and Pyun, J.-Y. (2017). Deep Recurrent Neural Networks for Human Activity Recognition. Sensors, 17.","DOI":"10.3390\/s17112556"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.inffus.2019.06.014","article-title":"Imaging and fusing time series for wearable sensors based human activity recognition","volume":"53","author":"Qin","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_17","unstructured":"Ning, Z., Zeyuan, H., Sukhwan, L., and Eungjoo, L. (2017, January 17\u201319). Human Action Recognition Based on Global Silhouette and Local Optical Flow. Proceedings of the International Symposium on Mechanical Engineering and Material Science, Suzhou, China."},{"key":"ref_18","unstructured":"Nicolas, B., Li, Y., Chris, P., and Aaron, C. (2016). Delving Deeper into Convolutional Networks for Learning Video Representations. Computer Vision and Pattern Recognition. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xu, Z., Hu, J., and Deng, W. (2016, January 11\u201315). Recurrent convolutional neural network for video classification. Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), Seattle, WA, USA.","DOI":"10.1109\/ICME.2016.7552971"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Baldominos, A., Saez, Y., and Isasi, P. (2018). Evolutionary Design of Convolutional Neural Networks for Human Activity Recognition in Sensor-Rich Environments. Sensors, 18.","DOI":"10.3390\/s18041288"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, L., Feng, Y., Han, J., and Zhen, X. (2016, January 20\u201325). Realistic human action recognition: When deep learning meets VLAD. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.","DOI":"10.1109\/ICASSP.2016.7471897"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhao, R., Ali, H., and van der Smagt, P. (2017, January 24\u201328). Two-stream RNN\/CNN for action recognition in 3D videos. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206288"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Faria, D.R., Premebida, C., and Nunes, U. (2014, January 25\u201329). A probabilistic approach for human everyday activities recognition using body motion from RGB-D images. Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, UK.","DOI":"10.1109\/ROMAN.2014.6926340"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1177\/0278364913478446","article-title":"Learning human activities and object affordances from RGB-D videos","volume":"32","author":"Koppula","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.1109\/TCYB.2013.2276433","article-title":"Multilevel Depth and Image Fusion for Human Activity Detection","volume":"43","author":"Ni","year":"2013","journal-title":"IEEE Trans. Cybern."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1109\/TPAMI.2013.198","article-title":"Learning Actionlet Ensemble for 3D Human Action Recognition","volume":"36","author":"Wang","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Machin. Intel."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shan, J., and Akella, S. (2014, January 11\u201313). 3D human action segmentation and recognition using pose kinetic energy. Proceedings of the 2014 IEEE International Workshop on Advanced Robotics and Its Social Impacts, Evanston, IL, USA.","DOI":"10.1109\/ARSO.2014.7020983"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2016\/4351435","article-title":"A Human Activity Recognition System Using Skeleton Data from RGBD Sensors","volume":"2016","author":"Cippitelli","year":"2016","journal-title":"Comput. Intel. Neurosci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1109\/THMS.2014.2377111","article-title":"Human Activity Recognition Process Using 3-D Posture Data","volume":"45","author":"Gaglio","year":"2015","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Manzi, A., Dario, P., and Cavallo, F. (2017). A Human Activity Recognition System Based on Dynamic Clustering of Skeleton Data. Sensors, 17.","DOI":"10.3390\/s17051100"},{"key":"ref_31","unstructured":"Srijan, D., Michal, K., Francois, B., and Gianpiero, F. (2018). A Fusion of Appearance based CNNs and Temporal evolution of Skeleton with LSTM for Daily Living Action Recognition. arXiv."},{"key":"ref_32","unstructured":"Cruz-Silva, J.E., Montiel-P\u00e9rez, J.Y., and Sossa-Azuela, H. (2013). 3-D Human Body Posture Reconstruction by Computer Vision, LNAI 11835; Springer."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Khaire, P., Kumar, P., and Imran, J. (2018). Combining CNN Streams of RGB-D and Skeletal Data for Human Activity Recognition. Pattern Recognition Letters, Elsevier.","DOI":"10.1016\/j.patrec.2018.04.035"},{"key":"ref_34","unstructured":"Ji, Y., Xu, F., Yang, Y., Shen, F., Shen, H.T., and Zheng, W.-S. (2020, January 12\u201316). A Large-scale RGB-D Database for Arbitrary-view Human Action Recognition. Proceedings of the ACM Multimedia Conference on Multimedia Conference\u2014MM \u201918, Seoul, Korea."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, P., Li, W., Ogunbona, P., Wan, J., and Escalera, S. (2018). RGB-D-based human motion recognition with deep learning: A survey. Computer Vision and Image Understanding, Elsevier.","DOI":"10.1016\/j.cviu.2018.04.007"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.patcog.2018.01.038","article-title":"Articulated motion and deformable objects","volume":"Volume 79","author":"Wan","year":"2010","journal-title":"Pattern Recognition"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Pham, H.H., Salmane, H., Khoudour, L., Crouzil, A., Zegers, P., and Velastin, S.A. (2019). Spatio\u2013Temporal Image Representation of 3D Skeletal Movements for View-Invariant Action Recognition with Deep Convolutional Neural Networks. Sensors, 19.","DOI":"10.20944\/preprints201903.0086.v1"},{"key":"ref_38","first-page":"447","article-title":"fvA New Hybrid Deep Learning Model for Human Action Recognition","volume":"32","author":"Jaouedi","year":"2020","journal-title":"J. King Saud Univ. Comput. Inf. Sci."},{"key":"ref_39","unstructured":"Sung, J., Ponce, C., Selman, B., and Saxena, A. (2012, January 14\u201318). Unstructured human activity detection from RGBD images. Proceedings of the IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA."},{"key":"ref_40","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"31319","DOI":"10.1007\/s11042-019-7740-z","article-title":"A weighting scheme for mining key skeletal joints for human action recognition","volume":"Volume 78","author":"Shabaninia","year":"2019","journal-title":"Multimedia Tools and Applications"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Sedmidubsky, J., Elias, P., and Zezula, P. (2018). Effective and efficient similarity searching in motion capture data. Multimedia Tools and Applications, Springer.","DOI":"10.1145\/3240508.3241468"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Yang, K., Ding, X., and Chen, W. (2019, January 29\u201331). Multi-Scale Spatial Temporal Graph Convolutional LSTM Network for Skeleton-Based Human Action Recognition. Proceedings of the 2019 International Conference on Video, Signal and Image Processing, Wuhan, China.","DOI":"10.1145\/3369318.3369325"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Elias, P., Sedmidubsky, J., and Zezula, P. (2019, January 9\u201311). Understanding the Gap between 2D and 3D Skeleton-Based Action Recognition. Proceedings of the IEEE International Symposium on Multimedia, San Diego, CA, USA.","DOI":"10.1109\/ISM46123.2019.00041"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"27309","DOI":"10.1007\/s11042-019-07827-3","article-title":"LSTM-based real-time action detection and prediction in human motion streams","volume":"78","author":"Carrara","year":"2019","journal-title":"Multimedia Tools and Applications"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/17\/4944\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:05:29Z","timestamp":1760177129000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/17\/4944"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,1]]},"references-count":45,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["s20174944"],"URL":"https:\/\/doi.org\/10.3390\/s20174944","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,1]]}}}