{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T18:39:09Z","timestamp":1763059149175,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,6,7]],"date-time":"2024-06-07T00:00:00Z","timestamp":1717718400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62102076","20220402033GH","RS-2022-00143336"],"award-info":[{"award-number":["62102076","20220402033GH","RS-2022-00143336"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Science and Technology Development Plan of Jilin Province, China","award":["62102076","20220402033GH","RS-2022-00143336"],"award-info":[{"award-number":["62102076","20220402033GH","RS-2022-00143336"]}]},{"name":"Korea Agency for Infrastructure Technology Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport","award":["62102076","20220402033GH","RS-2022-00143336"],"award-info":[{"award-number":["62102076","20220402033GH","RS-2022-00143336"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Video action recognition based on skeleton nodes is a highlighted issue in the computer vision field. In real application scenarios, the large number of skeleton nodes and behavior occlusion problems between individuals seriously affect recognition speed and accuracy. Therefore, we proposed a lightweight multi-stream feature cross-fusion (L-MSFCF) model to recognize abnormal behaviors such as fighting, vicious kicking, climbing over the wall, et al., which could obviously improve recognition speed based on lightweight skeleton node calculation, and improve recognition accuracy based on occluded skeleton node prediction analysis in order to effectively solve the behavior occlusion problem. The experiments show that our proposed All-MSFCF model has a video action recognition average accuracy rate of 92.7% for eight kinds of abnormal behavior recognition. Although our proposed lightweight L-MSFCF model has an 87.3% average accuracy rate, its average recognition speed is 62.7% higher than the full-skeleton recognition model, which is more suitable for solving real-time tracing problems. Moreover, our proposed Trajectory Prediction Tracking (TPT) model could real-time predict the moving positions based on the dynamically selected core skeleton node calculation, especially for the short-term prediction within 15 frames and 30 frames that have lower average loss errors.<\/jats:p>","DOI":"10.3390\/s24123711","type":"journal-article","created":{"date-parts":[[2024,6,7]],"date-time":"2024-06-07T08:05:17Z","timestamp":1717747517000},"page":"3711","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Video Abnormal Behavior Recognition and Trajectory Prediction Based on Lightweight Skeleton Feature Extraction"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8956-0511","authenticated-orcid":false,"given":"Ling","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China"}]},{"given":"Cong","family":"Ding","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China"}]},{"given":"Yifan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3258-0244","authenticated-orcid":false,"given":"Tie Hua","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, School of Computer Science, Northeast Electric Power University, Jilin 132013, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4402-5616","authenticated-orcid":false,"given":"Wei","family":"Ding","sequence":"additional","affiliation":[{"name":"Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan 250000, China"},{"name":"Shandong Provincial Key Laboratory of Computer Networks, Shandong Fundamental Research Center for Computer Science, Jinan 250000, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0394-9054","authenticated-orcid":false,"given":"Keun Ho","family":"Ryu","sequence":"additional","affiliation":[{"name":"Data Science Laboratory, Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City 700000, Vietnam"},{"name":"Research Institute, Bigsun System Co., Ltd., Seoul 06266, Republic of Korea"},{"name":"Database and Bioinformatics Laboratory, College of Electrical and Computer Engineering, Chungbuk National University, Cheongju 28644, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9970-7863","authenticated-orcid":false,"given":"Kwang Woo","family":"Nam","sequence":"additional","affiliation":[{"name":"Department of Computer and Information Engineering, Kunsan National University, Gunsan 54150, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/j.imavis.2017.01.010","article-title":"Going deeper into action recognition: A survey","volume":"60","author":"Herath","year":"2017","journal-title":"Image Vis. Comput."},{"key":"ref_2","unstructured":"Wu, X., Ou, Y., Qian, H., and Xu, Y. (2005, January 2\u20136). A detection system for human abnormal behavior. Proceedings of the 2005 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mehran, R., Oyama, A., and Shah, M. (2009, January 20\u201325). Abnormal crowd behavior detection using social force model. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206641"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1007\/s00521-018-3692-x","article-title":"Video crowd detection and abnormal behavior model detection based on machine learning method","volume":"31","author":"Xie","year":"2019","journal-title":"Neural Comput. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"704","DOI":"10.1109\/TSMC.2016.2638048","article-title":"A content-adaptively sparse reconstruction method for abnormal events detection with low-rank property","volume":"47","author":"Yu","year":"2016","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sabokrou, M., Fathy, M., Hoseini, M., and Klette, R. (2015, January 7\u201312). Real-time anomaly detection and localization in crowded scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA.","DOI":"10.1109\/CVPRW.2015.7301284"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1992","DOI":"10.1109\/TIP.2017.2670780","article-title":"Deep-cascade: Cascading 3d deep neural networks for fast anomaly detection and localization in crowded scenes","volume":"26","author":"Sabokrou","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1109\/TSMC.2013.2279661","article-title":"Camera selection for adaptive human\u2013computer interface","volume":"44","author":"Martinel","year":"2013","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_9","first-page":"683","article-title":"Toward abnormal trajectory and event detection in video surveillance","volume":"27","author":"Donatiello","year":"2016","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2814","DOI":"10.1109\/TCSVT.2017.2731866","article-title":"An efficient method of crowd aggregation computation in public areas","volume":"28","author":"Xu","year":"2017","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kerdvibulvech, C., and Yamauchi, K. (2014). 3D human motion analysis for reconstruction and recognition. Articulated Motion and Deformable Objects, Proceedings of the 8th International Conference, AMDO 2014, Palma de Mallorca, Spain, 16\u201318 July 2014, Springer. Proceedings 8.","DOI":"10.1007\/978-3-319-08849-5_12"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kerdvibulvech, C., and Yamauchi, K. (2014). Structural human shape analysis for modeling and recognition. Structural, Syntactic, and Statistical Pattern Recognition, Proceedings of the Joint IAPR International Workshop, S+ SSPR 2014, Joensuu, Finland, 20\u201322 August 2014, Springer. Proceedings.","DOI":"10.1007\/978-3-662-44415-3_29"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Houenou, A., Bonnifait, P., Cherfaoui, V., and Yao, W. (2013, January 3\u20137). Vehicle trajectory prediction based on motion model and maneuver recognition. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696982"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1016\/j.imavis.2006.07.027","article-title":"A particle filter for joint detection and tracking of color objects","volume":"25","author":"Czyz","year":"2007","journal-title":"Image Vis. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.jprocont.2009.11.002","article-title":"Constrained Bayesian state estimation\u2014A comparative study and a new particle filter based approach","volume":"20","author":"Shao","year":"2010","journal-title":"J. Process Control"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1016\/j.procs.2018.10.453","article-title":"Maximum likelihood multiple model filtering for path prediction in intelligent transportation systems","volume":"143","author":"Vashishtha","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kapania, S., Saini, D., Goyal, S., Thakur, N., Jain, R., and Nagrath, P. (2020, January 11). Multi object tracking with UAVs using deep SORT and YOLOv3 RetinaNet detection framework. Proceedings of the 1st ACM Workshop on Autonomous and Intelligent Mobile Systems, Bangalore, India.","DOI":"10.1145\/3377283.3377284"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Choi, D., Yim, J., Baek, M., and Lee, S. (2021). Machine learning-based vehicle trajectory prediction using v2v communications and on-board sensors. Electronics, 10.","DOI":"10.3390\/electronics10040420"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6951","DOI":"10.1109\/TAC.2023.3244116","article-title":"Data-driven self-triggered control via trajectory prediction","volume":"68","author":"Liu","year":"2023","journal-title":"IEEE Trans. Autom. Control."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"33401","DOI":"10.1007\/s11042-023-14733-2","article-title":"Gait reference trajectory generation at different walking speeds using LSTM and CNN","volume":"82","author":"Semwal","year":"2023","journal-title":"Multimed. Tools Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1097","DOI":"10.1007\/s00138-019-01040-w","article-title":"Trajectory prediction of vehicles turning at intersections using deep neural networks","volume":"30","author":"Morris","year":"2019","journal-title":"Mach. Vis. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1109\/TCSVT.2018.2825679","article-title":"Deep continuous conditional random fields with asymmetric inter-object constraints for online multi-object tracking","volume":"29","author":"Zhou","year":"2018","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1016\/j.ins.2020.10.002","article-title":"Online multiple pedestrians tracking using deep temporal appearance matching association","volume":"561","author":"Yoon","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.cviu.2018.02.006","article-title":"Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes","volume":"172","author":"Sabokrou","year":"2018","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1109\/TMM.2018.2846411","article-title":"Sparse coding guided spatiotemporal feature learning for abnormal event detection in large videos","volume":"21","author":"Chu","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yan, L. (2023, January 24\u201326). Researching on human abnormal behavior recognition algorithm. Proceedings of the 2023 IEEE 6th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chongqing, China.","DOI":"10.1109\/ITNEC56291.2023.10082144"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"14617","DOI":"10.1007\/s11042-016-3316-3","article-title":"Abnormal event detection in crowded scenes based on deep learning","volume":"75","author":"Fang","year":"2016","journal-title":"Multimed. Tools Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"97564","DOI":"10.1109\/ACCESS.2020.2997357","article-title":"Abnormal event detection via feature expectation subgraph calibrating classification in video surveillance scenes","volume":"8","author":"Ye","year":"2020","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Tay, N.C., Tee, C., Ong, T.S., and Teh, P.S. (2019, January 25). Abnormal behavior recognition using CNN-LSTM with attention mechanism. Proceedings of the 2019 1st International Conference on Electrical, Control and Instrumentation Engineering (ICECIE), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICECIE47765.2019.8974824"},{"key":"ref_30","unstructured":"Ionescu, C., Papava, D., Olaru, V., and Sminchisescu, C. (2018, August 18). Human 3.6M. Available online: http:\/\/vision.imar.ro\/human3.6m\/description.php."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sultani, W., Chen, C., and Shah, M. (2018, January 18\u201323). Real-world anomaly detection in surveillance videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00678"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, W., Luo, W., Lian, D., and Gao, S. (2018, January 18\u201323). Future frame prediction for anomaly detection\u2014A new baseline. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00684"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Shi, L., Zhang, Y., Cheng, J., and Lu, H. (2019, January 15\u201320). Two-stream adaptive graph convolutional networks for skeleton-based action recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01230"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Martinez, J., Black, M.J., and Romero, J. (2017, January 21\u201326). On human motion prediction using recurrent neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.497"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Liang, J., Jiang, L., Niebles, J.C., Hauptmann, A.G., and Fei-Fei, L. (2019, January 15\u201320). Peeking into the future: Predicting future person activities and locations in videos. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00358"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., and Alahi, A. (2018, January 18\u201323). Social gan: Socially acceptable trajectories with generative adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00240"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/12\/3711\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:55:21Z","timestamp":1760108121000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/12\/3711"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,7]]},"references-count":36,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["s24123711"],"URL":"https:\/\/doi.org\/10.3390\/s24123711","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2024,6,7]]}}}