{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T16:46:35Z","timestamp":1784133995649,"version":"3.55.0"},"reference-count":52,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2022,5,26]],"date-time":"2022-05-26T00:00:00Z","timestamp":1653523200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the German Federal Ministry of Education and Research","award":["01DH16027"],"award-info":[{"award-number":["01DH16027"]}]},{"name":"the German Federal Ministry of Education and Research","award":["491111487"],"award-info":[{"award-number":["491111487"]}]},{"name":"the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)","award":["01DH16027"],"award-info":[{"award-number":["01DH16027"]}]},{"name":"the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)","award":["491111487"],"award-info":[{"award-number":["491111487"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Crowded event entrances could threaten the comfort and safety of pedestrians, especially when some pedestrians push others or use gaps in crowds to gain faster access to an event. Studying and understanding pushing dynamics leads to designing and building more comfortable and safe entrances. Researchers\u2014to understand pushing dynamics\u2014observe and analyze recorded videos to manually identify when and where pushing behavior occurs. Despite the accuracy of the manual method, it can still be time-consuming, tedious, and hard to identify pushing behavior in some scenarios. In this article, we propose a hybrid deep learning and visualization framework that aims to assist researchers in automatically identifying pushing behavior in videos. The proposed framework comprises two main components: (i) Deep optical flow and wheel visualization; to generate motion information maps. (ii) A combination of an EfficientNet-B0-based classifier and a false reduction algorithm for detecting pushing behavior at the video patch level. In addition to the framework, we present a new patch-based approach to enlarge the data and alleviate the class imbalance problem in small-scale pushing behavior datasets. Experimental results (using real-world ground truth of pushing behavior videos) demonstrate that the proposed framework achieves an 86% accuracy rate. Moreover, the EfficientNet-B0-based classifier outperforms baseline CNN-based classifiers in terms of accuracy.<\/jats:p>","DOI":"10.3390\/s22114040","type":"journal-article","created":{"date-parts":[[2022,5,31]],"date-time":"2022-05-31T02:30:06Z","timestamp":1653964206000},"page":"4040","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["A Hybrid Deep Learning and Visualization Framework for Pushing Behavior Detection in Pedestrian Dynamics"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3049-4924","authenticated-orcid":false,"given":"Ahmed","family":"Alia","sequence":"first","affiliation":[{"name":"Institute for Advanced Simulation, Forschungszentrum J\u00fclich, 52425 J\u00fclich, Germany"},{"name":"Computer Simulation for Fire Protection and Pedestrian Traffic, Faculty of Architecture and Civil Engineering, University of Wuppertal, 42285 Wuppertal, Germany"},{"name":"Department of Management Information Systems, Faculty of Engineering and Information Technology, An-Najah National University, Nablus, Palestine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6114-4687","authenticated-orcid":false,"given":"Mohammed","family":"Maree","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Faculty of Engineering and Information Technology, Arab American University, Jenin, Palestine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0999-6807","authenticated-orcid":false,"given":"Mohcine","family":"Chraibi","sequence":"additional","affiliation":[{"name":"Institute for Advanced Simulation, Forschungszentrum J\u00fclich, 52425 J\u00fclich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Adrian, J., Boltes, M., Sieben, A., and Seyfried, A. (2020). Influence of Corridor Width and Motivation on Pedestrians in Front of Bottlenecks. Traffic and Granular Flow 2019, Springer.","DOI":"10.1007\/978-3-030-55973-1_1"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"20190871","DOI":"10.1098\/rsif.2019.0871","article-title":"Crowds in front of bottlenecks at entrances from the perspective of physics and social psychology","volume":"17","author":"Adrian","year":"2020","journal-title":"J. R. Soc. Interface"},{"key":"ref_3","unstructured":"L\u00fcgering, H., \u00dcsten, E., and Sieben, A. (2022). Pushing and Non-Pushing Forward Motion in Crowds: A Systematic Psychological Method for Rating Individual Behavior in Pedestrian Dynamics, Manuscript submitted for publication."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/j.tra.2019.02.007","article-title":"When \u2018push\u2019does not come to \u2018shove\u2019: Revisiting \u2018faster is slower\u2019in collective egress of human crowds","volume":"122","author":"Haghani","year":"2019","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sieben, A., Schumann, J., and Seyfried, A. (2017). Collective phenomena in crowds\u2014Where pedestrian dynamics need social psychology. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0177328"},{"key":"ref_6","unstructured":"Adrian, J., Boltes, M., Holl, S., Sieben, A., and Seyfried, A. (2018). Crowding and queuing in entrance scenarios: Influence of corridor width in front of bottlenecks. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Boltes, M., Seyfried, A., Steffen, B., and Schadschneider, A. (2010). Automatic extraction of pedestrian trajectories from video recordings. Pedestrian and Evacuation Dynamics 2008, Springer.","DOI":"10.1007\/978-3-642-04504-2_3"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"104078","DOI":"10.1016\/j.imavis.2020.104078","article-title":"A comprehensive review on deep learning-based methods for video anomaly detection","volume":"106","author":"Nayak","year":"2021","journal-title":"Image Vis. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1436","DOI":"10.1016\/j.cviu.2013.06.007","article-title":"An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions","volume":"117","author":"Roshtkhari","year":"2013","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_10","first-page":"40","article-title":"Crowd escape event detection via pooling features of optical flow for intelligent video surveillance systems","volume":"10","author":"Singh","year":"2019","journal-title":"Int. J. Image Graph. Signal Process."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"George, M., Bijitha, C., and Jose, B.R. (2018, January 13\u201315). Crowd panic detection using autoencoder with non-uniform feature extraction. Proceedings of the 8th International Symposium on Embedded Computing and System Design (ISED), Cochin, India.","DOI":"10.1109\/ISED.2018.8704021"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Santos, G.L., Endo, P.T., Monteiro, K.H.D.C., Rocha, E.D.S., Silva, I., and Lynn, T. (2019). Accelerometer-based human fall detection using convolutional neural networks. Sensors, 19.","DOI":"10.3390\/s19071644"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Mehmood, A. (2021). LightAnomalyNet: A Lightweight Framework for Efficient Abnormal Behavior Detection. Sensors, 21.","DOI":"10.3390\/s21248501"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, X., Zhang, Q., Hu, S., Guo, C., and Yu, H. (2018). Energy level-based abnormal crowd behavior detection. Sensors, 18.","DOI":"10.3390\/s18020423"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.cviu.2015.06.009","article-title":"Multi-modal human aggression detection","volume":"144","author":"Kooij","year":"2016","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.biosystemseng.2022.03.005","article-title":"Spatiotemporal graph convolutional network for automated detection and analysis of social behaviours among pre-weaning piglets","volume":"217","author":"Gan","year":"2022","journal-title":"Biosyst. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"106357","DOI":"10.1016\/j.compag.2021.106357","article-title":"Automated detection and analysis of social behaviors among preweaning piglets using key point-based spatial and temporal features","volume":"188","author":"Gan","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Vu, T.H., Boonaert, J., Ambellouis, S., and Taleb-Ahmed, A. (2021). Multi-Channel Generative Framework and Supervised Learning for Anomaly Detection in Surveillance Videos. Sensors, 21.","DOI":"10.3390\/s21093179"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1007\/s13244-018-0639-9","article-title":"Convolutional neural networks: An overview and application in radiology","volume":"9","author":"Yamashita","year":"2018","journal-title":"Insights Imaging"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Li, L., Zhang, S., and Wang, B. (2021). Apple leaf disease identification with a small and imbalanced dataset based on lightweight convolutional networks. Sensors, 22.","DOI":"10.3390\/s22010173"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.patrec.2020.07.042","article-title":"Comparative analysis of image classification algorithms based on traditional machine learning and deep learning","volume":"141","author":"Wang","year":"2021","journal-title":"Pattern Recognit. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"183914","DOI":"10.1109\/ACCESS.2019.2960654","article-title":"Anomaly detection in videos using optical flow and convolutional autoencoder","volume":"7","author":"Duman","year":"2019","journal-title":"IEEE Access"},{"key":"ref_23","unstructured":"Farneb\u00e4ck, G. (July, January 29). Two-frame motion estimation based on polynomial expansion. Proceedings of the 13th Scandinavian Conference on Image Analysis, Gothenburg, Sweden."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11042-021-10785-4","article-title":"A hybrid deep network based approach for crowd anomaly detection","volume":"80","author":"Ilyas","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"80408","DOI":"10.1109\/ACCESS.2020.2990355","article-title":"Abnormal crowd behavior detection using motion information images and convolutional neural networks","volume":"8","author":"Direkoglu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Almazroey, A.A., and Jarraya, S.K. (2020, January 8\u201310). Abnormal Events and Behavior Detection in Crowd Scenes Based on Deep Learning and Neighborhood Component Analysis Feature Selection. Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020), Cairo, Egypt.","DOI":"10.1007\/978-3-030-44289-7_25"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Teed, Z., and Deng, J. (2020, January 23\u201328). Raft: Recurrent all-pairs field transforms for optical flow. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.24963\/ijcai.2021\/662"},{"key":"ref_28","unstructured":"Tom Runia, D.F. (2020, April 02). Optical Flow Visualization. Available online: https:\/\/github.com\/tomrunia\/OpticalFlow_Visualization."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11263-010-0390-2","article-title":"A database and evaluation methodology for optical flow","volume":"92","author":"Baker","year":"2011","journal-title":"Int. J. Comput. Vis."},{"key":"ref_30","unstructured":"Tan, M., and Le, Q. (2019, January 9\u201315). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_32","first-page":"683","article-title":"Toward abnormal trajectory and event detection in video surveillance","volume":"27","author":"Donatiello","year":"2016","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Jiang, J., Wang, X., Gao, M., Pan, J., Zhao, C., and Wang, J. (2022). Abnormal behavior detection using streak flow acceleration. Appl. Intell., 1\u201318.","DOI":"10.1007\/s10489-021-02881-7"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Xu, M., Yu, X., Chen, D., Wu, C., and Jiang, Y. (2019). An efficient anomaly detection system for crowded scenes using variational autoencoders. Appl. Sci., 9.","DOI":"10.3390\/app9163337"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Tay, N.C., Connie, T., Ong, T.S., Goh, K.O.M., and Teh, P.S. (2019). A robust abnormal behavior detection method using convolutional neural network. Computational Science and Technology, Springer.","DOI":"10.1007\/978-981-13-2622-6_4"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.cviu.2018.02.006","article-title":"Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes","volume":"172","author":"Sabokrou","year":"2018","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Smeureanu, S., Ionescu, R.T., Popescu, M., and Alexe, B. (2017, January 11\u201315). Deep appearance features for abnormal behavior detection in video. Proceedings of the International Conference on Image Analysis and Processing, Catania, Italy.","DOI":"10.1007\/978-3-319-68548-9_70"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1017\/S026988891300043X","article-title":"One-class classification: Taxonomy of study and review of techniques","volume":"29","author":"Khan","year":"2014","journal-title":"Knowl. Eng. Rev."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/s10723-020-09506-2","article-title":"Design and implementation of abnormal behavior detection based on deep intelligent analysis algorithms in massive video surveillance","volume":"18","author":"Hu","year":"2020","journal-title":"J. Grid Comput."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1016\/j.image.2016.06.007","article-title":"Spatial\u2013temporal convolutional neural networks for anomaly detection and localization in crowded scenes","volume":"47","author":"Zhou","year":"2016","journal-title":"Signal Process. Image Commun."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhang, C., Xu, Y., Xu, Z., Huang, J., and Lu, J. (2022). Hybrid handcrafted and learned feature framework for human action recognition. Appl. Intell., 1\u201317.","DOI":"10.1007\/s10489-021-03068-w"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Adrian, J., Seyfried, A., and Sieben, A. (2020, April 02). Crowds in Front of Bottlenecks from the Perspective of Physics and Social Psychology. Available online: http:\/\/ped.fz-juelich.de\/da\/2018crowdqueue.","DOI":"10.1098\/rsif.2019.0871"},{"key":"ref_43","first-page":"2018","article-title":"Understanding Focal Length and Field of View","volume":"11","author":"Hollows","year":"2016","journal-title":"Retrieved Oct."},{"key":"ref_44","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1534","DOI":"10.3906\/elk-1807-212","article-title":"Optimal training and test sets design for machine learning","volume":"27","author":"Genc","year":"2019","journal-title":"Turk. J. Electr. Eng. Comput. Sci."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"101779","DOI":"10.1016\/j.artmed.2019.101779","article-title":"An enhanced deep learning approach for brain cancer MRI images classification using residual networks","volume":"102","author":"Ismael","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_47","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"027002","DOI":"10.1117\/1.OE.51.2.027002","article-title":"Real-time geometric lens distortion correction using a graphics processing unit","volume":"51","author":"Buytaert","year":"2012","journal-title":"Opt. Eng."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/B978-0-12-811889-4.00001-4","article-title":"Multiview video: Acquisition, processing, compression, and virtual view rendering","volume":"Volume 6","author":"Stankiewicz","year":"2018","journal-title":"Academic Press Library in Signal Processing"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1080\/21681163.2015.1072055","article-title":"Tracking futsal players with a wide-angle lens camera: Accuracy analysis of the radial distortion correction based on an improved Hough transform algorithm","volume":"5","author":"Vieira","year":"2017","journal-title":"Comput. Methods Biomech. Biomed. Eng. Imaging Vis."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/11\/4040\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:19:20Z","timestamp":1760138360000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/11\/4040"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,26]]},"references-count":52,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["s22114040"],"URL":"https:\/\/doi.org\/10.3390\/s22114040","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,26]]}}}