{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T16:42:12Z","timestamp":1781973732969,"version":"3.54.5"},"reference-count":69,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,5,10]],"date-time":"2024-05-10T00:00:00Z","timestamp":1715299200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008982","name":"Qatar National Research Fund","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100008982","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:p>Dependable utilization of computer vision applications, such as smart surveillance, requires training deep learning networks on datasets that sufficiently represent the classes of interest. However, the bottleneck in many computer vision applications lies in the limited availability of adequate datasets. One particular application that is of great importance for the safety of cities and crowded areas is smart surveillance. Conventional surveillance methods are reactive and often ineffective in enable real-time action. However, smart surveillance is a key component of smart and proactive security in a smart city. Motivated by a smart city application which aims at the automatic identification of concerning events for alerting law-enforcement and governmental agencies, we craft a large video dataset that focuses on the distinction between small-scale violence, large-scale violence, peaceful gatherings, and natural events. This dataset classifies public events along two axes, the size of the crowd observed and the level of perceived violence in the crowd. We name this newly-built dataset the Multi-Scale Violence and Public Gathering (<jats:bold>MSV-PG<\/jats:bold>) dataset. The videos in the dataset go through several pre-processing steps to prepare them to be fed into a deep learning architecture. We conduct several experiments on the <jats:bold>MSV-PG<\/jats:bold> datasets using a ResNet3D, a Swin Transformer and an R(2 + 1)D architecture. The results achieved by these models when trained on the <jats:bold>MSV-PG<\/jats:bold> dataset, 88.37%, 89.76%, and 89.3%, respectively, indicate that the dataset is well-labeled and is rich enough to train deep learning models for automatic smart surveillance for diverse scenarios.<\/jats:p>","DOI":"10.3389\/fcomp.2024.1242690","type":"journal-article","created":{"date-parts":[[2024,5,10]],"date-time":"2024-05-10T16:52:42Z","timestamp":1715359962000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["A novel multi-scale violence and public gathering dataset for crowd behavior classification"],"prefix":"10.3389","volume":"6","author":[{"given":"Almiqdad","family":"Elzein","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emrah","family":"Basaran","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yin David","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marwa","family":"Qaraqe","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,5,10]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IPTA.2019.8936070","article-title":"\u201cVision-based fight detection from surveillance cameras,\u201d","volume-title":"2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA)","author":"Akt\u0131","year":"2019"},{"key":"B2","doi-asserted-by":"publisher","first-page":"42","DOI":"10.3390\/fi11020042","article-title":"3d-cnn-based fused feature maps with LSTM applied to action recognition","volume":"11","author":"Arif","year":"2019","journal-title":"Future Internet"},{"key":"B3","first-page":"6836","article-title":"\u201cVivit: a video vision transformer,\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Arnab","year":"2021"},{"key":"B4","doi-asserted-by":"publisher","first-page":"332","DOI":"10.1007\/978-3-642-23678-5_39","article-title":"\u201cViolence detection in video using computer vision techniques,\u201d","author":"Bermejo Nievas","year":"2011","journal-title":"International Conference on Computer Analysis of Images and Patterns"},{"key":"B5","author":"Bertasius","year":"2021"},{"key":"B6","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1007\/978-3-030-58452-8_13","article-title":"\u201cEnd-to-end object detection with transformers,\u201d","volume-title":"Computer Vision - ECCV 2020","author":"Carion","year":"2020"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1808.01340","article-title":"A short note about kinetics-600","author":"Carreira","year":"2018","journal-title":"arXiv"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1907.06987","article-title":"A short note on the kinetics-700 human action dataset","author":"Carreira","year":"2019","journal-title":"arXiv"},{"key":"B9","first-page":"6299","article-title":"\u201cQuo vadis, action recognition? A new model and the kinetics dataset,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Carreira","year":"2017"},{"key":"B10","first-page":"59","article-title":"\u201cExploring recent advancements of transformer based architectures in computer vision,\u201d","volume-title":"Selected Topics in Applied Computer Science","author":"Chromiak","year":"2021"},{"key":"B11","author":"Clark","year":"2016","journal-title":"Mass Mobilization Protest Data"},{"key":"B12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IJCB48548.2020.9304905","article-title":"\u201cHuman activity analysis: iterative weak\/self-supervised learning frameworks for detecting abnormal events,\u201d","volume-title":"2020 IEEE International Joint Conference on Biometrics (IJCB)","author":"Degardin","year":"2020"},{"key":"B13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/CBMI.2014.6849827","article-title":"\u201cBenchmarking violent scenes detection in movies,\u201d","volume-title":"2014 12th International Workshop on Content-Based Multimedia Indexing (CBMI)","author":"Demarty","year":"2014"},{"key":"B14","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/j.engappai.2018.08.014","article-title":"A review of state-of-the-art techniques for abnormal human activity recognition","volume":"77","author":"Dhiman","year":"2019","journal-title":"Eng. Appl. Arti. Intell"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2010.11929","article-title":"An image is worth 16x16 words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2020","journal-title":"arXiv"},{"key":"B16","article-title":"\u201cAn image is worth 16x16 words: transformers for image recognition at scale,\u201d","author":"Dosovitskiy","year":"2021","journal-title":"International Conference on Learning Representations"},{"key":"B17","doi-asserted-by":"publisher","first-page":"112829","DOI":"10.1016\/j.eswa.2019.112829","article-title":"MULTID-CNN: a multi-dimensional feature learning approach based on deep convolutional networks for gesture recognition in rgb-d image sequences","volume":"139","author":"Elboushaki","year":"2020","journal-title":"Expert Syst. Appl"},{"key":"B18","first-page":"1933","article-title":"\u201cConvolutional two-stream network fusion for video action recognition,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Feichtenhofer","year":"2016"},{"key":"B19","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1016\/j.ins.2019.06.039","article-title":"Long short-term memory-based deep recurrent neural networks for target tracking","volume":"502","author":"Gao","year":"2019","journal-title":"Inf. Sci"},{"key":"B20","author":"Global","year":"2022","journal-title":"Role of CCTV Cameras: Public, Privacy and Protection"},{"key":"B21","first-page":"516","article-title":"\u201cIdentification of loitering human behaviour in video surveillance environments,\u201d","volume-title":"International Work-Conference on the Interplay Between Natural and Artificial Computation","author":"G\u00f3mez A","year":"2015"},{"key":"B22","doi-asserted-by":"crossref","DOI":"10.1109\/ICCVW.2019.00132","article-title":"\u201cLow quality video face recognition: multi-mode aggregation recurrent network (MARN),\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops","author":"Gong","year":"2019"},{"key":"B23","unstructured":"GoodfellowI.\n            BengioY.\n            CourvilleA.\n          Deep Learning2016"},{"key":"B24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/AVSS.2019.8909888","article-title":"\u201cA hybrid facial expression recognition system based on recurrent neural network,\u201d","volume-title":"2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)","author":"Guo","year":"2019"},{"key":"B25","first-page":"3154","article-title":"\u201cLearning spatio-temporal features with 3d residual networks for action recognition,\u201d","volume-title":"Proceedings of the IEEE International Conference on Computer Vision Workshops","author":"Hara","year":"2017"},{"key":"B26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/CVPRW.2012.6239348","article-title":"\u201cViolent flows: real-time detection of violent crowd behavior,\u201d","volume-title":"2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops","author":"Hassner","year":"2012"},{"key":"B27","first-page":"5353","article-title":"\u201cConvolutional neural networks at constrained time cost,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He","year":"2015"},{"key":"B28","first-page":"770","article-title":"\u201cDeep residual learning for image recognition,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He","year":"2016"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"B30","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1109\/ICOT.2015.7498476","article-title":"\u201cHuman action recognition system for elderly and children care using three stream convnet,\u201d","volume-title":"2015 International Conference on Orange Technologies (ICOT)","author":"Huang","year":"2015"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.217","article-title":"A hierarchical deep temporal model for group activity recognition","author":"Ibrahim","year":"2016","journal-title":"In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages"},{"key":"B32","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1109\/TPAMI.2012.59","article-title":"3d convolutional neural networks for human action recognition","volume":"35","author":"Ji","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B33","doi-asserted-by":"crossref","first-page":"1725","DOI":"10.1109\/CVPR.2014.223","article-title":"\u201cLarge-scale video classification with convolutional neural networks,\u201d","volume-title":"2014 IEEE Conference on Computer Vision and Pattern Recognition","author":"Karpathy","year":"2014"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1705.06950","article-title":"The kinetics human action video dataset","author":"Kay","year":"2017","journal-title":"arXiv"},{"key":"B35","doi-asserted-by":"crossref","first-page":"2556","DOI":"10.1109\/ICCV.2011.6126543","article-title":"\u201cHMDB51: a large video database for human motion recognition,\u201d","volume-title":"2011 International Conference on Computer Vision","author":"Kuehne","year":"2011"},{"key":"B36","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/CVPR.2008.4587756","article-title":"\u201cLearning realistic human actions from movies,\u201d","volume-title":"2008 IEEE Conference on Computer Vision and Pattern Recognition","author":"Laptev","year":"2008"},{"key":"B37","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2006.16668","article-title":"Gshard: scaling giant models with conditional computation and automatic sharding","author":"Lepikhin","year":"2020","journal-title":"arXiv"},{"key":"B39","first-page":"10012","article-title":"\u201cSwin transformer: hierarchical vision transformer using shifted windows,\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Liu","year":"2021"},{"key":"B40","first-page":"3202","article-title":"\u201cVideo swin transformer,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Liu","year":"2022"},{"key":"B41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/AVSS.2017.8078482","article-title":"\u201cResnetcrowd: a residual deep learning architecture for crowd counting, violent behaviour detection and crowd density level classification,\u201d","volume-title":"2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)","author":"Marsden","year":"2017"},{"key":"B42","doi-asserted-by":"publisher","first-page":"108487","DOI":"10.1016\/j.patcog.2021.108487","article-title":"Action transformer: a 520 self-attention model for short-time pose-based human action recognition","volume":"124","author":"Mazzia","year":"2022","journal-title":"Pattern Recognit."},{"key":"B43","doi-asserted-by":"publisher","first-page":"103303","DOI":"10.1016\/j.micpro.2020.103303","article-title":"Efficient anomaly detection in surveillance videos based on multi layer perception recurrent neural network","volume":"79","author":"Murugesan","year":"2020","journal-title":"Microprocess. Microsyst"},{"key":"B44","doi-asserted-by":"publisher","first-page":"104078","DOI":"10.1016\/j.imavis.2020.104078","article-title":"A comprehensive review on deep learning-based methods for video anomaly detection","volume":"106","author":"Nayak","year":"2021","journal-title":"Image Vis. Comput"},{"key":"B45","doi-asserted-by":"publisher","first-page":"3163","DOI":"10.1109\/ICCVW54120.2021.00355","author":"Neimark","year":"2021"},{"key":"B46","doi-asserted-by":"crossref","first-page":"2662","DOI":"10.1109\/ICASSP.2019.8683676","article-title":"\u201cDetection of real-world fights in surveillance videos,\u201d","volume-title":"ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":"Perez","year":"2019"},{"key":"B47","doi-asserted-by":"publisher","first-page":"971","DOI":"10.1007\/s00138-012-0450-4","article-title":"Recognizing 50 human action categories of web videos","volume":"24","author":"Reddy","year":"2013","journal-title":"Mach. Vis. Appl"},{"key":"B48","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1016\/j.cviu.2018.02.006","article-title":"Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes","volume":"172","author":"Sabokrou","year":"2018","journal-title":"Comput. Vis. Image Underst"},{"key":"B49","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1109\/ICIT48102.2019.00063","article-title":"\u201cUnusual event detection in surveillance video using transfer learning,\u201d","volume-title":"2019 International Conference on Information Technology (ICIT)","author":"Sahoo","year":"2019"},{"key":"B50","doi-asserted-by":"publisher","first-page":"318","DOI":"10.1016\/j.inffus.2020.07.008","article-title":"Revisiting crowd behaviour analysis through deep learning: taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects","volume":"64","author":"S\u00e1nchez","year":"2020","journal-title":"Inf. Fusion"},{"key":"B51","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1109\/SPIN.2018.8474279","article-title":"\u201cSemantic background estimation in video sequences,\u201d","volume-title":"2018 5th International Conference on Signal Processing and Integrated Networks (SPIN)","author":"Savakis","year":"2018"},{"key":"B52","doi-asserted-by":"publisher","first-page":"139489","DOI":"10.1109\/ACCESS.2021.3118541","article-title":"Video processing using deep learning techniques: a systematic literature review","volume":"9","author":"Sharma","year":"2021","journal-title":"IEEE Access"},{"key":"B53","first-page":"28","article-title":"Convolutional LSTM network: a machine learning approach for precipitation nowcasting","volume":"28","author":"Shi","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B54","doi-asserted-by":"crossref","first-page":"3090","DOI":"10.1109\/ICIP.2017.8296851","article-title":"\u201cConvolutional gated recurrent networks for video segmentation,\u201d","volume-title":"2017 IEEE International Conference on Image Processing (ICIP)","author":"Siam","year":"2017"},{"key":"B55","first-page":"27","article-title":"Two-stream convolutional networks for action recognition in videos","volume":"1","author":"Simonyan","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B56","first-page":"79","article-title":"\u201cViolence recognition from videos using deep learning techniques\u201d","author":"Soliman","year":"2019","journal-title":"Proceeding of 9th International Conference on Intelligent Computing and Information Systems (ICICIS'19)"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1212.0402","article-title":"Ucf101: a dataset of 101 human actions classes from videos in the wild","author":"Soomro","year":"2012","journal-title":"arXiv"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1505.00387","article-title":"Highway networks","author":"Srivastava","year":"2015","journal-title":"arXiv"},{"key":"B59","year":"2017","journal-title":"Learning to Detect Violent Videos Using Convolutional Long Short-Term Memory"},{"key":"B60","doi-asserted-by":"publisher","first-page":"6479","DOI":"10.1109\/CVPR.2018.00678","author":"Sultani","year":"2018"},{"key":"B61","article-title":"\u201cTraining data-efficient image transformers and distillation through attention,\u201d","author":"Touvron","year":"2021","journal-title":"International Conference on Machine Learning"},{"key":"B62","first-page":"4489","article-title":"\u201cLearning spatiotemporal features with 3d convolutional networks,\u201d","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Tran","year":"2015"},{"key":"B63","first-page":"6450","article-title":"\u201cA closer look at spatiotemporal convolutions for action recognition,\u201d","volume-title":"Proceedings of the IEEE conference on Computer Vision and Pattern Recognition","author":"Tran","year":"2018"},{"key":"B64","doi-asserted-by":"publisher","first-page":"1510","DOI":"10.1109\/TPAMI.2017.2712608","article-title":"Long-term temporal convolutions for action recognition","volume":"40","author":"Varol","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B65","first-page":"30","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B66","doi-asserted-by":"publisher","first-page":"032035","DOI":"10.1088\/1757-899X\/569\/3\/032035","article-title":"I3d-LSTM: a new model for human action recognition","volume":"569","author":"Wang","year":"2019","journal-title":"IOP Conf. Ser.: Mater. Sci. Eng"},{"key":"B67","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.1109\/CVPR.2015.7298789","article-title":"\u201cA discriminative cnn video representation for event detection,\u201d","author":"Xu","year":"2015","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B68","doi-asserted-by":"publisher","first-page":"9349","DOI":"10.1007\/s00521-019-04232-7","article-title":"Real-time yoga recognition using deep learning","volume":"31","author":"Yadav","year":"2019","journal-title":"Neural Comput. Appl"},{"key":"B69","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2010.04159","article-title":"Deformable detr: Deformable transformers for end-to-end object detection","author":"Zhu","year":"2020","journal-title":"arXiv"}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2024.1242690\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,10]],"date-time":"2024-05-10T16:53:14Z","timestamp":1715359994000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2024.1242690\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,10]]},"references-count":69,"alternative-id":["10.3389\/fcomp.2024.1242690"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2024.1242690","relation":{},"ISSN":["2624-9898"],"issn-type":[{"value":"2624-9898","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,10]]},"article-number":"1242690"}}