{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T15:08:15Z","timestamp":1767625695445,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2020,9,28]],"date-time":"2020-09-28T00:00:00Z","timestamp":1601251200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61601382"],"award-info":[{"award-number":["61601382"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Doctoral Fund of Southwest University of Science and Technology","award":["No. 16zx7148"],"award-info":[{"award-number":["No. 16zx7148"]}]},{"name":"Scientific Research Fund of Sichuan Provincial Education Department","award":["No. 17ZB0454"],"award-info":[{"award-number":["No. 17ZB0454"]}]},{"name":"Longshan academic talent research supporting program of SWUST","award":["18LZX632"],"award-info":[{"award-number":["18LZX632"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Visual-based object detection and understanding is an important problem in computer vision and signal processing. Due to their advantages of high mobility and easy deployment, unmanned aerial vehicles (UAV) have become a flexible monitoring platform in recent years. However, visible-light-based methods are often greatly influenced by the environment. As a result, a single type of feature derived from aerial monitoring videos is often insufficient to characterize variations among different abnormal crowd behaviors. To address this, we propose combining two types of features to better represent behavior, namely, multitask cascading CNN (MC-CNN) and multiscale infrared optical flow (MIR-OF), capturing both crowd density and average speed and the appearances of the crowd behaviors, respectively. First, an infrared (IR) camera and Nvidia Jetson TX1 were chosen as an infrared vision system. Since there are no published infrared-based aerial abnormal-behavior datasets, we provide a new infrared aerial dataset named the IR-flying dataset, which includes sample pictures and videos in different scenes of public areas. Second, MC-CNN was used to estimate the crowd density. Third, MIR-OF was designed to characterize the average speed of crowd. Finally, considering two typical abnormal crowd behaviors of crowd aggregating and crowd escaping, the experimental results show that the monitoring UAV system can detect abnormal crowd behaviors in public areas effectively.<\/jats:p>","DOI":"10.3390\/s20195550","type":"journal-article","created":{"date-parts":[[2020,9,28]],"date-time":"2020-09-28T08:02:58Z","timestamp":1601280178000},"page":"5550","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV"],"prefix":"10.3390","volume":"20","author":[{"given":"Yanhua","family":"Shao","sequence":"first","affiliation":[{"name":"School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China"}]},{"given":"Wenfeng","family":"Li","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China"}]},{"given":"Hongyu","family":"Chu","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China"}]},{"given":"Zhiyuan","family":"Chang","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China"}]},{"given":"Xiaoqiang","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China"}]},{"given":"Huayi","family":"Zhan","sequence":"additional","affiliation":[{"name":"Electrical Engineering and Computer Science, Northwestern University, Evanston, IL 60208, USA"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,28]]},"reference":[{"doi-asserted-by":"crossref","unstructured":"Zhang, X., Zhang, Q., Hu, S., Guo, C., and Yu, H. (2018). Energy level-based abnormal crowd behavior detection. Sensors, 18.","key":"ref_1","DOI":"10.3390\/s18020423"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1016\/j.neucom.2015.11.021","article-title":"Crowd behavior analysis: A review where physics meets biology","volume":"177","author":"Kok","year":"2016","journal-title":"Neurocomputing"},{"doi-asserted-by":"crossref","unstructured":"Zhang, Y., Zhou, D., Chen, S., Gao, S., and Ma, Y. (2016, January 27\u201330). Single-image crowd counting via multi-column convolutional neural network. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","key":"ref_3","DOI":"10.1109\/CVPR.2016.70"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1408","DOI":"10.1109\/TCSVT.2018.2837153","article-title":"Beyond counting: Comparisons of density maps for crowd analysis tasks\u2014counting, detection, and tracking","volume":"29","author":"Kang","year":"2019","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3052930","article-title":"Crowd scene understanding from video: A survey","volume":"13","author":"Grant","year":"2017","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"doi-asserted-by":"crossref","unstructured":"Sindagi, V.A., and Patel, V.M. (September, January 29). Cnn-based cascaded multi-task learning of high-level prior and density estimation for crowd counting. Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy.","key":"ref_6","DOI":"10.1109\/AVSS.2017.8078491"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1109\/JIOT.2016.2612119","article-title":"Low-altitude unmanned aerial vehicles-based internet of things services: Comprehensive survey and future perspectives","volume":"3","author":"Motlagh","year":"2016","journal-title":"IEEE Int. Things J."},{"doi-asserted-by":"crossref","unstructured":"Gonzalez, L.F., Montes, G.A., Puig, E., Johnson, S., Mengersen, K., and Gaston, K.J. (2016). Unmanned aerial vehicles (uavs) and artificial intelligence revolutionizing wildlife monitoring and conservation. Sensors, 16.","key":"ref_8","DOI":"10.3390\/s16010097"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.ijtst.2017.02.001","article-title":"Unmanned aerial aircraft systems for transportation engineering: Current practice and future challenges","volume":"5","author":"Barmpounakis","year":"2016","journal-title":"Int. J. Transport. Sci. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1109\/TITS.2017.2782790","article-title":"Effective and efficient detection of moving targets from a uav\u2019s camera","volume":"19","author":"Minaeian","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"doi-asserted-by":"crossref","unstructured":"Wu, K., Cai, Z., Zhao, J., and Wang, Y. (2017). Target tracking based on a nonsingular fast terminal sliding mode guidance law by fixed-wing uav. Appl. Sci., 7.","key":"ref_11","DOI":"10.3390\/app7040333"},{"doi-asserted-by":"crossref","unstructured":"Sandino, J., Gonzalez, F., Mengersen, K., and Gaston, K.J. (2018). Uavs and machine learning revolutionising invasive grass and vegetation surveys in remote arid lands. Sensors, 18.","key":"ref_12","DOI":"10.3390\/s18020605"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1109\/TPAMI.2011.155","article-title":"Pedestrian detection: An evaluation of the state of the art","volume":"34","author":"Dollar","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.neucom.2017.01.043","article-title":"Discriminative latent semantic feature learning for pedestrian detection","volume":"238","author":"Zhu","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_15","first-page":"985","article-title":"Scale-aware fast r-cnn for pedestrian detection","volume":"20","author":"Li","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1007\/s11263-009-0308-z","article-title":"Volumetric features for video event detection","volume":"88","author":"Ke","year":"2010","journal-title":"Int. J. Comput. Vis."},{"doi-asserted-by":"crossref","unstructured":"Idrees, H., Saleemi, I., Seibert, C., and Shah, M. (2013, January 23\u201328). Multi-source multi-scale counting in extremely dense crowd images. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","key":"ref_17","DOI":"10.1109\/CVPR.2013.329"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1109\/TPAMI.2018.2875002","article-title":"Detecting coherent groups in crowd scenes by multiview clustering","volume":"42","author":"Wang","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"doi-asserted-by":"crossref","unstructured":"Monajjemi, M., Mohaimenianpour, S., and Vaughan, R. (2016, January 9\u201314). Uav, come to me: End-to-end, multi-scale situated hri with an uninstrumented human and a distant uav. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","key":"ref_19","DOI":"10.1109\/IROS.2016.7759649"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1109\/TITS.2014.2331353","article-title":"Efficient road detection and tracking for unmanned aerial vehicle","volume":"16","author":"Hailing","year":"2015","journal-title":"IEEE Trans. Intell. Trans. Syst."},{"key":"ref_21","first-page":"106151","article-title":"Using infrared hog-based pedestrian detection for outdoor autonomous searching uav with embedded system","volume":"Volume 10615","author":"Shao","year":"2018","journal-title":"Proceedings of the 9th International Conference on Graphic and Image Processing, ICGIP 2017"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"482","DOI":"10.1016\/j.image.2016.05.007","article-title":"Deep convolutional neural networks for pedestrian detection","volume":"47","author":"Tome","year":"2016","journal-title":"Signal Proc.Image"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1215","DOI":"10.1109\/JBHI.2018.2852718","article-title":"Multitask cascade convolution neural networks for automatic thyroid nodule detection and recognition","volume":"23","author":"Song","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"doi-asserted-by":"crossref","unstructured":"Ali, S., Nishino, K., Manocha, D., and Shah, M. (2013). Crowd counting and profiling: Methodology and evaluation. Modeling, Simulation and Visual Analysis of Crowds: A Multidisciplinary Perspective, Springer.","key":"ref_24","DOI":"10.1007\/978-1-4614-8483-7"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1016\/j.engappai.2019.04.011","article-title":"Crowd analysis using bayesian risk kernel density estimation","volume":"82","author":"Razavi","year":"2019","journal-title":"Eng. Appl. Artif. Intell."},{"unstructured":"Su, H., Dong, Y., Zhu, J., Ling, H., and Zhang, B. (2016, January 22). Crowd scene understanding with coherent recurrent neural networks. Proceedings of the International Joint Conference On Artificial Intelligence, New York, NY, USA.","key":"ref_26"},{"doi-asserted-by":"crossref","unstructured":"Liu, N., Long, Y., Zou, C., Niu, Q., Pan, L., and Wu, H. (2019, January 15\u201321). Adcrowdnet: An attention-injective deformable convolutional network for crowd understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition CVPR, Long Beach, CA, USA.","key":"ref_27","DOI":"10.1109\/CVPR.2019.00334"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1109\/TCSVT.2014.2358029","article-title":"Crowded scene analysis: A survey","volume":"25","author":"Li","year":"2015","journal-title":"IEEE Trans. Circ. Syst. Video Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"4474","DOI":"10.1109\/JSYST.2019.2910080","article-title":"A method for optimized deployment of a network of surveillance aerial drones","volume":"13","author":"Savkin","year":"2019","journal-title":"IEEE Syst. J."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1109\/MAES.2019.2914986","article-title":"Conflict Detection and Resolution for Civil Aviation: A Literature Survey","volume":"34","author":"Tang","year":"2019","journal-title":"IEEE Aerosp. Electr. Syst. Mag."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1016\/j.trc.2016.03.001","article-title":"Coloured Petri net-based traffic collision avoidance system encounter model for the analysis of potential induced collisions","volume":"67","author":"Tang","year":"2016","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1109\/TPAMI.2010.143","article-title":"Large displacement optical flow: Descriptor matching in variational motion estimation","volume":"33","author":"Brox","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"doi-asserted-by":"crossref","unstructured":"Dai, J., He, K., and Sun, J. (2016, January 27\u201330). Instance-aware semantic segmentation via multi-task network cascades. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","key":"ref_33","DOI":"10.1109\/CVPR.2016.343"},{"doi-asserted-by":"crossref","unstructured":"Chen, J., Kumar, A., Ranjan, R., Patel, V.M., Alavi, A., and Chellappa, R. (2016, January 6\u20139). A cascaded convolutional neural network for age estimation of unconstrained faces. Proceedings of the 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS), Washinton, DC, USA.","key":"ref_34","DOI":"10.1109\/BTAS.2016.7791154"},{"doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., and Fergus, R. (2014, January 6\u201312). Visualizing and understanding convolutional networks. Proceedings of the European Conference On Computer Vision 2014, Zurich, Switzerland.","key":"ref_35","DOI":"10.1007\/978-3-319-10590-1_53"},{"doi-asserted-by":"crossref","unstructured":"Jianbo, S., and Tomasi, C. (1994, January 21\u201323). Good features to track. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","key":"ref_36","DOI":"10.1109\/CVPR.1994.323794"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1023\/B:VISI.0000027790.02288.f2","article-title":"Scale & affine invariant interest point detectors","volume":"60","author":"Mikolajczyk","year":"2004","journal-title":"Int. J. Comput. Vision"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1109\/TPAMI.2017.2679193","article-title":"Binary online learned descriptors","volume":"40","author":"Balntas","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"doi-asserted-by":"crossref","unstructured":"Acevedo, J.J., Maza, I., Ollero, A., and Arrue, B.C. (2020). An Efficient Distributed Area Division Method for Cooperative Monitoring Applications with Multiple UAVs. Sensors, 20.","key":"ref_39","DOI":"10.3390\/s20123448"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5550\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:14:29Z","timestamp":1760177669000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5550"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,28]]},"references-count":39,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["s20195550"],"URL":"https:\/\/doi.org\/10.3390\/s20195550","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,9,28]]}}}