{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T04:40:44Z","timestamp":1777092044569,"version":"3.51.4"},"reference-count":59,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2018,6,6]],"date-time":"2018-06-06T00:00:00Z","timestamp":1528243200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Vehicle behavior recognition is an attractive research field which is useful for many computer vision and intelligent traffic analysis tasks. This paper presents an all-in-one behavior recognition framework for moving vehicles based on the latest deep learning techniques. Unlike traditional traffic analysis methods which rely on low-resolution videos captured by road cameras, we capture 4K (    3840 \u00d7 2178    ) traffic videos at a busy road intersection of a modern megacity by flying a unmanned aerial vehicle (UAV) during the rush hours. We then manually annotate locations and types of road vehicles. The proposed method consists of the following three steps: (1) vehicle detection and type recognition based on deep neural networks; (2) vehicle tracking by data association and vehicle trajectory modeling; (3) vehicle behavior recognition by nearest neighbor search and by bidirectional long short-term memory network, respectively. This paper also presents experimental results of the proposed framework in comparison with state-of-the-art approaches on the 4K testing traffic video, which demonstrated the effectiveness and superiority of the proposed method.<\/jats:p>","DOI":"10.3390\/rs10060887","type":"journal-article","created":{"date-parts":[[2018,6,6]],"date-time":"2018-06-06T10:53:28Z","timestamp":1528282408000},"page":"887","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Bidirectional Long Short-Term Memory Network for Vehicle Behavior Recognition"],"prefix":"10.3390","volume":"10","author":[{"given":"Jiasong","family":"Zhu","sequence":"first","affiliation":[{"name":"Key Laboratory of Spatial Information Smarting Sensing and Services, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3116-3203","authenticated-orcid":false,"given":"Ke","family":"Sun","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spatial Information Smarting Sensing and Services, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sen","family":"Jia","sequence":"additional","affiliation":[{"name":"Computer Vision Research Institute, College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weidong","family":"Lin","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spatial Information Smarting Sensing and Services, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xianxu","family":"Hou","sequence":"additional","affiliation":[{"name":"College of Information Engineering, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bozhi","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Information Engineering, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guoping","family":"Qiu","sequence":"additional","affiliation":[{"name":"College of Information Engineering, Shenzhen University, Shenzhen 518060, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,6,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Choi, E.H. (2010). Crash Factors in Intersection-Related Crashes: An on-Scene Perspective, The National Highway Traffic Safety Administration. Technical Report.","DOI":"10.1037\/e621942011-001"},{"key":"ref_2","unstructured":"Miller, T., Kolosh, K., Fearn, K., and Porretta, K. (2015). Injury Facts, National Safety Council. Technical Report."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kim, Z., and Malik, J. (2003, January 13\u201316). Fast vehicle detection with probabilistic feature grouping and its application to vehicle tracking. Proceedings of the Ninth IEEE International Conference on Computer Vision, Nice, France.","DOI":"10.1109\/ICCV.2003.1238392"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1114","DOI":"10.1109\/TCSVT.2008.927109","article-title":"A survey of vision-based trajectory learning and analysis for surveillance","volume":"18","author":"Morris","year":"2008","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1109\/6979.880969","article-title":"Image analysis and rule-based reasoning for a traffic monitoring system","volume":"1","author":"Cucchiara","year":"2000","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.autcon.2016.05.008","article-title":"Detecting and tracking vehicles in traffic by unmanned aerial vehicles","volume":"72","author":"Wang","year":"2016","journal-title":"Autom. Constr."},{"key":"ref_7","unstructured":"Brooks, C., Dobson, R., Banach, D., Dean, D., Oommen, T., Wolf, R., Havens, T., Ahlborn, T., and Hart, B. (2015). Evaluating the Use of Unmanned Aerial Vehicles for Transportation Purposes, Department of Transportation (MDOT). Technical Report."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1109\/TITS.2014.2331353","article-title":"Efficient Road Detection and Tracking for Unmanned Aerial Vehicle","volume":"16","author":"Zhou","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","unstructured":"Hinz, S. (2003, January 14\u201317). Detection and counting of cars in aerial images. Proceedings of the 2003 International Conference on Image Processing, Barcelona, Spain."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1109\/LGRS.2010.2097239","article-title":"Comparison of two methods for vehicle extraction from airborne LiDAR data toward motion analysis","volume":"8","author":"Yao","year":"2011","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Mundhenk, T.N., Konjevod, G., Sakla, W.A., and Boakye, K. (2016, January 8\u201316). A large contextual dataset for classification, detection and counting of cars with deep learning. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_48"},{"key":"ref_12","unstructured":"Coifman, B., McCord, M., Mishalani, R.G., and Redmill, K. (2004, January 11\u201315). Surface transportation surveillance from unmanned aerial vehicles. Proceedings of the 83rd Annual Meeting of the Transportation Research Board, Washington, DC, USA."},{"key":"ref_13","unstructured":"Puri, A. (2005). A Survey of Unmanned Aerial Vehicles (UAV) for Traffic Surveillance, Department of Computer Science and Engineering, University of South Florida."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1635","DOI":"10.1109\/TGRS.2013.2253108","article-title":"Automatic car counting method for unmanned aerial vehicle images","volume":"52","author":"Moranduzzo","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1083","DOI":"10.1016\/j.sbspro.2014.01.143","article-title":"Urban traffic analysis through a UAV","volume":"111","author":"Salvo","year":"2014","journal-title":"Procedia-Soc. Behav. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kanistras, K., Martins, G., Rutherford, M.J., and Valavanis, K.P. (2015). Survey of unmanned aerial vehicles (UAVs) for traffic monitoring. Handbook of Unmanned Aerial Vehicles, Springer.","DOI":"10.1007\/978-90-481-9707-1_122"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Gallagher, K., and Lawrence, P. (2016). Unmanned systems and managing from above: the practical implications of UAVs for research applications addressing urban sustainability. Urban Sustainability: Policy and Praxis, Springer.","DOI":"10.1007\/978-3-319-26218-5_14"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1016\/j.trpro.2017.03.043","article-title":"UAV-Based Traffic Analysis: A Universal Guiding Framework Based on Literature Survey","volume":"22","author":"Khan","year":"2017","journal-title":"Transp. Res. Procedia"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1049\/ip-its:20055014","article-title":"Roadway traffic monitoring from an unmanned aerial vehicle","volume":"153","author":"Coifman","year":"2006","journal-title":"IEE Proc.-Intell. Transp. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2076","DOI":"10.3390\/rs4072076","article-title":"Road target search and tracking with gimballed vision sensor on an unmanned aerial vehicle","volume":"4","author":"Skoglar","year":"2012","journal-title":"Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6356","DOI":"10.1109\/TGRS.2013.2296351","article-title":"Detecting Cars in UAV Images With a Catalog-Based Approach","volume":"52","author":"Moranduzzo","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017). Focal loss for dense object detection. arXiv.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1109\/34.910878","article-title":"The recognition of human movement using temporal templates","volume":"23","author":"Bobick","year":"2001","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"628","DOI":"10.1109\/TITS.2009.2026440","article-title":"Learning to recognize video-based spatiotemporal events","volume":"10","author":"Veeraraghavan","year":"2009","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0022-4375(99)00024-9","article-title":"Predicting red-light running behavior: A traffic safety study in three urban settings","volume":"31","author":"Porter","year":"2000","journal-title":"J. Saf. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1109\/MITS.2012.2203229","article-title":"Object-oriented Bayesian networks for detection of lane change maneuvers","volume":"4","author":"Kasper","year":"2012","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_27","unstructured":"Hu, W., Xiao, X., Xie, D., and Tan, T. (2003, January 12\u201315). Traffic accident prediction using vehicle tracking and trajectory analysis. Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems, Shanghai, China."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3580","DOI":"10.1109\/TVT.2014.2307958","article-title":"Vehicle behavior analysis using target motion trajectories","volume":"63","author":"Song","year":"2014","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1109\/TITS.2009.2029643","article-title":"Estimating the driving state of oncoming vehicles from a moving platform using stereo vision","volume":"10","author":"Barth","year":"2009","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/MSP.2010.937393","article-title":"Video anomaly identification","volume":"27","author":"Saligrama","year":"2010","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_31","unstructured":"Huang, H., Cai, Z., Shi, S., Ma, X., and Zhu, Y. (2009, January 26\u201328). Automatic detection of vehicle activities based on particle filter tracking. Proceedings of the Second Symposium International Computer Science and Computational Technology (ISCSCT \u201909), Huangshan, China."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Pucher, M., Schabus, D., Schallauer, P., Lypetskyy, Y., Graf, F., Rainer, H., Stadtschnitzer, M., Sternig, S., Birchbauer, J., and Schneider, W. (2010, January 19\u201322). Multimodal highway monitoring for robust incident detection. Proceedings of the 13th International IEEE Conference on Intelligent Transportation Systems, Funchal, Portugal.","DOI":"10.1109\/ITSC.2010.5625035"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature pyramid networks for object detection. Proceedings of the Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_34","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas Valley, NV, USA."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Neubeck, A., and Van Gool, L. (2006, January 20\u201324). Efficient non-maximum suppression. Proceedings of the 18th International Conference on Pattern Recognition (ICPR \u201906), Hong Kong, China.","DOI":"10.1109\/ICPR.2006.479"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Erhan, D., Szegedy, C., Toshev, A., and Anguelov, D. (2014, January 24\u201327). Scalable object detection using deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.276"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Washington, DC, USA.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A new approach to linear filtering and prediction problems","volume":"82","author":"Kalman","year":"1960","journal-title":"J. Basic Eng."},{"key":"ref_39","unstructured":"Mills-Tettey, G.A., Stentz, A., and Dias, M.B. (2007). The Dynamic Hungarian Algorithm for the Assignment Problem with Changing Costs, Carnegie Mellon University."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1109\/TITS.2010.2048101","article-title":"Clustering of vehicle trajectories","volume":"11","author":"Atev","year":"2010","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Kamijo, K.i., and Tanigawa, T. (1990, January 17\u201321). Stock price pattern recognition\u2014A recurrent neural network approach. Proceedings of the 1990 IJCNN International Joint Conference on Neural Networks, San Diego, CA, USA.","DOI":"10.1109\/IJCNN.1990.137572"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Tai, K.S., Socher, R., and Manning, C.D. (2015). Improved semantic representations from tree-structured long short-term memory networks. arXiv.","DOI":"10.3115\/v1\/P15-1150"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1093\/bioinformatics\/btw678","article-title":"Improving protein disorder prediction by deep bidirectional long short-term memory recurrent neural networks","volume":"33","author":"Hanson","year":"2016","journal-title":"Bioinformatics"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhou, P., Shi, W., Tian, J., Qi, Z., Li, B., Hao, H., and Xu, B. (2016, January 7\u201312). Attention-based bidirectional long short-term memory networks for relation classification. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.","DOI":"10.18653\/v1\/P16-2034"},{"key":"ref_46","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_47","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2006, January 8\u201316). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands."},{"key":"ref_48","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7\u201312). Faster R-CNN: Towards real-time object detection with region proposal networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3\u20137). Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA.","DOI":"10.1145\/2647868.2654889"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"11315","DOI":"10.3390\/rs61111315","article-title":"An operational system for estimating road traffic information from aerial images","volume":"6","author":"Leitloff","year":"2014","journal-title":"Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1409","DOI":"10.1109\/TPAMI.2011.239","article-title":"Tracking-Learning-Detection","volume":"34","author":"Kalal","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Bae, S.H., and Yoon, K.J. (2014, January 23\u201328). Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.159"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Alahi, A., and Savarese, S. (2015, January 7\u201313). Learning to track: Online multi-object tracking by decision making. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.534"},{"key":"ref_54","unstructured":"Chollet, F. (2018, June 05). Github. Available online: https:\/\/github.com\/fchollet\/keras."},{"key":"ref_55","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2\u20134). TensorFlow: A System for Large-Scale Machine Learning. Proceedings of the 12th USENIX Symposium on Operating Systems Designand Implementation (OSDI \u201916), Savannah, GA, USA."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Hervieu, A., Bouthemy, P., and Le Cadre, J.P. (October, January 16). A HMM-based method for recognizing dynamic video contents from trajectories. Proceedings of the 2007 IEEE International Conference on Image Processing, San Antonio, TX, USA.","DOI":"10.1109\/ICIP.2007.4380072"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"2287","DOI":"10.1109\/TPAMI.2011.64","article-title":"Trajectory learning for activity understanding: Unsupervised, multilevel, and long-term adaptive approach","volume":"33","author":"Morris","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_58","first-page":"288","article-title":"Vehicle Behavior Recognition Method Based on Quadratic Spectral Clustering and HMM-RF Hybrid Model","volume":"43","author":"Fan","year":"2016","journal-title":"Comput. Sci."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Altch\u00e9, F., and De La Fortelle, A. (2017, January 16\u201319). An LSTM network for highway trajectory prediction. Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan.","DOI":"10.1109\/ITSC.2017.8317913"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/10\/6\/887\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:07:37Z","timestamp":1760195257000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/10\/6\/887"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,6]]},"references-count":59,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2018,6]]}},"alternative-id":["rs10060887"],"URL":"https:\/\/doi.org\/10.3390\/rs10060887","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,6,6]]}}}