{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T22:35:28Z","timestamp":1764714928317,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2019,11,21]],"date-time":"2019-11-21T00:00:00Z","timestamp":1574294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Nature Science Foundation of China","award":["61751308, 61603057, 61773311"],"award-info":[{"award-number":["61751308, 61603057, 61773311"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>For analyzing the traffic anomaly within dashcam videos from the perspective of ego-vehicles, the agent should spatial-temporally localize the abnormal occasion and regions and give a semantically recounting of what happened. Most existing formulations concentrate on the former spatial-temporal aspect and mainly approach this goal by training normal pattern classifiers\/regressors\/dictionaries with large-scale availably labeled data. However, anomalies are context-related, and it is difficult to distinguish the margin of abnormal and normal clearly. This paper proposes a progressive unsupervised driving anomaly detection and recounting (D&amp;R) framework. The highlights are three-fold: (1) We formulate driving anomaly D&amp;R as a temporal-spatial-semantic (TSS) model, which achieves a coarse-to-fine focusing and generates convincing driving anomaly D&amp;R. (2) This work contributes an unsupervised D&amp;R without any training data while performing an effective performance. (3) We novelly introduce the traffic saliency, isolation forest, visual semantic causal relations of driving scene to effectively construct the TSS model. Extensive experiments on a driving anomaly dataset with 106 video clips (temporal-spatial-semantically labeled carefully by ourselves) demonstrate superior performance over existing techniques.<\/jats:p>","DOI":"10.3390\/s19235098","type":"journal-article","created":{"date-parts":[[2019,11,22]],"date-time":"2019-11-22T02:49:27Z","timestamp":1574390967000},"page":"5098","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Progressive Temporal-Spatial-Semantic Analysis of Driving Anomaly Detection and Recounting"],"prefix":"10.3390","volume":"19","author":[{"given":"Rixing","family":"Zhu","sequence":"first","affiliation":[{"name":"School of Electronic and Control Engineering, Chang\u2019an University, Xi\u2019an 710064, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0300-6892","authenticated-orcid":false,"given":"Jianwu","family":"Fang","sequence":"additional","affiliation":[{"name":"School of Electronic and Control Engineering, Chang\u2019an University, Xi\u2019an 710064, China"},{"name":"Institute of Artificial Intelligence and Robotics (IAIR), Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongke","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Electronic and Control Engineering, Chang\u2019an University, Xi\u2019an 710064, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianru","family":"Xue","sequence":"additional","affiliation":[{"name":"Institute of Artificial Intelligence and Robotics (IAIR), Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,11,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Luo, W., Liu, W., and Gao, S. (2017, January 22\u201329). A Revisit of Sparse Coding Based Anomaly Detection in Stacked RNN Framework. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.45"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Tran, H.T., and Hogg, D. (2017, January 4\u20137). Anomaly Detection using a Convolutional Winner-Take-All Autoencoder. Proceedings of the 2017 British Machine Vision Conference, London, UK.","DOI":"10.5244\/C.31.139"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1992","DOI":"10.1109\/TIP.2017.2670780","article-title":"Deep-Cascade: Cascading 3D Deep Neural Networks for Fast Anomaly Detection and Localization in Crowded Scenes","volume":"26","author":"Sabokrou","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_4","unstructured":"Giorno, A.D., Bagnell, J.A., and Hebert, M. (2016, January 11\u201314). A Discriminative Framework for Anomaly Detection in Large Videos. Proceedings of the 2016 European Conference on Computer Vision, Amsterdam, The Netherlands."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kratz, L., and Nishino, K. (2009, January 20\u201325). Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206771"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/TPAMI.2013.111","article-title":"Anomaly detection and localization in crowded scenes","volume":"36","author":"Li","year":"2014","journal-title":"IEEE Trans. Patten Anal. Mach. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hu, J., Zhu, E., Wang, S., Liu, X., Guo, X., and Yin, J. (2019). An Efficient and Robust Unsupervised Anomaly Detection Method Using Ensemble Random Projection in Surveillance Videos. Sensors, 19.","DOI":"10.3390\/s19194145"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Basan, E., Basan, A., and Nekrasov, A. (2019). Method for Detecting Abnormal Activity in a Group of Mobile Robots. Sensors, 19.","DOI":"10.3390\/s19184007"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Venskus, J., Treigys, P., Bernatavi\u010dien\u0117, J., Tamulevi\u010dius, G., and Medvedev, V. (2019). Real-time maritime traffic anomaly detection based on sensors and history data embedding. Sensors, 19.","DOI":"10.3390\/s19173782"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wang, H., Wen, H., Yi, F., Zhu, H., and Sun, L. (2017). Road traffic anomaly detection via collaborative path inference from GPS snippets. Sensors, 17.","DOI":"10.3390\/s17030550"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1541880.1541882","article-title":"Anomaly detection: A survey","volume":"41","author":"Chandola","year":"2009","journal-title":"ACM Comput. Surv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2051","DOI":"10.1109\/TITS.2016.2535402","article-title":"Where Does the Driver Look? Top-Down-Based Saliency Detection in a Traffic Driving Environment","volume":"17","author":"Deng","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_13","unstructured":"Palazzi, A., Solera, F., Calderara, S., Alletto, S., and Cucchiara, R. (2016). Where Should You Attend While Driving?. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Fang, J., Yan, D., Qiao, J., Xue, J., Wang, H., and Li, S. (2019). DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmark. arXiv.","DOI":"10.1109\/ITSC.2019.8917218"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (July, January 26). The Cityscapes Dataset for Semantic Urban Scene Understanding. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.jsr.2011.11.004","article-title":"On-board safety monitoring systems for driving: Review, knowledge gaps, and framework","volume":"43","author":"Horrey","year":"2012","journal-title":"J. Saf. Res."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1049\/iet-its.2010.0129","article-title":"Wireless sensor networks for traffic management and road safety","volume":"6","author":"Pascale","year":"2012","journal-title":"Intell. Transp. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1016\/j.compind.2018.02.014","article-title":"On-board monitoring system for road traffic safety analysis","volume":"98","author":"Battiato","year":"2018","journal-title":"Comput. Ind."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1016\/j.trc.2015.04.007","article-title":"Large-scale automated proactive road safety analysis using video data","volume":"58","author":"Saunier","year":"2015","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"90","DOI":"10.3141\/2365-12","article-title":"Flexible, mobile video camera system and open source video analysis software for road safety and behavioral analysis","volume":"2365","author":"Jackson","year":"2013","journal-title":"Transp. Res. Rec."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2087","DOI":"10.1109\/TII.2017.2674661","article-title":"SafeDrive: Online Driving Anomaly Detection From Large-Scale Vehicle Data","volume":"13","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1158","DOI":"10.1109\/TPAMI.2013.172","article-title":"Online Learning and Sequential Anomaly Detection in Trajectories","volume":"36","author":"Laxhammar","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1109\/TIP.2008.2012070","article-title":"A dynamic hierarchical clustering method for trajectory-based unusual video event detection","volume":"18","author":"Jiang","year":"2009","journal-title":"IEEE Trans. Image Process."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"5288","DOI":"10.1109\/TIP.2015.2479561","article-title":"Gaussian Process Regression-Based Video Anomaly Detection and Localization With Hierarchical Feature Representation","volume":"24","author":"Cheng","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhao, B., Li, F., and Xing, E.P. (2011, January 20\u201325). Online detection of unusual events in videos via dynamic sparse coding. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995524"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, Y., Qian, J., and Saligrama, V. (2013, January 26\u201331). A new one-class SVM for anomaly detection. Proceedings of the 2013 International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638322"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Kim, J., and Grauman, K. (2009, January 20\u201325). Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206569"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2133360.2133363","article-title":"Isolation-Based Anomaly Detection","volume":"6","author":"Liu","year":"2012","journal-title":"ACM Trans. Knowl. Discov. Data"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Mehran, R., Oyama, A., and Shah, M. (2009, January 20\u201325). Abnormal crowd behavior detection using social force model. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206641"},{"key":"ref_31","unstructured":"Sabokrou, M., Fayyaz, M., Fathy, M., and Klette, R. (2016). Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chan, F., Chen, Y., Xiang, Y., and Sun, M. (2016, January 20\u201324). Anticipating Accidents in Dashcam Videos. Proceedings of the 2016 Asian Conference on Computer Vision, Taipei, Taiwan.","DOI":"10.1007\/978-3-319-54190-7_9"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kiran, B., Thomas, D., and Parakkal, R. (2018). An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos. J. Imaging, 4.","DOI":"10.3390\/jimaging4020036"},{"key":"ref_34","unstructured":"Medel, J.R., and Savakis, A.E. (2016). Anomaly Detection in Video Using Predictive Convolutional Long Short-Term Memory Networks. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Basharat, A., Gritai, A., and Shah, M. (2008, January 23\u201328). Learning object motion patterns for anomaly detection and improved object detection. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.","DOI":"10.1109\/CVPR.2008.4587510"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1109\/TCYB.2014.2330853","article-title":"Online Anomaly Detection in Crowd Scenes via Structure Analysis","volume":"45","author":"Yuan","year":"2015","journal-title":"IEEE Trans. Cybern."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.neucom.2017.12.017","article-title":"Incrementally perceiving hazards in driving","volume":"282","author":"Yuan","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1524","DOI":"10.1109\/TCYB.2015.2451136","article-title":"Crowd Event Detection on Optical Flow Manifolds","volume":"46","author":"Rao","year":"2016","journal-title":"IEEE Trans. Cybern."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1590","DOI":"10.1109\/TIFS.2013.2272243","article-title":"Video Anomaly Search in Crowded Scenes via Spatio-Temporal Motion Context","volume":"8","author":"Cong","year":"2013","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2147","DOI":"10.1109\/TCYB.2013.2242059","article-title":"Laplacian eigenmap with temporal constraints for local abnormality detection in crowded scenes","volume":"43","author":"Thida","year":"2013","journal-title":"IEEE Trans. Cybern."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Liu, W., Luo, W., Lian, D., and Gao, S. (2018, January 18\u201322). Future frame prediction for anomaly detection\u2014A new baseline. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00684"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Gan, C., Wang, N., Yang, Y., Yeung, D.Y., and Hauptmann, A.G. (2015, January 7\u201312). DevNet: A Deep Event Network for multimedia event detection and evidence recounting. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298872"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Hinami, R., Mei, T., and Satoh, S.I. (2017, January 22\u201329). Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.391"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Alletto, S., Palazzi, A., Solera, F., Calderara, S., and Cucchiara, R. (July, January 26). DR(eye)VE: A Dataset for Attention-Based Tasks with Applications to Autonomous and Assisted Driving. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA.","DOI":"10.1109\/CVPRW.2016.14"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Hawkins, D.M. (1980). Identification of Outliers, Chapman and Hall.","DOI":"10.1007\/978-94-015-3994-4"},{"key":"ref_47","unstructured":"Liu, C. (2009). Beyond Pixels: Exploring New Representations and Applications for Motion Analysis, Massachusetts Institute of Technology."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2809782","article-title":"Learning Perceptual Causality from Video","volume":"7","author":"Fire","year":"2015","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1443","DOI":"10.1162\/089976601750264965","article-title":"Estimating the Support of a High-Dimensional Distribution","volume":"13","author":"Platt","year":"2001","journal-title":"Neural Comput."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Zhou, C., and Paffenroth, R.C. (2017, January 13\u201317). Anomaly Detection with Robust Deep Autoencoders. Proceedings of the 2017 ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.","DOI":"10.1145\/3097983.3098052"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/23\/5098\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:36:24Z","timestamp":1760189784000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/23\/5098"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,21]]},"references-count":50,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["s19235098"],"URL":"https:\/\/doi.org\/10.3390\/s19235098","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,11,21]]}}}