{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,22]],"date-time":"2025-12-22T14:39:40Z","timestamp":1766414380982,"version":"build-2065373602"},"reference-count":49,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2019,11,24]],"date-time":"2019-11-24T00:00:00Z","timestamp":1574553600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Foreground detection is an important theme in video surveillance. Conventional background modeling approaches build sophisticated temporal statistical model to detect foreground based on low-level features, while modern semantic\/instance segmentation approaches generate high-level foreground annotation, but ignore the temporal relevance among consecutive frames. In this paper, we propose a Spatio-Temporal Attention Model (STAM) for cross-scene foreground detection. To fill the semantic gap between low and high level features, appearance and optical flow features are synthesized by attention modules via the feature learning procedure. Experimental results on CDnet 2014 benchmarks validate it and outperformed many state-of-the-art methods in seven evaluation metrics. With the attention modules and optical flow, its F-measure increased     9 %     and     6 %     respectively. The model without any tuning showed its cross-scene generalization on Wallflower and PETS datasets. The processing speed was 10.8 fps with the frame size 256 by 256.<\/jats:p>","DOI":"10.3390\/s19235142","type":"journal-article","created":{"date-parts":[[2019,11,25]],"date-time":"2019-11-25T03:10:00Z","timestamp":1574651400000},"page":"5142","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Spatio-Temporal Attention Model for Foreground Detection in Cross-Scene Surveillance Videos"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2784-3449","authenticated-orcid":false,"given":"Dong","family":"Liang","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiaxing","family":"Pan","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Han","family":"Sun","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huiyu","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Informatics, University of Leicester, Leicester LE1 7RH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,11,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1709","DOI":"10.1109\/TIP.2010.2101613","article-title":"ViBe: A Universal Background Subtraction Algorithm for Video Sequences","volume":"20","author":"Barnich","year":"2011","journal-title":"IEEE Trans. Image Process."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1109\/JPROC.2002.801448","article-title":"Background and foreground modeling using nonparametric kernel density estimation for visual surveillance","volume":"90","author":"Elgammal","year":"2002","journal-title":"Proc. IEEE"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.rti.2004.12.004","article-title":"Real-time foreground\u2013background segmentation using codebook model","volume":"11","author":"Kim","year":"2005","journal-title":"Real-Time Imaging"},{"key":"ref_4","unstructured":"Wang, H., and Suter, D. (2006, January 20\u201324). Background subtraction based on a robust consensus method. Proceedings of the 18th International Conference on Pattern Recognition (ICPR\u201906), Hong Kong, China."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1109\/TPAMI.2006.68","article-title":"A texture-based method for modeling the background and detecting moving objects","volume":"28","author":"Heikkila","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","unstructured":"Learned-Miller, E.G., Narayana, M., and Hanson, A. (2012, January 16\u201321). Background modeling using adaptive pixelwise kernel variances in a hybrid feature space. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liao, S., Zhao, G., Kellokumpu, V., Pietikainen, M., and Li, S.Z. (2010, January 13\u201318). Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539817"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1778","DOI":"10.1109\/TPAMI.2005.213","article-title":"Bayesian modeling of dynamic scenes for object detection","volume":"27","author":"Sheikh","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1296","DOI":"10.1016\/j.patcog.2010.11.022","article-title":"Object detection based on a robust and accurate statistical multi-point-pair model","volume":"44","author":"Zhao","year":"2011","journal-title":"Pattern Recognit."},{"key":"ref_10","first-page":"1478","article-title":"NIC: A Robust Background Extraction Algorithm for Foreground Detection in Dynamic Scenes","volume":"27","author":"Banos","year":"2016","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chen, M., Yang, Q., Li, Q., Wang, G., and Yang, M.H. (2014, January 6\u201312). Spatiotemporal Background Subtraction Using Minimum Spanning Tree and Optical Flow. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10584-0_34"},{"key":"ref_12","unstructured":"Larochelle, H., and Hinton, G.E. (2010). Learning to combine foveal glimpses with a third-order Boltzmann machine. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_13","unstructured":"Kim, J., Lee, S., Kwak, D., Heo, M., Kim, J., Ha, J., and Zhang, B. (2016). Multimodal Residual Learning for Visual QA. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Goyette, N., Jodoin, P.M., Porikli, F., Konrad, J., and Ishwar, P. (2012, January 16\u201321). Changedetection.net: A new change detection benchmark dataset. Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.","DOI":"10.1109\/CVPRW.2012.6238919"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Spagnolo, P., D\u2019Orazio, T., Leo, M., and Distante, A. (2005, January 5\u20138). Advances in Background Updating and Shadow Removing for Motion Detection Algorithms. Proceedings of the International Conference on Computer Analysis of Images and Patterns, Versailles, France.","DOI":"10.1007\/11556121_49"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1016\/j.imavis.2006.01.001","article-title":"Moving Object Segmentation by Background Subtraction and Temporal Analysis","volume":"24","author":"Spagnolo","year":"2006","journal-title":"Image Vision Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"780","DOI":"10.1109\/34.598236","article-title":"Pfinder: Real-time tracking of the human body","volume":"19","author":"Wren","year":"1997","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_18","unstructured":"Stauffer, C., and Grimson, W.E.L. (1999, January 23\u201325). Adaptive background mixture models for real-time tracking. Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), Fort Collins, CO, USA."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Rittscher, J., Kato, J., Joga, S., and Blake, A. (2000). A probabilistic background model for tracking. European Conference on Computer Vision, Springer.","DOI":"10.1007\/3-540-45053-X_22"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1109\/34.868684","article-title":"A Bayesian computer vision system for modeling human interactions","volume":"22","author":"Oliver","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-010-0399-6","article-title":"Incremental tensor subspace learning and its applications to foreground segmentation and tracking","volume":"91","author":"Hu","year":"2011","journal-title":"Int. J. Comput. Vis."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhao, X., Satoh, Y., Takauji, H., Kaneko, S., Iwata, K., and Ozaki, R. (September, January 30). Robust adapted object detection under complex environment. Proceedings of the 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Klagenfurt, Austria.","DOI":"10.1109\/AVSS.2011.6027334"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liang, D., Kaneko, S., Sun, H., and Kang, B. (2018, January 17\u201320). Adaptive local spatial modeling for online change detection under abrupt dynamic background. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.","DOI":"10.1109\/ICIP.2017.8296636"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1374","DOI":"10.1016\/j.patcog.2014.10.020","article-title":"Co-occurrence probability-based pixel pairs background model for robust object detection in dynamic scenes","volume":"48","author":"Liang","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.sigpro.2019.02.021","article-title":"Foreground detection based on co-occurrence background model with hypothesis on degradation modification in dynamic scenes","volume":"60","author":"Zhou","year":"2019","journal-title":"Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Dai, J., He, K., and Sun, J. (2016, January 27\u201330). Instance-Aware Semantic Segmentation via Multi-task Network Cascades. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.343"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Pinheiro, P.H.O., Lin, T., Collobert, R., and Dollar, P. (2016, January 8\u201316). Learning to Refine Object Segments. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_5"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Kirillov, A., Levinkov, E., Andres, B., Savchynskyy, B., and Rother, C. (2017, January 21\u201326). InstanceCut: From Edges to Instances with MultiCut. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.774"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Bai, M., and Urtasun, R. (2017, January 21\u201326). Deep Watershed Transform for Instance Segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.305"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Dollar, P., and Girshick, R.B. (2017, January 22\u201326). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE TPAMI"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Braham, M., and Droogenbroeck, M.V. (2016, January 23\u201325). Deep background subtraction with scene-specific convolutional neural networks. Proceedings of the 2016 International Conference on Systems, Signals and Image Processing (IWSSIP), Bratislava, Slovakia.","DOI":"10.1109\/IWSSIP.2016.7502717"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1016\/j.patcog.2017.09.040","article-title":"A deep convolutional neural network for video sequence background subtraction","volume":"76","author":"Babaee","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.patrec.2016.09.014","article-title":"Interactive deep learning method for segmenting moving objects","volume":"96","author":"Wang","year":"2017","journal-title":"Pattern Recognit. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1016\/j.patrec.2019.06.006","article-title":"Foreground detection by probabilistic modeling of the features discovered by stacked denoising autoencoders in noisy video sequences","volume":"125","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_36","unstructured":"Mnih, V., Heess, N., Graves, A., and Kavukcuoglu, K. (2014). Recurrent Models of Visual Attention. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_37","unstructured":"Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A.C., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015, January 6\u201311). Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. Proceedings of the 32nd International Conference on Machine Learning, Lille, France."},{"key":"ref_38","unstructured":"Li, H., Xiong, P., An, J., and Wang, L. (2018). Pyramid Attention Network for Semantic Segmentation. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-Image Translation with Conditional Adversarial Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_41","unstructured":"Liu, C. (2009). Beyond Pixels: Exploring New Representations and Applications for Motion Analysis. [Ph.D. Thesis, Massachusetts Institute of Technology]."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Hu, M., Ali, S., and Shah, M. (2008, January 8\u201311). Learning motion patterns in crowded scenes using motion flow field. Proceedings of the 2008 19th International Conference on Pattern Recognition, Tampa, FL, USA.","DOI":"10.1109\/ICPR.2008.4761183"},{"key":"ref_43","unstructured":"Lucas, B.D., and Kanade, T. (1981, January 24\u201328). An Iterative Image Registration Technique with an Application to Stereo Vision (IJCAI). Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI \u201981), British, Columbia."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Toyama, K., Krumm, J., Brumitt, B., and Meyers, B. (1999, January 20\u201327). Wallflower: Principles and practice of background maintenance. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece.","DOI":"10.1109\/ICCV.1999.791228"},{"key":"ref_45","unstructured":"(2015, November 10). Performance Evaluation of PETS Surveillance Dataset 2001. Available online: http:\/\/limu.ait.kyushuu.ac.jp\/en\/dataset\/."},{"key":"ref_46","first-page":"359","article-title":"SuBSENSE: A universal change detection method with local adaptive sensitivity","volume":"24","author":"Robert","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1109\/TEVC.2017.2694160","article-title":"Combination of Video Change Detection Algorithms by Genetic Programming","volume":"21","author":"Bianco","year":"2017","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"St-Charles, P.L., Bilodeau, G.A., and Bergevin, R. (2015, January 5\u20139). A Self-Adjusting Approach to Change Detection Based on Background Word Consensus. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV.2015.137"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Hofmann, M., Tiefenbacher, P., and Rigoll, G. (2012, January 16\u201321). Background segmentation with feedback: The Pixel-Based Adaptive Segmenter. Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.","DOI":"10.1109\/CVPRW.2012.6238925"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/23\/5142\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:37:04Z","timestamp":1760189824000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/23\/5142"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,24]]},"references-count":49,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["s19235142"],"URL":"https:\/\/doi.org\/10.3390\/s19235142","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,11,24]]}}}