{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,7]],"date-time":"2025-08-07T20:44:54Z","timestamp":1754599494586,"version":"3.37.3"},"reference-count":21,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2021,9,6]],"date-time":"2021-09-06T00:00:00Z","timestamp":1630886400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,9,6]],"date-time":"2021-09-06T00:00:00Z","timestamp":1630886400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Vis Comput"],"published-print":{"date-parts":[[2022,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Unmanned aerial vehicles (UAV) can be used to great effect for wide-area searches such as search and rescue operations. UAV enable search and rescue teams to cover large areas more efficiently and in less time. However, using UAV for this purpose involves the creation of large amounts of data, typically in video format, which must be analysed before any potential findings can be uncovered and actions taken. This is a slow and expensive process which can result in significant delays to the response time after a target is seen by the UAV. To solve this problem we propose a deep model architecture using a visual saliency approach to automatically analyse and detect anomalies in UAV video. Our Temporal Contextual Saliency (TeCS) approach is based on the state-of-the-art in visual saliency detection using deep Convolutional Neural Networks (CNN) and considers local and scene context, with novel additions in utilizing temporal information through a convolutional Long Short-Term Memory (LSTM) layer and modifications to the base model architecture. We additionally evaluate the impact of temporal vs non-temporal reasoning for this task. Our model achieves improved results on a benchmark dataset with the addition of temporal reasoning showing significantly improved results compared to the state-of-the-art in saliency detection.<\/jats:p>","DOI":"10.1007\/s00371-021-02264-6","type":"journal-article","created":{"date-parts":[[2021,9,6]],"date-time":"2021-09-06T15:03:05Z","timestamp":1630940585000},"page":"2033-2040","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Temporal and non-temporal contextual saliency analysis for generalized wide-area search within unmanned aerial vehicle (UAV) video"],"prefix":"10.1007","volume":"38","author":[{"given":"Simon G. E.","family":"G\u00f6kstorp","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1666-7590","authenticated-orcid":false,"given":"Toby P.","family":"Breckon","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,9,6]]},"reference":[{"key":"2264_CR1","doi-asserted-by":"crossref","unstructured":"Itti, L., Koch, C., Niebur, E.: A model of saliency\u2013based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254\u20131259 (1998)","DOI":"10.1109\/34.730558"},{"key":"2264_CR2","unstructured":"Sokalski, J., Breckon, T. P., Cowling, I.: Automatic salient object detection in UAV Imagery. In: Proc. 25th International Conference on Unmanned Air Vehicle Systems, pp. 11.1\u201311.12 (2010)"},{"key":"2264_CR3","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Su, A., Zhu, X., Zhang, X., Shang, Y.: Salient Object detection approach in UAV video. In: Proc. SPIE Automatic Target Recognition and Navigation, vol. 8918, p. 89180Y (2013)","DOI":"10.1117\/12.2032141"},{"key":"2264_CR4","doi-asserted-by":"crossref","unstructured":"Gotovac, S., Papi\u0107, V., Maru\u0161i\u0107, \u017d.: Analysis of saliency object detection algorithms for search and rescue operations. In: Proc. International Conference on Software, Telecommunications and Computer Networks, pp. 1\u20136 (2016)","DOI":"10.1109\/SOFTCOM.2016.7772118"},{"key":"2264_CR5","doi-asserted-by":"crossref","unstructured":"Liu, N., Han, J.: A deep spatial contextual long\u2013term recurrent convolutional network for saliency detection. IEEE Trans. Image Process. 27(7), 3264\u20133274 (2018)","DOI":"10.1109\/TIP.2018.2817047"},{"key":"2264_CR6","doi-asserted-by":"crossref","unstructured":"Mueller, M., Smith, N., Ghanem, B.: A benchmark and simulator for UAV tracking. In: Proc. of the European Conference on Computer Vision (ECCV) (2016)","DOI":"10.1007\/978-3-319-46448-0_27"},{"key":"2264_CR7","doi-asserted-by":"crossref","unstructured":"Wang, C., Yang, B.: Saliency\u2013guided object proposal for refined salient region detection. In: Proc. Visual Communications and Image Processing, pp. 1\u20134 (2016)","DOI":"10.1109\/VCIP.2016.7805479"},{"key":"2264_CR8","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Wang, X., Xie, X., Li, Y.: Salient object detection via recursive sparse representation. Remote Sens. 10(4), 652 (2018)","DOI":"10.3390\/rs10040652"},{"key":"2264_CR9","doi-asserted-by":"crossref","unstructured":"Wang, L., Xue, J., Zheng, N., Hua, G.: Automatic salient object extraction with contextual cue. In: Proc. International Conference on Computer Vision, pp. 105\u2013112 (2011)","DOI":"10.1109\/ICCV.2011.6126231"},{"key":"2264_CR10","doi-asserted-by":"crossref","unstructured":"Luo, Z., Mishra, A.K., Achkar, A., Eichel, J.A., Li, S., Jodoin, P.-M.: Non-local deep features for salient object detection. In: Proc. Computer Vision and Pattern Recognition, pp. 6593\u20136601 (2017)","DOI":"10.1109\/CVPR.2017.698"},{"key":"2264_CR11","doi-asserted-by":"crossref","unstructured":"Bo\u017ei\u0107-\u0160tuli\u0107, D., Maru\u0161i\u0107, \u017d, Gotovac, S.: Deep learning approach in aerial imagery for supporting land search and rescue missions. Int. J. Comput. Vis. 1\u201323 (2019)","DOI":"10.1007\/s11263-019-01177-1"},{"issue":"1","key":"2264_CR12","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1109\/TMM.2012.2225034","volume":"15","author":"N Imamoglu","year":"2013","unstructured":"Imamoglu, N., Lin, W., Fang, Y.: A saliency detection model using low-level features based on wavelet transform. IEEE Trans. Multimed. 15(1), 96\u2013105 (2013)","journal-title":"IEEE Trans. Multimed."},{"key":"2264_CR13","doi-asserted-by":"crossref","unstructured":"Ren, S., He, K., Girshick, R., Sun, J.: Faster r\u2013cnn: towards realtime object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137\u20131149 (2016)","DOI":"10.1109\/TPAMI.2016.2577031"},{"key":"2264_CR14","doi-asserted-by":"crossref","unstructured":"Azaza,A., Douik, A.: Deep saliency features for video saliency prediction. In: Proc. International Conference on Advanced Systems and Electric Technologies, pp. 355\u2013359 (2018)","DOI":"10.1109\/ASET.2018.8379878"},{"key":"2264_CR15","doi-asserted-by":"crossref","unstructured":"Song, H., Wang, W., Zhao, S., Shen, J., Lam, K.-M.: PyramidYilated deeper convLSTM for video salient object detection. In: Proc. European Conference in Computer Vision, pp. 744\u2013760. Springer (2018)","DOI":"10.1007\/978-3-030-01252-6_44"},{"key":"2264_CR16","unstructured":"Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.K., Woo, W.-C.: Convolutional LSTM network: a machine learning approach for Precipitation nowcasting. In: Proc. Advances in Neural Information Processing Systems, pp. 802\u2013810 (2015)"},{"key":"2264_CR17","doi-asserted-by":"crossref","unstructured":"Perrin, A.-F., Zhang, L., Le Meur, O.: How well current saliency prediction models perform on uavs videos? In: Proc. International Conference on Computer Analysis of Images and Patterns, pp. 311\u2013323. Springer (2019)","DOI":"10.1007\/978-3-030-29888-3_25"},{"key":"2264_CR18","doi-asserted-by":"crossref","unstructured":"Peters, R.J., Iyer, A., Itti, L., Koch, C.: Components of bottom\u2013up gaze allocation in natural images. Vis. Res. 45(18), 2397\u20132416 (2005)","DOI":"10.1016\/j.visres.2005.03.019"},{"issue":"3","key":"2264_CR19","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1109\/TPAMI.2018.2815601","volume":"41","author":"Z Bylinskii","year":"2019","unstructured":"Bylinskii, Z., Judd, T., Oliva, A., Torralba, A., Durand, F.: What do different evaluation metrics tell us about saliency models? IEEE Trans. Pattern Anal. Mach. Intell. 41(3), 740\u2013757 (2019)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"2264_CR20","unstructured":"Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR, arXiv:1412.6980 (2015)"},{"key":"2264_CR21","doi-asserted-by":"crossref","unstructured":"Krassanakis, V., Perreira Da Silva, M., Ricordel, V.: Monitoring human visual behavior during the observation of unmanned aerial vehicles videos. Drones 2(4) (2018)","DOI":"10.3390\/drones2040036"}],"container-title":["The Visual Computer"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00371-021-02264-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00371-021-02264-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00371-021-02264-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,7]],"date-time":"2024-09-07T21:22:37Z","timestamp":1725744157000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00371-021-02264-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,6]]},"references-count":21,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,6]]}},"alternative-id":["2264"],"URL":"https:\/\/doi.org\/10.1007\/s00371-021-02264-6","relation":{},"ISSN":["0178-2789","1432-2315"],"issn-type":[{"type":"print","value":"0178-2789"},{"type":"electronic","value":"1432-2315"}],"subject":[],"published":{"date-parts":[[2021,9,6]]},"assertion":[{"value":"14 July 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 September 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest in the submission of this article for publication.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}