{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T12:05:47Z","timestamp":1777982747616,"version":"3.51.4"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,3,14]],"date-time":"2024-03-14T00:00:00Z","timestamp":1710374400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,14]],"date-time":"2024-03-14T00:00:00Z","timestamp":1710374400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001778","name":"Deakin University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001778","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["SN COMPUT. SCI."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>This paper presents a novel approach to video super-resolution (VSR) by focusing on the selection of input frames, a process critical to VSR. VSR methods typically rely on deep learning techniques, those that are able to learn features from a large dataset of low-resolution (LR) and corresponding high-resolution (HR) videos and generate high-quality HR frames from any new LR input frames using the learned features. However, these methods often use as input the immediate neighbouring frames to a given target frame without considering the importance and dynamics of the frames across the temporal dimension of a video. This work aims to address the limitations of the conventional sliding-window mechanisms by developing input frame selection algorithms. By dynamically selecting the most representative neighbouring frames based on content-aware selection measures, our proposed algorithms enable VSR models to extract more informative and accurate features that are better aligned with the target frame, leading to improved performance and higher-quality HR frames. Through an empirical study, we demonstrate that the proposed dynamic content-aware selection mechanism improves super-resolution results without any additional architectural overhead, offering a counter-intuitive yet effective alternative to the long-established trend of increasing architectural complexity to improve VSR results.<\/jats:p>","DOI":"10.1007\/s42979-024-02710-x","type":"journal-article","created":{"date-parts":[[2024,3,14]],"date-time":"2024-03-14T11:05:50Z","timestamp":1710414350000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Frame Selection Using Spatiotemporal Dynamics and Key Features as Input Pre-processing for Video Super-Resolution Models"],"prefix":"10.1007","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9359-6506","authenticated-orcid":false,"given":"Arbind","family":"Agrahari Baniya","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4176-2215","authenticated-orcid":false,"given":"Tsz-Kwan","family":"Lee","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2313-8603","authenticated-orcid":false,"given":"Peter","family":"Eklund","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6639-6824","authenticated-orcid":false,"given":"Sunil","family":"Aryal","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,3,14]]},"reference":[{"key":"2710_CR1","doi-asserted-by":"crossref","unstructured":"Wang Z, Chen J, Hoi SC. Deep learning for image super-resolution: A survey. IEEE transactions on pattern analysis and machine intelligence; 2020.","DOI":"10.1109\/TPAMI.2020.2982166"},{"key":"2710_CR2","unstructured":"Arefin MR, Michalski V, St-Charles P-L, Kalaitzis A, Kim S, Kahou SE, Bengio Y. Multi-image super-resolution for remote sensing using deep recurrent networks. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020;pp. 206\u2013207."},{"key":"2710_CR3","doi-asserted-by":"publisher","unstructured":"Liang M, Du J, Li L, Xue Z, Wang X, Kou F, Wang X. Video super-resolution reconstruction based on deep learning and spatio-temporal feature self-similarity. IEEE Transactions on Knowledge and Data Engineering. 2020;1\u20131. https:\/\/doi.org\/10.1109\/TKDE.2020.3034261.","DOI":"10.1109\/TKDE.2020.3034261"},{"key":"2710_CR4","doi-asserted-by":"publisher","first-page":"106049","DOI":"10.1109\/ACCESS.2021.3098326","volume":"9","author":"Z-S Liu","year":"2021","unstructured":"Liu Z-S, Siu W-C, Chan Y-L. Efficient video super-resolution via hierarchical temporal residual networks. IEEE Access. 2021;9:106049\u201364. https:\/\/doi.org\/10.1109\/ACCESS.2021.3098326.","journal-title":"IEEE Access"},{"key":"2710_CR5","unstructured":"Liu H, Ruan Z, Zhao P, Dong C, Shang F, Liu Y, Yang L. Video super resolution based on deep learning: A comprehensive survey. arXiv preprint arXiv:2007.12928 2020."},{"key":"2710_CR6","doi-asserted-by":"crossref","unstructured":"Haris M, Shakhnarovich G, Ukita N. Recurrent back-projection network for video super-resolution. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 2019;pp. 3897\u20133906.","DOI":"10.1109\/CVPR.2019.00402"},{"key":"2710_CR7","doi-asserted-by":"crossref","unstructured":"Wang X, Chan KC, Yu K, Dong C, Change Loy C. Edvr: Video restoration with enhanced deformable convolutional networks. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2019;pp. 0\u20130.","DOI":"10.1109\/CVPRW.2019.00247"},{"key":"2710_CR8","doi-asserted-by":"crossref","unstructured":"Jo Y, Oh SW, Kang J, Kim SJ. Deep video super-resolution network using dynamic upsampling filters without explicit motion compensation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018;pp. 3224\u20133232.","DOI":"10.1109\/CVPR.2018.00340"},{"issue":"3","key":"2710_CR9","doi-asserted-by":"publisher","first-page":"933","DOI":"10.1109\/TPAMI.2019.2941941","volume":"43","author":"W Bao","year":"2021","unstructured":"Bao W, Lai W-S, Zhang X, Gao Z, Yang M-H. Memc-net: Motion estimation and motion compensation driven neural network for video interpolation and enhancement. IEEE Trans Pattern Anal Mach Intell. 2021;43(3):933\u201348. https:\/\/doi.org\/10.1109\/TPAMI.2019.2941941.","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2710_CR10","doi-asserted-by":"crossref","unstructured":"Tian Y, Zhang Y, Fu Y, Xu C. Tdan: Temporally-deformable alignment network for video super-resolution. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2020.","DOI":"10.1109\/CVPR42600.2020.00342"},{"key":"2710_CR11","doi-asserted-by":"crossref","unstructured":"Isobe T, Li S, Jia X, Yuan S, Slabaugh G, Xu C, Li Y-L, Wang S, Tian Q. Video super-resolution with temporal group attention. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2020.","DOI":"10.1109\/CVPR42600.2020.00803"},{"key":"2710_CR12","doi-asserted-by":"crossref","unstructured":"Chan KCK, Wang X, Yu K, Dong C, Loy CC. Basicvsr: The search for essential components in video super-resolution and beyond. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021;pp. 4947\u20134956.","DOI":"10.1109\/CVPR46437.2021.00491"},{"key":"2710_CR13","doi-asserted-by":"publisher","first-page":"645","DOI":"10.1007\/978-3-030-58610-2_38","volume-title":"Computer Vision - ECCV 2020","author":"T Isobe","year":"2020","unstructured":"Isobe T, Jia X, Gu S, Li S, Wang S, Tian Q. Video super-resolution with recurrent structure-detail network. In: Vedaldi A, Bischof H, Brox T, Frahm J-M, editors. Computer Vision - ECCV 2020. Cham: Springer; 2020. p. 645\u201360."},{"key":"2710_CR14","doi-asserted-by":"crossref","unstructured":"Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y. Deformable convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision. 2017;pp. 764\u2013773.","DOI":"10.1109\/ICCV.2017.89"},{"key":"2710_CR15","doi-asserted-by":"publisher","unstructured":"Wu Y, Zhang L, Gu Z, Lu H, Wan S. Edge-ai-driven framework with efficient mobile network design for facial expression recognition. ACM Trans Embed Comput Syst. 2023;22(3). https:\/\/doi.org\/10.1145\/3587038.","DOI":"10.1145\/3587038"},{"key":"2710_CR16","doi-asserted-by":"publisher","unstructured":"Wu Y, Guo H, Chakraborty C, Khosravi M, Berretti S, Wan S. Edge computing driven low-light image dynamic enhancement for object detection. IEEE Transactions on Network Science and Engineering. 2022;1\u20131. https:\/\/doi.org\/10.1109\/TNSE.2022.3151502.","DOI":"10.1109\/TNSE.2022.3151502"},{"key":"2710_CR17","doi-asserted-by":"publisher","unstructured":"Agrahari Baniya A, Lee, T-K, Eklund P, Aryal S, Robles-Kelly A. Online video super-resolution using unidirectional recurrent model. 2022. https:\/\/doi.org\/10.36227\/techrxiv.21500235.v1.","DOI":"10.36227\/techrxiv.21500235.v1"},{"key":"2710_CR18","doi-asserted-by":"crossref","unstructured":"Sajjadi MS, Vemulapalli R, Brown M. Frame-recurrent video super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018;pp. 6626\u20136634.","DOI":"10.1109\/CVPR.2018.00693"},{"key":"2710_CR19","doi-asserted-by":"publisher","unstructured":"Agrahari Baniya A, Lee, T-K, Eklund, PW, Aryal S. STIFS: spatio-temporal input frame selection for learning-based video super-resolution models. In: Proceedings of the 19th international conference on signal processing and multimedia applications - SIGMAP. SciTePress; 2022. p. 48\u201358. https:\/\/doi.org\/10.5220\/0011339900003289.","DOI":"10.5220\/0011339900003289"},{"key":"2710_CR20","doi-asserted-by":"crossref","unstructured":"Haris M, Shakhnarovich G, Ukita N. Space-time-aware multi-resolution video enhancement. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2020.","DOI":"10.1109\/CVPR42600.2020.00293"},{"issue":"8","key":"2710_CR21","doi-asserted-by":"publisher","first-page":"1106","DOI":"10.1007\/s11263-018-01144-2","volume":"127","author":"T Xue","year":"2019","unstructured":"Xue T, Chen B, Wu J, Wei D, Freeman WT. Video enhancement with task-oriented flow. Int J Comput Vision. 2019;127(8):1106\u201325.","journal-title":"Int J Comput Vision"},{"key":"2710_CR22","doi-asserted-by":"publisher","first-page":"74711","DOI":"10.1109\/ACCESS.2019.2920774","volume":"7","author":"K-W Hung","year":"2019","unstructured":"Hung K-W, Qiu C, Jiang J. Video super resolution via deep global-aware network. IEEE Access. 2019;7:74711\u201320. https:\/\/doi.org\/10.1109\/ACCESS.2019.2920774.","journal-title":"IEEE Access"},{"key":"2710_CR23","doi-asserted-by":"publisher","first-page":"177734","DOI":"10.1109\/ACCESS.2019.2958030","volume":"7","author":"H Wang","year":"2019","unstructured":"Wang H, Su D, Liu C, Jin L, Sun X, Peng X. Deformable non-local network for video super-resolution. IEEE Access. 2019;7:177734\u201344. https:\/\/doi.org\/10.1109\/ACCESS.2019.2958030.","journal-title":"IEEE Access"},{"issue":"7","key":"2710_CR24","doi-asserted-by":"publisher","first-page":"3312","DOI":"10.1109\/tip.2019.2895768","volume":"28","author":"A Lucas","year":"2019","unstructured":"Lucas A, Lopez-Tapia S, Molina R, Katsaggelos AK. Generative adversarial networks and perceptual losses for video super-resolution. IEEE Trans Image Process. 2019;28(7):3312\u201327. https:\/\/doi.org\/10.1109\/tip.2019.2895768.","journal-title":"IEEE Trans Image Process"},{"key":"2710_CR25","unstructured":"Kim SY, Lim J, Na T, Kim M. 3dsrnet: Video super-resolution using 3d convolutional neural networks. arXiv preprint arXiv:1812.09079 2018."},{"key":"2710_CR26","unstructured":"Isobe T, Zhu F, Jia X, Wang S. Revisiting temporal modeling for video super-resolution. arXiv preprint arXiv:2008.05765 2020."},{"key":"2710_CR27","doi-asserted-by":"crossref","unstructured":"Zhu X, Li Z, Zhang X-Y, Li C, Liu Y, Xue Z. Residual invertible spatio-temporal network for video super-resolution. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 2019;pp. 5981\u20135988.","DOI":"10.1609\/aaai.v33i01.33015981"},{"issue":"2","key":"2710_CR28","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1109\/TPAMI.2013.127","volume":"36","author":"C Liu","year":"2013","unstructured":"Liu C, Sun D. On bayesian adaptive video super resolution. IEEE Trans Pattern Anal Mach Intell. 2013;36(2):346\u201360.","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2710_CR29","doi-asserted-by":"publisher","first-page":"596","DOI":"10.1007\/978-3-540-24672-5_47","volume-title":"Computer Vision - ECCV 2004","author":"DB Russakoff","year":"2004","unstructured":"Russakoff DB, Tomasi C, Rohlfing T, Maurer CR. Image similarity using mutual information of regions. In: Pajdla T, Matas J, editors. Computer Vision - ECCV 2004. Berlin, Heidelberg: Springer; 2004. p. 596\u2013607."},{"issue":"2","key":"2710_CR30","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1117\/1.1455011","volume":"11","author":"I Avcibas","year":"2002","unstructured":"Avcibas I, Sankur B, Sayood K. Statistical evaluation of image quality measures. J Electron Imaging. 2002;11(2):206\u201323. https:\/\/doi.org\/10.1117\/1.1455011.","journal-title":"J Electron Imaging"},{"key":"2710_CR31","doi-asserted-by":"crossref","unstructured":"Hore A, Ziou D. Image quality metrics: Psnr vs. ssim. In 2010 20th International Conference on Pattern Recognition. 2010;pp. 2366\u20132369. IEEE.","DOI":"10.1109\/ICPR.2010.579"},{"key":"2710_CR32","doi-asserted-by":"crossref","unstructured":"Zhang R, Isola P, Efros AA, Shechtman E, Wang O. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018;pp. 586\u2013595.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"2710_CR33","doi-asserted-by":"crossref","unstructured":"Lindeberg T. Scale invariant feature transform; 2012.","DOI":"10.4249\/scholarpedia.10491"},{"key":"2710_CR34","unstructured":"Viswanathan DG. Features from accelerated segment test (fast). In Proceedings of the 10th Workshop on Image Analysis for Multimedia Interactive Services, London, UK. 2009;pp. 6\u20138."},{"key":"2710_CR35","doi-asserted-by":"crossref","unstructured":"Calonder M, Lepetit V, Strecha C, Fua P. Brief: Binary robust independent elementary features. In European Conference on Computer Vision. 2010;pp. 778\u2013792. Springer.","DOI":"10.1007\/978-3-642-15561-1_56"},{"key":"2710_CR36","doi-asserted-by":"crossref","unstructured":"Rublee E, Rabaud V, Konolige K, Bradski G. Orb: An efficient alternative to sift or surf. In 2011 International Conference on Computer Vision. 2011;pp. 2564\u20132571. Ieee.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"2710_CR37","doi-asserted-by":"crossref","unstructured":"Leutenegger S, Chli M, Siegwart RY. Brisk: Binary robust invariant scalable keypoints. In 2011 International Conference on Computer Vision. 2011;pp. 2548\u20132555. Ieee.","DOI":"10.1109\/ICCV.2011.6126542"},{"key":"2710_CR38","unstructured":"Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 2014."},{"key":"2710_CR39","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016,pp. 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"2710_CR40","doi-asserted-by":"publisher","unstructured":"Tareen SAK, Saleem Z. A comparative analysis of sift, surf, kaze, akaze, orb, and brisk. In 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET). 2018;pp. 1\u201310. https:\/\/doi.org\/10.1109\/ICOMET.2018.8346440.","DOI":"10.1109\/ICOMET.2018.8346440"},{"key":"2710_CR41","doi-asserted-by":"publisher","unstructured":"Agrahari Baniya A, Lee T-K, Eklund PW, Aryal S. Omnidirectional video super-resolution using deep learning. 2022. https:\/\/doi.org\/10.36227\/techrxiv.20494851.v1.","DOI":"10.36227\/techrxiv.20494851.v1"},{"key":"2710_CR42","doi-asserted-by":"publisher","unstructured":"Noble FK. Comparison of opencv\u2019s feature detectors and feature matchers. In 2016 23rd International Conference on Mechatronics and Machine Vision in Practice (M2VIP). 2016;pp. 1\u20136. https:\/\/doi.org\/10.1109\/M2VIP.2016.7827292.","DOI":"10.1109\/M2VIP.2016.7827292"},{"issue":"8","key":"2710_CR43","doi-asserted-by":"publisher","first-page":"2503","DOI":"10.1109\/TCSVT.2019.2925844","volume":"30","author":"P Yi","year":"2020","unstructured":"Yi P, Wang Z, Jiang K, Shao Z, Ma J. Multi-temporal ultra dense memory network for video super-resolution. IEEE Trans Circuits Syst Video Technol. 2020;30(8):2503\u201316. https:\/\/doi.org\/10.1109\/TCSVT.2019.2925844.","journal-title":"IEEE Trans Circuits Syst Video Technol"},{"key":"2710_CR44","unstructured":"Sun Y, Lu A, Yu L. Ahg8: Ws-psnr for 360 video objective quality evaluation. In Joint Video Exploration Team of ITU-T SG16 WP3 and ISO\/IEC JTC1\/SC29\/WG11, JVET-D0040, 4th Meeting; 2016."}],"container-title":["SN Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-024-02710-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s42979-024-02710-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-024-02710-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,14]],"date-time":"2024-03-14T11:17:29Z","timestamp":1710415049000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s42979-024-02710-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,14]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["2710"],"URL":"https:\/\/doi.org\/10.1007\/s42979-024-02710-x","relation":{"has-preprint":[{"id-type":"doi","id":"10.36227\/techrxiv.22591426.v1","asserted-by":"object"}]},"ISSN":["2661-8907"],"issn-type":[{"value":"2661-8907","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,14]]},"assertion":[{"value":"27 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Research Involving Human and\/or Animals"}}],"article-number":"323"}}