{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T05:34:56Z","timestamp":1772516096827,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,11,8]],"date-time":"2024-11-08T00:00:00Z","timestamp":1731024000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Japan Society for the Promotion of Science (JSPS) KAKENHI","award":["JP22H00540"],"award-info":[{"award-number":["JP22H00540"]}]},{"name":"Japan Society for the Promotion of Science (JSPS) KAKENHI","award":["JP23K11176"],"award-info":[{"award-number":["JP23K11176"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Super-resolution is a technique for generating a high-resolution image or video from a low-resolution counterpart by predicting natural and realistic texture information. It has various applications such as medical image analysis, surveillance, remote sensing, etc. However, traditional single-image super-resolution methods can lead to a blurry visual effect. Reference-based super-resolution methods have been proposed to recover detailed information accurately. In reference-based methods, a high-resolution image is also used as a reference in addition to the low-resolution input image. Reference-based methods aim at transferring high-resolution textures from the reference image to produce visually pleasing results. However, it requires texture alignment between low-resolution and reference images, which generally requires a lot of time and memory. This paper proposes a lightweight reference-based video super-resolution method using deformable convolution. The proposed method makes the reference-based super-resolution a technology that can be easily used even in environments with limited computational resources. To verify the effectiveness of the proposed method, we conducted experiments to compare the proposed method with baseline methods in two aspects: runtime and memory usage, in addition to accuracy. The experimental results showed that the proposed method restored a high-quality super-resolved image from a very low-resolution level in 0.0138 s using two NVIDIA RTX 2080 GPUs, much faster than the representative method.<\/jats:p>","DOI":"10.3390\/info15110718","type":"journal-article","created":{"date-parts":[[2024,11,8]],"date-time":"2024-11-08T04:02:54Z","timestamp":1731038574000},"page":"718","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Lightweight Reference-Based Video Super-Resolution Using Deformable Convolution"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5205-0542","authenticated-orcid":false,"given":"Tomo","family":"Miyazaki","sequence":"first","affiliation":[{"name":"Graduate School of Engineering, Tohoku University, Sendai 9808579, Japan"}]},{"given":"Zirui","family":"Guo","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Tohoku University, Sendai 9808579, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7706-9995","authenticated-orcid":false,"given":"Shinichiro","family":"Omachi","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Tohoku University, Sendai 9808579, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.procs.2020.06.005","article-title":"Super-resolution using GANs for medical imaging","volume":"173","author":"Gupta","year":"2020","journal-title":"Procedia Comput. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"848","DOI":"10.1016\/j.sigpro.2009.09.002","article-title":"A super-resolution reconstruction algorithm for surveillance images","volume":"90","author":"Zhang","year":"2010","journal-title":"Signal Process."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"104110","DOI":"10.1016\/j.earscirev.2022.104110","article-title":"A comprehensive review on deep learning based remote sensing image super-resolution methods","volume":"232","author":"Wang","year":"2022","journal-title":"Earth-Sci. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Pan, L., Chen, L., Zhu, S., Tong, W., and Guo, L. (2022). Research on small sample data-driven inspection technology of UAV for transmission line insulator defect detection. Information, 13.","DOI":"10.3390\/info13060276"},{"key":"ref_5","unstructured":"Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Weinberger, K. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zheng, H., Ji, M., Wang, H., Liu, Y., and Fang, L. (2018, January 8\u201314). CrossNet: An end-to-end reference-based super resolution network using cross-scale warping. Proceedings of the Computer Vision\u2014ECCV 2018, Munich, Germany.","DOI":"10.1007\/978-3-030-01231-1_6"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Wang, Z., Lin, Z., and Qi, H. (2019, January 15\u201320). Image super-resolution by neural texture Transfer. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00817"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Dosovitskiy, A., Fischer, P., Ilg, E., H\u00e4usser, P., Hazirbas, C., Golkov, V., Smagt, P.V.D., Cremers, D., and Brox, T. (2015, January 7\u201313). FlowNet: Learning optical flow with convolutional networks. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.316"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yang, F., Yang, H., Fu, J., Lu, H., and Guo, B. (2020, January 14\u201319). Learning texture Transformer network for image super-resolution. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, DC, USA.","DOI":"10.1109\/CVPR42600.2020.00583"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22\u201329). Deformable convolutional networks. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.89"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhu, X., Hu, H., Lin, S., and Dai, J. (2019, January 15\u201320). Deformable ConvNets V2: More deformable, better results. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00953"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dong, C., Loy, C.C., He, K., and Tang, X. (2014, January 6\u201312). Learning a deep convolutional network for image super-resolution. Proceedings of the Computer Vision\u2014ECCV 2014, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10593-2_13"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Agustsson, E., and Timofte, R. (2017, January 21\u201326). NTIRE 2017 challenge on single image super-resolution: Dataset and study. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.150"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lim, B., Son, S., Kim, H., Nah, S., and Lee, K.M. (2017, January 21\u201326). Enhanced deep residual networks for single image super-resolution. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.151"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., and Fu, Y. (2018, January 8\u201314). Image super-resolution using very deep residual channel attention networks. Proceedings of the Computer Vision\u2014ECCV 2018, Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_18"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Johnson, J., Alahi, A., and Fei-Fei, L. (2016, January 11\u201314). Perceptual losses for real-time style transfer and super-resolution. Proceedings of the Computer Vision\u2014 ECCV 2016, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ledig, C., Theis, L., Husz\u00e1r, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21\u201326). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.19"},{"key":"ref_19","unstructured":"Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_20","unstructured":"Kolesnikov, A., Dosovitskiy, A., Weissenborn, D., Heigold, G., Uszkoreit, J., Beyer, L., Minderer, M., Dehghani, M., Houlsby, N., and Gelly, S. (2021, January 3\u20137). An image is worth 16 \u00d7 16 words: Transformers for image recognition at scale. Proceedings of the International Conference on Learning Representations (ICLR) 2021, Virtual."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., and Timofte, R. (2021, January 11\u201317). SwinIR: Image restoration using Swin Transformer. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00210"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10\u201317). Swin Transformer: Hierarchical vision Transformer using shifted windows. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yao, X., Pan, Y., and Wang, J. (2024). An omnidirectional image super-resolution method based on enhanced SwinIR. Information, 15.","DOI":"10.3390\/info15050248"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"108035","DOI":"10.1016\/j.engappai.2024.108035","article-title":"Efficient mixed transformer for single image super-resolution","volume":"133","author":"Zheng","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Haris, M., Shakhnarovich, G., and Ukita, N. (2019, January 15\u201320). Recurrent back-projection network for video super-resolution. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00402"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chan, K.C., Wang, X., Yu, K., Dong, C., and Loy, C.C. (2021, January 19\u201325). BasicVSR: The search for essential components in video super-resolution and beyond. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual.","DOI":"10.1109\/CVPR46437.2021.00491"},{"key":"ref_27","unstructured":"De Brabandere, B., Jia, X., Tuytelaars, T., and Van Gool, L. (2016, January 5\u201310). Dynamic filter networks. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jo, Y., Oh, S.W., Kang, J., and Kim, S.J. (2018, January 18\u201323). Deep Video super-resolution network using dynamic upsampling filters without explicit motion compensation. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00340"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Tian, Y., Zhang, Y., Fu, Y., and Xu, C. (2020, January 14\u201319). TDAN: Temporally-deformable alignment network for video super-resolution. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, DC, USA.","DOI":"10.1109\/CVPR42600.2020.00342"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, X., Chan, K.C., Yu, K., Dong, C., and Loy, C.C. (2019, January 16\u201317). EDVR: Video restoration with enhanced deformable convolutional networks. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00247"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chan, K.C., Wang, X., Yu, K., Dong, C., and Loy, C.C. (2021, January 2\u20139). Understanding deformable alignment in video super-resolution. Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, Virtual.","DOI":"10.1609\/aaai.v35i2.16181"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2116","DOI":"10.1109\/TCSVT.2022.3222875","article-title":"Optical flow reusing for high-efficiency space-time video super resolution","volume":"33","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4855","DOI":"10.1109\/TCSVT.2023.3250443","article-title":"Deep vdeo super-resolution using hybrid imaging system","volume":"33","author":"Feng","year":"2023","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"54649","DOI":"10.1109\/ACCESS.2022.3174194","article-title":"DSMA: Reference-based image super-resolution method based on dual-view supervised learning and multi-attention mechanism","volume":"10","author":"Liu","year":"2022","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Shim, G., Park, J., and Kweon, I.S. (2020, January 13\u201319). Robust reference-based super-resolution with similarity-aware deformable convolution. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00845"},{"key":"ref_36","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference on Learning Representations (ICLR) 2015, Diego, CA, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Gatys, L.A., Ecker, A.S., and Bethge, M. (2016, January 27\u201330). Image style transfer using convolutional neural networks. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.265"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Shi, W., Caballero, J., Husz\u00e1r, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., and Wang, Z. (2016, January 27\u201330). Real-time single image and video super-resolution using an efficient sub-pixel convolutional Neural Network. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.207"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lai, W.S., Huang, J.B., Ahuja, N., and Yang, M.H. (2017, January 21\u201326). Deep Laplacian pyramid networks for fast and accurate super-resolution. Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.618"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Nah, S., Baik, S., Hong, S., Moon, G., Son, S., Timofte, R., and Lee, K.M. (2019, January 16\u201317). NTIRE 2019 challenge on video deblurring and super-resolution: Dataset and study. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00251"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1109\/TPAMI.2013.127","article-title":"On Bayesian adaptive video super resolution","volume":"36","author":"Liu","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/11\/718\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:28:39Z","timestamp":1760113719000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/11\/718"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,8]]},"references-count":41,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["info15110718"],"URL":"https:\/\/doi.org\/10.3390\/info15110718","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,8]]}}}