{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T16:11:49Z","timestamp":1781021509340,"version":"3.54.1"},"reference-count":41,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,1,3]],"date-time":"2020-01-03T00:00:00Z","timestamp":1578009600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61705173"],"award-info":[{"award-number":["61705173"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61675160"],"award-info":[{"award-number":["61675160"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["51801142"],"award-info":[{"award-number":["51801142"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["No.201813019"],"award-info":[{"award-number":["No.201813019"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Project funded by China Postdoctoral Science Foundation","award":["No.2019M652472"],"award-info":[{"award-number":["No.2019M652472"]}]},{"DOI":"10.13039\/501100013314","name":"111 Project","doi-asserted-by":"publisher","award":["B17035"],"award-info":[{"award-number":["B17035"]}],"id":[{"id":"10.13039\/501100013314","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Deep learning-based image super-resolution has shown significantly good performance in improving image quality. In this paper, the RGB-IR cross input and sub-pixel upsampling network is proposed to increase the spatial resolution of an Infrared (IR) image by combining it with a color image of higher spatial resolution obtained with a different imaging modality. Specifically, this is accomplished by fusion of the features map of two RGB-IR inputs in the reconstruction of an infrared image. To improve the accuracy of feature extraction, deconvolution is replaced by sub-pixel convolution to upsample image in the network. Then, the guided filter layer is introduced for image denoising of IR images, and it can preserve the image detail. In addition, the experimental dataset, which is collected by us, contains large numbers of RGB images and corresponding IR images with the same scene. Experimental results on our dataset and other datasets demonstrate that the method is superior to existing methods in accuracy and visual improvement.<\/jats:p>","DOI":"10.3390\/s20010281","type":"journal-article","created":{"date-parts":[[2020,1,3]],"date-time":"2020-01-03T11:55:07Z","timestamp":1578052507000},"page":"281","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["RGB-IR Cross Input and Sub-Pixel Upsampling Network for Infrared Image Super-Resolution"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5855-870X","authenticated-orcid":false,"given":"Juan","family":"Du","sequence":"first","affiliation":[{"name":"School of Physics and Optoelectronic Engineering, Xidian University, No. 2 South Taibai Road, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huixin","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Physics and Optoelectronic Engineering, Xidian University, No. 2 South Taibai Road, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kun","family":"Qian","sequence":"additional","affiliation":[{"name":"Research and Development of Infrared Detection Technology, China Aerospace Science and Technology Corporation, Shanghai 201109, China"},{"name":"Shanghai Aerospace Control Technology Institute, Shanghai 201109, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0024-8009","authenticated-orcid":false,"given":"Wei","family":"Tan","sequence":"additional","affiliation":[{"name":"School of Physics and Optoelectronic Engineering, Xidian University, No. 2 South Taibai Road, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhe","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Physics and Optoelectronic Engineering, Xidian University, No. 2 South Taibai Road, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Gu","sequence":"additional","affiliation":[{"name":"National Institute of Informatics, Tokyo 101-8430, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yue","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Physics and Optoelectronic Engineering, Xidian University, No. 2 South Taibai Road, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,3]]},"reference":[{"key":"ref_1","unstructured":"Chang, H., Yeung, D.Y., and Xiong, Y. (July, January 27). Super-resolution through neighbor embedding. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Timofte, R., De, V., and Gool, L.V. (2014, January 1\u20138). Anchored Neighborhood Regression for Fast Example-Based Super-Resolution. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, NSW, Australia.","DOI":"10.1109\/ICCV.2013.241"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Timofte, R., Smet, V.D., and Gool, L.V. (2014, January 1\u20135). A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution. Proceedings of the Asian Conference on Computer Vision, Singapore.","DOI":"10.1109\/ICCV.2013.241"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2861","DOI":"10.1109\/TIP.2010.2050625","article-title":"Image Super-Resolution via Sparse Representation","volume":"19","author":"Jianchao","year":"2010","journal-title":"IEEE Trans. Image Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1016\/j.infrared.2015.06.017","article-title":"A novel infrared image super-resolution method based on sparse representation","volume":"71","author":"Zhao","year":"2015","journal-title":"Infrared Phys. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1049\/trit.2018.1045","article-title":"Convolutional neural network based detection and judgement of environmental obstacle in vehicle operation","volume":"4","author":"Qi","year":"2019","journal-title":"CAAI Trans. Intell. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image SuperResolution Using Deep Convolutional Networks","volume":"38","author":"Dong","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Dong, C., Chen, C.L., and Tang, X. (2016, January 8\u201316). Accelerating the Super-Resolution Convolutional Neural Network. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46475-6_25"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kim, J., Lee, J.K., and Lee, K.M. (2016, January 27\u201330). Accurate Image Super-Resolution Using Very Deep Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.182"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kim, J., Kwon Lee, J., and Mu Lee, K. (2016, January 27\u201330). Deeply-recursive convolutional network for image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.181"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ma, J., Wang, X., and Jiang, J. (2019). Image Super-Resolution via Dense Discriminative Network. IEEE Trans. Ind. Electron.","DOI":"10.1109\/ICASSP.2019.8683166"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.inffus.2018.09.004","article-title":"FusionGAN: A generative adversarial network for infrared and visible image fusion","volume":"48","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_13","unstructured":"Han, T.Y., Kim, Y.J., and Song, B.C. (September, January 28). Convolutional neural network-based infrared image super resolution under low light environment. Proceedings of the 25th European Signal Processing Conference (EUSIPCO), Kos, Greece."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tseng, C.W., Su, H.R., Lai, S.H., and Liu, J. (2016, January 13\u201316). Depth image super-resolution via multi-frame registration and deep learning. Proceedings of the 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Jeju, South Korea.","DOI":"10.1109\/APSIPA.2016.7820834"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.infrared.2017.04.006","article-title":"Image super-resolution reconstruction based on regularization technique and guided filter","volume":"83","author":"Huang","year":"2017","journal-title":"Infrared Phys. Technol."},{"key":"ref_16","first-page":"2599","article-title":"Fast and accurate image super-resolution with deep laplacian pyramid networks","volume":"11","author":"Lai","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vision"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Huang, J.B., Singh, A., and Ahuja, N. (2015, January 7\u201312). Single Image Super-resolution from Transformed Self-Exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299156"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fujimoto, A., Ogawa, T., Yamamoto, K., Matsui, Y., and Aizawa, K. (2016, January 4). Manga109 dataset and creation of metadata. Proceedings of the 1st International Workshop on coMics ANalysis, Processing and Understanding, Cancun, Mexico.","DOI":"10.1145\/3011549.3011551"},{"key":"ref_20","unstructured":"Martin, D.R., Fowlkes, C., Tal, D., and Malik, J. (2001, January 7\u201314). A Database of Human Segmented Natural Images and its Application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of the Eighth International Conference On Computer Vision (ICCV-01), Vancouver, BC, Canada."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Choi, Y., Kim, N., Hwang, S., and Kweon, I.S. (2016, January 9\u201314). Thermal Image Enhancement using Convolutional Neural Network. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, South Korea.","DOI":"10.1109\/IROS.2016.7759059"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.patrec.2012.08.009","article-title":"Multispectral piecewise planar stereo using Manhattan-world assumption","volume":"34","author":"Barrera","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hwang, S., Park, J., Kim, N., Choi, Y., and Kweon, I.S. (2015, January 7\u201312). Multispectral Pedestrian Detection: Benchmark Dataset and Baseline. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298706"},{"key":"ref_24","unstructured":"Zou, X. (2017, January 18\u201320). Research on Infrared and Visible Image Registration Algorithm Based on Self-similarity. Proceedings of the International Conference on Information Science and Technology (IST 2017), Beijing, China."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/0167-2789(92)90242-F","article-title":"Nonlinear total variation based noise removal algorithms","volume":"60","author":"Rudin","year":"1992","journal-title":"Physica D"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, X., Wang, R., and Jiao, L.C. (2011, January 10\u201312). Partial Differential Equation Model Method Based on Image Feature for Denoising. Proceedings of the 2011 International Workshop on Multi-Platform\/Multi-Sensor Remote Sensing and Mapping, Xiamen, China.","DOI":"10.1109\/M2RSM.2011.5697408"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11432-014-5112-x","article-title":"Fractional partial differential equation denoising models for texture image","volume":"57","author":"Pu","year":"2014","journal-title":"Sci. Chin. Inf. Sci."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Dabov, K., Foi, A., and Egiazarian, K. (2006, January 15\u201319). Image denoising with block-matching and 3D filtering. Proceedings of the Image Processing: Algorithms and Systems, Neural Networks, and Machine Learning, San Jose, CA, USA.","DOI":"10.1117\/12.643267"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Honz\u00e1tko, D., and Kruli\u0161, M. (2017). Accelerating block-matching and 3D filtering method for image denoising on GPUs. J. Real-Time Image Process., 1\u201315.","DOI":"10.1007\/s11554-017-0737-9"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"44544","DOI":"10.1109\/ACCESS.2019.2908720","article-title":"Wavelet Deep Neural Network for Stripe Noise Removal","volume":"7","author":"Guan","year":"2019","journal-title":"IEEE Access"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"10092","DOI":"10.1364\/AO.57.010092","article-title":"Fusion of multi-focus images via a Gaussian curvature filter and synthetic focusing degree criterion","volume":"57","author":"Tan","year":"2018","journal-title":"Appl. Opt."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"He, K., Sun, J., and Tang, X. (2010, January 5\u201311). Guided image filtering. Proceedings of the European Conference on Computer Vision, Crete, Greece.","DOI":"10.1007\/978-3-642-15549-9_1"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Wu, H., Shuai, Z., Zhang, J., and Huang, K. (2018, January 18\u201322). Fast End-to-End Trainable Guided Filter. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00197"},{"key":"ref_34","unstructured":"Shi, W., Caballero, J., Theis, L., Huszar, F., and Wang, Z. (2016). Is the deconvolution layer the same as a convolutional layer?. arXiv, 7009."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, Z., Liu, D., Yang, J., Han, W., and Huang, T. (2015, January 7\u201313). Deep networks for image super-resolution with sparse prior. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.50"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lee, S., Cho, M.S., Jung, K., and Kim, J.H. (2010, January 23\u201326). Scene text extraction with edge constraint and text collinearity. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.969"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Morris, N.J.W., Avidan, S., Matusik, W., and Pfister, H. (2007, January 17\u201322). Statistics of Infrared Images. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.","DOI":"10.1109\/CVPR.2007.383003"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Tian, Y., Kong, Y., Zhong, B., and Fu, Y. (2018, January 18\u201323). Residual Dense Network for Image Super-Resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00262"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ledig, C., Theis, L., Husz\u00e1r, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21\u201326). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.19"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Yuyang, W., Feng, S., and Ye, Q. (2018, January 21\u201322). Super-Resolution of Text Image Based on Conditional Generative Adversarial Network. Proceedings of the Pacific Rim Conference on Multimedia, Hefei, China.","DOI":"10.1007\/978-3-030-00764-5_25"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Yu, L., Long, X., and Tong, C. (2018, January 21\u201322). Single Image Super-Resolution Based on Improved WGAN. Proceedings of the 2018 International Conference on Advanced Control, Automation and Artificial Intelligence (ACAAI 2018), Shenzhen, China.","DOI":"10.2991\/acaai-18.2018.24"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/281\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:42:15Z","timestamp":1760362935000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/281"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,3]]},"references-count":41,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["s20010281"],"URL":"https:\/\/doi.org\/10.3390\/s20010281","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1,3]]}}}