{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T16:21:15Z","timestamp":1761582075164,"version":"build-2065373602"},"reference-count":54,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2021,3,28]],"date-time":"2021-03-28T00:00:00Z","timestamp":1616889600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001700","name":"Ministry of Education, Culture, Sports, Science and Technology","doi-asserted-by":"publisher","award":["20K11867"],"award-info":[{"award-number":["20K11867"]}],"id":[{"id":"10.13039\/501100001700","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Hyperspectral image (HSI) super-resolution (SR) is a challenging task due to its ill-posed nature, and has attracted extensive attention by the research community. Previous methods concentrated on leveraging various hand-crafted image priors of a latent high-resolution hyperspectral (HR-HS) image to regularize the degradation model of the observed low-resolution hyperspectral (LR-HS) and HR-RGB images. Different optimization strategies for searching a plausible solution, which usually leads to a limited reconstruction performance, were also exploited. Recently, deep-learning-based methods evolved for automatically learning the abundant image priors in a latent HR-HS image. These methods have made great progress for HS image super resolution. Current deep-learning methods have faced difficulties in designing more complicated and deeper neural network architectures for boosting the performance. They also require large-scale training triplets, such as the LR-HS, HR-RGB, and their corresponding HR-HS images for neural network training. These training triplets significantly limit their applicability to real scenarios. In this work, a deep unsupervised fusion-learning framework for generating a latent HR-HS image using only the observed LR-HS and HR-RGB images without previous preparation of any other training triplets is proposed. Based on the fact that a convolutional neural network architecture is capable of capturing a large number of low-level statistics (priors) of images, the automatic learning of underlying priors of spatial structures and spectral attributes in a latent HR-HS image using only its corresponding degraded observations is promoted. Specifically, the parameter space of a generative neural network used for learning the required HR-HS image to minimize the reconstruction errors of the observations using mathematical relations between data is investigated. Moreover, special convolutional layers for approximating the degradation operations between observations and the latent HR-HS image are specifically to construct an end-to-end unsupervised learning framework for HS image super-resolution. Experiments on two benchmark HS datasets, including the CAVE and Harvard, demonstrate that the proposed method can is capable of producing very promising results, even under a large upscaling factor. Furthermore, it can outperform other unsupervised state-of-the-art methods by a large margin, and manifests its superiority and efficiency.<\/jats:p>","DOI":"10.3390\/s21072348","type":"journal-article","created":{"date-parts":[[2021,3,28]],"date-time":"2021-03-28T23:27:25Z","timestamp":1616974045000},"page":"2348","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Deep Unsupervised Fusion Learning for Hyperspectral Image Super Resolution"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3756-2678","authenticated-orcid":false,"given":"Zhe","family":"Liu","sequence":"first","affiliation":[{"name":"Graduate School of Science and Technology for Innovation, Yamaguchi University, Yamaguchi 753-8511, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7434-5069","authenticated-orcid":false,"given":"Yinqiang","family":"Zheng","sequence":"additional","affiliation":[{"name":"National Institute of Informatics, Tokyo 101-8430, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5003-3180","authenticated-orcid":false,"given":"Xian-Hua","family":"Han","sequence":"additional","affiliation":[{"name":"Graduate School of Science and Technology for Innovation, Yamaguchi University, Yamaguchi 753-8511, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MGRS.2013.2244672","article-title":"Hyperspectral remote sensing data analysis and future challenges","volume":"1","author":"Plaza","year":"2013","journal-title":"IEEE Geosci. Remote. Sens. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1039","DOI":"10.1080\/10408398.2011.651542","article-title":"Application of hyperspectral imaging in food safety inspection and control: A review","volume":"52","author":"Feng","year":"2012","journal-title":"Crit. Rev. Food Sci. Nutr."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Qin, J., Kim, M.S., Chao, K., Chan, D.E., Delwiche, S.R., and Cho, B.K. (2017). Line-scan hyperspectral imaging techniques for food safety and quality applications. Appl. Sci., 7.","DOI":"10.3390\/app7020125"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"110148","DOI":"10.1016\/j.jfoodeng.2020.110148","article-title":"Effectiveness of specularity removal from hyperspectral images on the quality of spectral signatures of food products","volume":"289","author":"ElMasry","year":"2021","journal-title":"J. Food Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1109\/TGRS.2005.846154","article-title":"Kernel-based methods for hyperspectral image classification","volume":"43","author":"Bruzzone","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3639","DOI":"10.1109\/TGRS.2016.2636241","article-title":"Deep recurrent neural networks for hyperspectral image classification","volume":"55","author":"Mou","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1080\/01431161.2020.1807650","article-title":"Information-theoretic feature selection with segmentation-based folded principal component analysis (PCA) for hyperspectral image classification","volume":"42","author":"Uddin","year":"2021","journal-title":"Int. J. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liang, J., Zhou, J., Bai, X., and Qian, Y. (2013, January 15\u201318). Salient object detection in hyperspectral imagery. Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, Australia.","DOI":"10.1109\/ICIP.2013.6738493"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5673","DOI":"10.1109\/TGRS.2018.2823866","article-title":"Detection and correction of mislabeled training samples for hyperspectral image classification","volume":"56","author":"Kang","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"108493","DOI":"10.1016\/j.measurement.2020.108493","article-title":"Metal object detection for electric vehicle inductive power transfer systems based on hyperspectral imaging","volume":"168","author":"Tian","year":"2021","journal-title":"Measurement"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1080\/05704928.2013.838678","article-title":"Hyperspectral imaging in the medical field: Present and future","volume":"49","author":"Calin","year":"2014","journal-title":"Appl. Spectrosc. Rev."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2265","DOI":"10.1049\/iet-ipr.2018.5398","article-title":"Tongue colour and coating prediction in traditional Chinese medicine based on visible hyperspectral imaging","volume":"13","author":"Zhang","year":"2019","journal-title":"IET Image Process."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Dremin, V., Marcinkevics, Z., Zherebtsov, E., Popov, A., Grabovskis, A., Kronberga, H., Geldnere, K., Doronin, A., Meglinski, I., and Bykov, A. (2021). Skin complications of diabetes mellitus revealed by polarized hyperspectral imaging and machine learning. IEEE Trans. Med. Imaging.","DOI":"10.1109\/TMI.2021.3049591"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2082","DOI":"10.1016\/j.sigpro.2012.01.020","article-title":"A super-resolution reconstruction algorithm for hyperspectral images","volume":"92","author":"Zhang","year":"2012","journal-title":"Signal Process."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1109\/TGRS.2011.2161320","article-title":"Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion","volume":"50","author":"Yokoya","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kawakami, R., Matsushita, Y., Wright, J., Ben-Ezra, M., Tai, Y.W., and Ikeuchi, K. (2011, January 20\u201325). High-resolution hyperspectral imaging via matrix factorization. Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995457"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3973","DOI":"10.1109\/TGRS.2011.2129595","article-title":"Hyperspectral image classification using dictionary-based sparse representation","volume":"49","author":"Chen","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1376","DOI":"10.1109\/TGRS.2015.2479299","article-title":"A low-rank and sparse matrix decomposition-based Mahalanobis distance method for hyperspectral anomaly detection","volume":"54","author":"Zhang","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"866","DOI":"10.1109\/TGRS.2018.2862384","article-title":"Intracluster structured low-rank matrix analysis method for hyperspectral denoising","volume":"57","author":"Wei","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2872","DOI":"10.1109\/TGRS.2017.2785359","article-title":"Simultaneous spatial and spectral low-rank representation of hyperspectral images for classification","volume":"56","author":"Mei","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_21","unstructured":"Han, X.H., Zheng, Y., and Chen, Y.W. (November, January 27). Multi-Level and Multi-Scale Spatial and Spectral Fusion CNN for Hyperspectral Image Super-Resolution. Proceedings of the IEEE International Conference on Computer Vision Workshops, Seoul, Korea."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Han, X.H., Shi, B., and Zheng, Y. (2018, January 7\u201310). Ssf-cnn: Spatial and spectral fusion with cnn for hyperspectral image super-resolution. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.","DOI":"10.1109\/ICIP.2018.8451142"},{"key":"ref_23","unstructured":"Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2018, January 18\u201323). Deep image prior. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA."},{"key":"ref_24","unstructured":"Sidorov, O., and Yngve Hardeberg, J. (November, January 27). Deep Hyperspectral Prior: Single-Image Denoising, Inpainting, Super-Resolution. Proceedings of the IEEE International Conference on Computer Vision Workshops, Seoul, Korea."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1418","DOI":"10.1109\/LGRS.2013.2294476","article-title":"Fusion of hyperspectral and multispectral images: A novel framework based on generalization of pan-sharpening methods","volume":"11","author":"Chen","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1687-6180-2012-207","article-title":"Fusion of hyperspectral and panchromatic images using multiresolution analysis and nonlinear PCA band reduction","volume":"2012","author":"Licciardi","year":"2012","journal-title":"Eurasip J. Adv. Signal Process."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Vivone, G., Restaino, R., Licciardi, G., Dalla Mura, M., and Chanussot, J. (2014, January 13\u201318). Multiresolution analysis and component substitution techniques for hyperspectral pansharpening. Proceedings of the 2014 IEEE Geoscience and Remote Sensing Symposium, Quebec City, QC, Canada.","DOI":"10.1109\/IGARSS.2014.6947018"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Han, X.H., Wang, J., Shi, B., Zheng, Y., and Chen, Y.W. (2017, January 11\u201313). Hyper-spectral image super-resolution using non-negative spectral representation with data-guided sparsity. Proceedings of the 2017 IEEE International Symposium on Multimedia (ISM), Taichung, Taiwan.","DOI":"10.1109\/ISM.2017.99"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lanaras, C., Baltsavias, E., and Schindler, K. (2015, January 7\u201313). Hyperspectral super-resolution by coupled spectral unmixing. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.409"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Chen, Y., He, W., Yokoya, N., and Huang, T.Z. (2019). Hyperspectral image restoration using weighted group sparsity-regularized low-rank tensor decomposition. IEEE Trans. Cybern.","DOI":"10.1109\/IGARSS.2019.8897967"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Akhtar, N., Shafait, F., and Mian, A. (2015, January 7\u201312). Bayesian sparse representation for hyperspectral image super resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298986"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2664","DOI":"10.1109\/TGRS.2015.2504261","article-title":"Exploiting joint sparsity for pansharpening: The J-SparseFI algorithm","volume":"54","author":"Zhu","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1016\/j.neucom.2018.10.023","article-title":"Low-rank Bayesian tensor factorization for hyperspectral image denoising","volume":"331","author":"Wei","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2337","DOI":"10.1109\/TIP.2016.2542360","article-title":"Hyperspectral image super-resolution via non-negative structured sparse representation","volume":"25","author":"Dong","year":"2016","journal-title":"IEEE Trans. Image Process."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"5625","DOI":"10.1109\/TIP.2018.2855418","article-title":"Self-similarity constrained sparse representation for hyperspectral image super-resolution","volume":"27","author":"Han","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Han, X.H., Shi, B., and Zheng, Y. (2018, January 20\u201324). Residual hsrcnn: Residual hyper-spectral reconstruction cnn from an rgb image. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8545634"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Han, X.H., and Chen, Y.W. (2019, January 11\u201313). Deep residual network of spectral and spatial fusion for hyperspectral image super-resolution. Proceedings of the 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), Singapore.","DOI":"10.1109\/BigMM.2019.00-13"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2652","DOI":"10.1109\/TGRS.2014.2363477","article-title":"Model-based fusion of multi-and hyperspectral images using PCA and wavelets","volume":"53","author":"Palsson","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"5345","DOI":"10.1109\/TNNLS.2018.2798162","article-title":"Deep hyperspectral image sharpening","volume":"29","author":"Dian","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"7565","DOI":"10.1109\/TIP.2020.3004261","article-title":"FusionNet: An Unsupervised Convolutional Variational Network for Hyperspectral and Multispectral Image Fusion","volume":"29","author":"Wang","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Qu, Y., Qi, H., and Kwan, C. (2018, January 18\u201323). Unsupervised sparse dirichlet-net for hyperspectral image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00266"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhang, L., Nie, J., Wei, W., Zhng, Y., Liao, S., and Shao, L. (2020, January 13\u201319). Unsupervised Adaptation Learning for Hyperspectral Imagery Super-resolution. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00314"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Nie, J., Zhang, L., Wei, W., Lang, Z., and Zhang, Y. (2020). Unsupervised Alternating Optimization for Blind Hyperspectral Imagery Super-resolution. arXiv.","DOI":"10.1109\/CVPR42600.2020.00314"},{"key":"ref_44","unstructured":"Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1016\/j.neuroimage.2018.07.043","article-title":"Generative adversarial networks for reconstructing natural images from brain activity","volume":"181","author":"Seeliger","year":"2018","journal-title":"NeuroImage"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"2290","DOI":"10.1109\/TGRS.2018.2872830","article-title":"Deep few-shot learning for hyperspectral image classification","volume":"57","author":"Liu","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_47","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2241","DOI":"10.1109\/TIP.2010.2046811","article-title":"Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum","volume":"19","author":"Yasuma","year":"2010","journal-title":"IEEE Trans. Image Process."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Chakrabarti, A., and Zickler, T. (2011, January 20\u201325). Statistics of real-world hyperspectral images. Proceedings of the Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995660"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"6202","DOI":"10.1109\/TSP.2012.2218810","article-title":"Generalized orthogonal matching pursuit","volume":"60","author":"Wang","year":"2012","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Wycoff, E., Chan, T.H., Jia, K., Ma, W.K., and Ma, Y. (2013, January 26\u201331). A non-negative sparse promoting algorithm for high resolution hyperspectral imaging. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6637883"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"5443","DOI":"10.1109\/TGRS.2018.2817393","article-title":"Target-Adaptive CNN-Based Pansharpening","volume":"56","author":"Scarpa","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1109\/LGRS.2017.2668299","article-title":"Multispectral and hyperspectral image fusion using a 3-D-convolutional neural network","volume":"14","author":"Palsson","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Xie, Q., Zhou, M., Zhao, Q., Meng, D., Zuo, W., and Xu, Z. (2019, January 16\u201320). Multispectral and hyperspectral image fusion by MS\/HS fusion net. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00168"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/7\/2348\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:32:59Z","timestamp":1760362379000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/7\/2348"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,28]]},"references-count":54,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["s21072348"],"URL":"https:\/\/doi.org\/10.3390\/s21072348","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,3,28]]}}}