{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:23:39Z","timestamp":1760145819714,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T00:00:00Z","timestamp":1725408000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Key Lab of Intelligent and Green Flexographic Printing","award":["ZBKT202203","ZBKT202301"],"award-info":[{"award-number":["ZBKT202203","ZBKT202301"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Due to the inadequacy in utilizing complementary information from different modalities and the biased estimation of degraded parameters, the unsupervised hyperspectral super-resolution algorithm suffers from low precision and limited applicability. To address this issue, this paper proposes an approach for hyperspectral image super-resolution, namely, the Unsupervised Multimodal Multilevel Feature Fusion network (UMMFF). The proposed approach employs a gated cross-retention module to learn shared patterns among different modalities. This module effectively eliminates the intermodal differences while preserving spatial\u2013spectral correlations, thereby facilitating information interaction. A multilevel spatial\u2013channel attention and parallel fusion decoder are constructed to extract features at three levels (low, medium, and high), enriching the information of the multimodal images. Additionally, an independent prior-based implicit neural representation blind estimation network is designed to accurately estimate the degraded parameters. The utilization of UMMFF on the \u201cWashington DC\u201d, Salinas, and Botswana datasets exhibited a superior performance compared to existing state-of-the-art methods in terms of primary performance metrics such as PSNR and ERGAS, and the PSNR values improved by 18.03%, 8.55%, and 5.70%, respectively, while the ERGAS values decreased by 50.00%, 75.39%, and 53.27%, respectively. The experimental results indicate that UMMFF demonstrates excellent algorithm adaptability, resulting in high-precision reconstruction outcomes.<\/jats:p>","DOI":"10.3390\/rs16173282","type":"journal-article","created":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T05:54:47Z","timestamp":1725429287000},"page":"3282","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["UMMFF: Unsupervised Multimodal Multilevel Feature Fusion Network for Hyperspectral Image Super-Resolution"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2962-7969","authenticated-orcid":false,"given":"Zhongmin","family":"Jiang","sequence":"first","affiliation":[{"name":"College of Publishing, University of Shanghai for Science and Technology, Shanghai 200093, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-3773-8977","authenticated-orcid":false,"given":"Mengyao","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Publishing, University of Shanghai for Science and Technology, Shanghai 200093, China"}]},{"given":"Wenju","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Publishing, University of Shanghai for Science and Technology, Shanghai 200093, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,9,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Pande, C.B., and Moharir, K.N. (2023). Application of hyperspectral remote sensing role in precision farming and sustainable agriculture under climate change: A review. Climate Change Impacts on Natural Resources, Ecosystems and Agricultural Systems, Springer. Springer Climate.","DOI":"10.1007\/978-3-031-19059-9_21"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1073346","DOI":"10.3389\/fpls.2023.1073346","article-title":"Hyperspectral remote sensing for tobacco quality estimation, yield prediction, and stress detection: A review of applications and methods","volume":"14","author":"Zhang","year":"2023","journal-title":"Front. Plant Sci."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pan, B., Cai, S., Zhao, M., Cheng, H., Yu, H., Du, S., Du, J., and Xie, F. (2023). Predicting the Surface Soil Texture of Cultivated Land via Hyperspectral Remote Sensing and Machine Learning: A Case Study in Jianghuai Hilly Area. Appl. Sci., 13.","DOI":"10.3390\/app13169321"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, L., Miteva, T., Delnevo, G., Mirri, S., Walter, P., de Viguerie, L., and Pouyet, E. (2023). Neural networks for hyperspectral imaging of historical paintings: A practical review. Sensors, 23.","DOI":"10.3390\/s23052419"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3159","DOI":"10.3390\/heritage6030168","article-title":"Mapping materials and dyes on historic tapestries using hyperspectral imaging","volume":"6","author":"Danskin","year":"2023","journal-title":"Heritage"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Huang, S.-Y., Mukundan, A., Tsao, Y.-M., Kim, Y., Lin, F.-C., and Wang, H.-C. (2022). Recent advances in counterfeit art, document, photo, hologram, and currency detection using hyperspectral imaging. Sensors, 22.","DOI":"10.3390\/s22197308"},{"key":"ref_7","unstructured":"da Lomba Magalh\u00e3es, M.J. (2022). Hyperspectral Image Fusion\u2014A Comprehensive Review. [Master\u2019s Thesis, It\u00e4-Suomen Yliopisto]."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhang, M., Sun, X., Zhu, Q., and Zheng, G. (2021, January 11\u201316). A survey of hyperspectral image super-resolution technology. Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS 2021, Brussels, Belgium.","DOI":"10.1109\/IGARSS47720.2021.9554409"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.inffus.2020.11.001","article-title":"Recent advances and new guidelines on hyperspectral and multispectral image fusion","volume":"69","author":"Dian","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1418","DOI":"10.1109\/LGRS.2013.2294476","article-title":"Fusion of hyperspectral and multispectral images: A novel framework based on generalization of pan-sharpening methods","volume":"11","author":"Chen","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3867","DOI":"10.1109\/TGRS.2007.898443","article-title":"Spectral and spatial complexity-based hyperspectral unmixing","volume":"45","author":"Jia","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Akhtar, N., Shafait, F., and Mian, A. (2015, January 7\u201312). Bayesian sparse representation for hyperspectral image super resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2015, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298986"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"6055","DOI":"10.1109\/TGRS.2019.2904108","article-title":"Hyperspectral image super-resolution using deep feature matrix factorization","volume":"57","author":"Xie","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Dian, R., Fang, L., and Li, S. (2017, January 21\u201326). Hyperspectral image super-resolution via non-local sparse tensor factorization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.411"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"8028","DOI":"10.1109\/TIP.2020.3009830","article-title":"A truncated matrix decomposition for hyperspectral image super-resolution","volume":"29","author":"Liu","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"8384","DOI":"10.1109\/TGRS.2020.2987530","article-title":"Nonnegative and nonlocal sparse tensor factorization-based hyperspectral image super-resolution","volume":"58","author":"Wan","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4304","DOI":"10.1109\/TGRS.2019.2962713","article-title":"Hyperspectral image super-resolution by band attention through adversarial learning","volume":"58","author":"Li","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"7251","DOI":"10.1109\/TNNLS.2021.3084682","article-title":"Hyperspectral image super-resolution via deep spatiospectral attention convolutional neural networks","volume":"33","author":"Hu","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_19","first-page":"6012305","article-title":"Fusformer: A transformer-based fusion network for hyperspectral image super-resolution","volume":"19","author":"Hu","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Qu, Y., Qi, H., and Kwan, C. (2018, January 18\u201323). Unsupervised sparse dirichlet-net for hyperspectral image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00266"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yao, J., Hong, D., Chanussot, J., Meng, D., Zhu, X., and Xu, Z. (2020, January 23\u201328). Cross-attention in coupled unmixing nets for unsupervised hyperspectral super-resolution. Proceedings of the Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part XXIX 16.","DOI":"10.1007\/978-3-030-58526-6_13"},{"key":"ref_22","first-page":"6007305","article-title":"Deep Unsupervised Blind Hyperspectral and Multispectral Data Fusion","volume":"19","author":"Li","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_23","first-page":"5507018","article-title":"Unsupervised and unregistered hyperspectral image super-resolution with mutual Dirichlet-Net","volume":"60","author":"Qu","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","first-page":"5522412","article-title":"Model inspired autoencoder for unsupervised hyperspectral image super-resolution","volume":"60","author":"Liu","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_25","first-page":"1","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_26","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16 \u00d7 16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lin, H., Cheng, X., Wu, X., and Shen, D. (2022, January 18\u201322). Cat: Cross attention in vision transformer. Proceedings of the 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei, Taiwan.","DOI":"10.1109\/ICME52920.2022.9859720"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11\u201317). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., and Timofte, R. (2021, January 11\u201317). Swinir: Image restoration using swin transformer. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00210"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Conde, M.V., Choi, U.-J., Burchi, M., and Timofte, R. (2022). Swin2SR: Swinv2 transformer for compressed image super-resolution and restoration. Computer Vision\u2013ECCV 2022 Workshops, Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel, 23\u201327 October 2022, Springer Nature.","DOI":"10.1007\/978-3-031-25063-7_42"},{"key":"ref_31","unstructured":"Sun, Y., Dong, L., Huang, S., Ma, S., Xia, Y., Xue, J., Wang, J., and Wei, F. (2023). Retentive network: A successor to transformer for large language models. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2487","DOI":"10.1109\/TGRS.2020.3006534","article-title":"Coupled convolutional neural network with adaptive response function learning for unsupervised hyperspectral super resolution","volume":"59","author":"Zheng","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_33","first-page":"5509417","article-title":"Enhanced Autoencoders with Attention-Embedded Degradation Learning for Unsupervised Hyperspectral Image Super-Resolution","volume":"61","author":"Gao","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1109\/TCI.2023.3241549","article-title":"Busifusion: Blind unsupervised single image fusion of hyperspectral and rgb images","volume":"9","author":"Li","year":"2023","journal-title":"IEEE Trans. Comput. Imaging"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1109\/TGRS.2015.2476513","article-title":"Quantitative quality evaluation of pansharpened imagery: Consistency versus synthesis","volume":"54","author":"Palsson","year":"2015","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/0034-4257(93)90013-N","article-title":"The spectral image processing system (SIPS)\u2014Interactive visualization and analysis of imaging spectrometer data","volume":"44","author":"Kruse","year":"1993","journal-title":"Remote Sens. Environ."},{"key":"ref_38","unstructured":"Wald, L. (2000, January 26\u201328). Quality of high resolution synthesised images: Is there a simple criterion?. Proceedings of the Third Conference\u201d Fusion of Earth Data: Merging Point Measurements, Raster maps and Remotely Sensed Images\u201d. SEE\/URISCA, Sophia Antipolis, France."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Han, X.-H., Shi, B., and Zheng, Y. (2018, January 7\u201310). SSF-CNN: Spatial and spectral fusion with CNN for hyperspectral image super-resolution. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.","DOI":"10.1109\/ICIP.2018.8451142"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"5953","DOI":"10.1109\/TGRS.2020.3018732","article-title":"SSR-NET: Spatial\u2013spectral reconstruction network for hyperspectral and multispectral image fusion","volume":"59","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"110362","DOI":"10.1016\/j.knosys.2023.110362","article-title":"MCT-Net: Multi-hierarchical cross transformer for hyperspectral and multispectral image fusion","volume":"264","author":"Wang","year":"2023","journal-title":"Knowl.-Based Syst."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhang, L., Nie, J., Wei, W., Zhang, Y., Liao, S., and Shao, L. (2020, January 13\u201319). Unsupervised adaptation learning for hyperspectral imagery super-resolution. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2020, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00314"},{"key":"ref_43","first-page":"5525614","article-title":"Msdformer: Multi-scale deformable transformer for hyperspectral image super-resolution","volume":"601","author":"Chen","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"102148","DOI":"10.1016\/j.inffus.2023.102148","article-title":"Reciprocal transformer for hyperspectral and multispectral image fusion","volume":"104","author":"Ma","year":"2024","journal-title":"Inf. Fusion"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/17\/3282\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:48:25Z","timestamp":1760111305000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/17\/3282"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,4]]},"references-count":44,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["rs16173282"],"URL":"https:\/\/doi.org\/10.3390\/rs16173282","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2024,9,4]]}}}