{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:55:34Z","timestamp":1760147734060,"version":"build-2065373602"},"reference-count":47,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,2,26]],"date-time":"2023-02-26T00:00:00Z","timestamp":1677369600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Youth Innovation Promotion Association","award":["41042201503"],"award-info":[{"award-number":["41042201503"]}]},{"name":"CAS","award":["41042201503"],"award-info":[{"award-number":["41042201503"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41042201503"],"award-info":[{"award-number":["41042201503"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Defense Industrial Technology Development Program","award":["41042201503"],"award-info":[{"award-number":["41042201503"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Due to the limitations of imaging systems, the swath width of available hyperspectral images (HSIs) is relatively narrow, making it difficult to meet the demands of various applications. Therefore, an appealing idea is to expand the width of HSIs by using widely covered multispectral images (MSIs), called spectral super-resolution (SSR) of MSIs. According to the radiation transmission process of the imaging system, the spectral mixing characteristics of ground objects can be described by the linear spectral mixing model (LSMM). Inspired by the linear mixed part and nonlinear residual part of the LSMM, we propose a double-branch SSR network. To generate wide HSIs, a spectral mixing branch is designed to extract abundances from wide MSIs and adaptively learn hyperspectral endmembers from narrow HSIs. Furthermore, considering the nonlinear factors in the imaging system and atmospheric transmission, a nonlinear residual branch is built to complement the spectral and spatial details. Finally, the SSR result can be obtained with the fusion of linear and nonlinear features. To make the network structure achieve corresponding physical significance, we constrain the network through joint loss functions at different stages. In addition to two simulated datasets with limited coverage, our model is also evaluated on a real MSI\u2013HSI dataset in a larger area. Extensive experiments show the superiority of the proposed model compared with state-of-the-art baselines. Moreover, we visualized the internal results of our network and conducted ablation experiments on a single branch to further demonstrate its effectiveness. In the end, the influence of network hyperparameters, including endmembers and loss function weight coefficient, is discussed and analyzed in detail.<\/jats:p>","DOI":"10.3390\/rs15051308","type":"journal-article","created":{"date-parts":[[2023,2,27]],"date-time":"2023-02-27T02:04:11Z","timestamp":1677463451000},"page":"1308","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Spectral Mixing Theory-Based Double-Branch Network for Spectral Super-Resolution"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3358-3671","authenticated-orcid":false,"given":"Lingyu","family":"Sha","sequence":"first","affiliation":[{"name":"Aerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences (UCAS), Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0534-0974","authenticated-orcid":false,"given":"Wenjuan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7311-9844","authenticated-orcid":false,"given":"Bing","family":"Zhang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences (UCAS), Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8185-4945","authenticated-orcid":false,"given":"Zhiqiang","family":"Liu","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences (UCAS), Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2049-2108","authenticated-orcid":false,"given":"Zhen","family":"Li","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute (AIR), Chinese Academy of Sciences (CAS), Beijing 100094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,26]]},"reference":[{"key":"ref_1","first-page":"1091","article-title":"Development and Prospect of Sparse Representation-Based Hyperspectral Image Processing and Analysis","volume":"20","author":"Zhang","year":"2016","journal-title":"J. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1126\/science.228.4704.1147","article-title":"Imaging Spectrometry for Earth Remote Sensing","volume":"228","author":"Goetz","year":"1985","journal-title":"Science"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"6754","DOI":"10.1109\/TGRS.2020.2969577","article-title":"Estimation of Mineral Abundance From Hyperspectral Data Using a New Supervised Neighbor-Band Ratio Unmixing Approach","volume":"58","author":"Siebels","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/S0034-4257(01)00345-5","article-title":"Lake water quality monitoring using hyperspectral airborne data\u2014A semiempirical multisensor and multitemporal approach for the Mecklenburg Lake District, Germany","volume":"81","author":"Thiemann","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"111716","DOI":"10.1016\/j.rse.2020.111716","article-title":"Deep learning in environmental remote sensing: Achievements and challenges","volume":"241","author":"Yuan","year":"2020","journal-title":"Remote Sens. Environ."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhang, N., Yang, G., Pan, Y., Yang, X., Chen, L., and Zhao, C. (2020). A Review of Advanced Technologies and Development for Hyperspectral-Based Plant Disease Detection in the Past Three Decades. Remote Sens., 12.","DOI":"10.3390\/rs12193188"},{"key":"ref_7","first-page":"1","article-title":"Dual-Channel Convolution Network with Image-Based Global Learning Framework for Hyperspectral Image Classification","volume":"19","author":"Yu","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yokoya, N., Yairi, T., and Iwasaki, A. (2011, January 24\u201329). Coupled non-negative matrix factorization (CNMF) for hyperspectral and multispectral data fusion: Application to pasture classification. Proceedings of the 2011 IEEE International Geoscience and Remote Sensing Symposium, Vancouver, BC, Canada.","DOI":"10.1109\/IGARSS.2011.6049465"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1109\/79.974724","article-title":"Detection algorithms for hyperspectral imaging applications","volume":"19","author":"Manolakis","year":"2002","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1109\/TIP.2006.888195","article-title":"The Regularized Iteratively Reweighted MAD Method for Change Detection in Multi- and Hyperspectral Data","volume":"16","author":"Nielsen","year":"2007","journal-title":"IEEE Trans. Image Process."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1109\/TGRS.2019.2936486","article-title":"Nonlocal Coupled Tensor CP Decomposition for Hyperspectral and Multispectral Image Fusion","volume":"58","author":"Xu","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yang, J., Zhao, Y.Q., and Chan, J.C.W. (2018). Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network. Remote Sens., 10.","DOI":"10.3390\/rs10050800"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"809","DOI":"10.1109\/TGRS.2011.2162649","article-title":"Spectral\u2013Spatial Hyperspectral Image Segmentation Using Subspace Multinomial Logistic Regression and Markov Random Fields","volume":"50","author":"Li","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1080\/10095020.2020.1860653","article-title":"Advances in spaceborne hyperspectral remote sensing in China","volume":"24","author":"Zhong","year":"2021","journal-title":"Geo-Spat. Inf. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4590","DOI":"10.1109\/TGRS.2020.2964288","article-title":"Spatial and Spectral Joint Super-Resolution Using Convolutional Neural Network","volume":"58","author":"Mei","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_16","first-page":"1","article-title":"Semisupervised Spectral Degradation Constrained Network for Spectral Super-Resolution","volume":"19","author":"Chen","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4213","DOI":"10.1109\/TNNLS.2021.3056181","article-title":"Spectral Response Function-Guided Deep Optimization-Driven Network for Spectral Super-Resolution","volume":"33","author":"He","year":"2022","journal-title":"IEEE Trans. Neural Networks Learn. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016, January 11\u201314). Sparse Recovery of Hyperspectral Signal from Natural RGB Images. Proceedings of the Computer Vision, ECCV 2016, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46478-7"},{"key":"ref_19","unstructured":"Rubinstein, R., Zibulevsky, M., and Elad, M. (2008). Efficient Implementation of the K-SVD Algorithm Using Batch Orthogonal Matching Pursuit, Computer Science Department, Technion. Technical report."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2269","DOI":"10.1109\/TGRS.2020.3000684","article-title":"Spectral Superresolution of Multispectral Imagery with Joint Sparse and Low-Rank Learning","volume":"59","author":"Gao","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Huang, Z., Chen, Q., Chen, Q., Liu, X., and He, H. (2019). A Novel Hyperspectral Image Simulation Method Based on Nonnegative Matrix Factorization. Remote Sens., 11.","DOI":"10.3390\/rs11202416"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014, January 6\u201312). Training-Based Spectral Reconstruction from a Single RGB Image. Proceedings of the Computer Vision, ECCV 2014, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10590-1"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Alvarez-Gila, A., van de Weijer, J., and Garrote, E. (2017). Adversarial Networks for Spatial Context-Aware Spectral Image Reconstruction from RGB. CoRR.","DOI":"10.1109\/ICCVW.2017.64"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"J\u00e9gou, S., Drozdzal, M., Vazquez, D., Romero, A., and Bengio, Y. (2017, January 21\u201326). The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.156"},{"key":"ref_25","unstructured":"Galliani, S., Lanaras, C., Marmanis, D., Baltsavias, E., and Schindler, K. (2017). Learned Spectral Super-Resolution. CoRR."},{"key":"ref_26","unstructured":"Can, Y.B., and Timofte, R. (2018). An efficient CNN for spectral reconstruction from RGB images. CoRR."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shi, Z., Chen, C., Xiong, Z., Liu, D., and Wu, F. (2018, January 18\u201322). HSCNN+: Advanced CNN-Based Hyperspectral Recovery from RGB Images. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00139"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1109\/TNNLS.2021.3098767","article-title":"Deep Hybrid 2-D-3-D CNN Based on Dual Second-Order Attention with Camera Spectral Sensitivity Prior for Spectral Super-Resolution","volume":"34","author":"Li","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2321","DOI":"10.1109\/TGRS.2020.3004934","article-title":"Hybrid 2-D\u20133-D Deep Residual Attentional Network with Structure Tensor Constraints for Spectral Super-Resolution of RGB Images","volume":"59","author":"Li","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1109\/TPAMI.2012.59","article-title":"3D Convolutional Neural Networks for Human Action Recognition","volume":"35","author":"Ji","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"4520","DOI":"10.1109\/TGRS.2017.2693346","article-title":"Learning Sensor-Specific Spatial-Spectral Features of Hyperspectral Images via Convolutional Neural Networks","volume":"55","author":"Mei","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1016\/j.patcog.2018.11.033","article-title":"Convolutional network architectures for super-resolution\/sub-pixel mapping of drone-derived images","volume":"88","author":"Arun","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2022.3225843","article-title":"Spectral Super-Resolution of Multispectral Images Using Spatial\u2013Spectral Residual Attention Network","volume":"60","author":"Zheng","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1062","DOI":"10.1109\/TGRS.2015.2473705","article-title":"Advancement of Hyperspectral Image Processing and Information Extraction","volume":"20","author":"Zhang","year":"2016","journal-title":"J. Remote Sens."},{"key":"ref_35","unstructured":"Peng, Y., Liu, Q., Lu, H., Sun, Z., Liu, C., Chen, X., Zha, H., and Yang, J. (2020, January 16\u201318). Unregistered Hyperspectral and Multispectral Image Fusion with Synchronous Nonnegative Matrix Factorization. Proceedings of the Pattern Recognition and Computer Vision, Nanjing, China."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1109\/MGRS.2017.2762087","article-title":"Advances in Hyperspectral Image and Signal Processing: A Comprehensive Overview of the State of the Art","volume":"5","author":"Ghamisi","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2487","DOI":"10.1109\/TGRS.2020.3006534","article-title":"Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral super-resolution","volume":"59","author":"Zheng","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","unstructured":"Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11\u201313). Deep sparse rectifier neural networks. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_39","unstructured":"Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16\u201321). Rectifier nonlinearities improve neural network acoustic models. Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_40","first-page":"1","article-title":"HASIC-Net: Hybrid Attentional Convolutional Neural Network with Structure Information Consistency for Spectral Super-Resolution of RGB Images","volume":"60","author":"Li","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2337","DOI":"10.1109\/TIP.2016.2542360","article-title":"Hyperspectral Image Super-Resolution via Non-Negative Structured Sparse Representation","volume":"25","author":"Dong","year":"2016","journal-title":"IEEE Trans. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"5969","DOI":"10.1109\/TIP.2018.2862629","article-title":"Exploiting Clustering Manifold Structure for Hyperspectral Imagery Super-Resolution","volume":"27","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhang, L., Lang, Z., Wang, P., Wei, W., Liao, S., Shao, L., and Zhang, Y. (2019). Pixel-aware Deep Function-mixture Network for Spectral Super-Resolution. CoRR.","DOI":"10.1609\/aaai.v34i07.6978"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"3857","DOI":"10.1109\/TGRS.2009.2029340","article-title":"Decision Fusion for the Classification of Hyperspectral Data: Outcome of the 2008 GRS-S Data Fusion Contest","volume":"47","author":"Licciardi","year":"2009","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_45","first-page":"1","article-title":"Radiometric Cross-Calibration of the ZY1-02D Hyperspectral Imager Using the GF-5 AHSI Imager","volume":"60","author":"Niu","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1109\/MGRS.2016.2637824","article-title":"Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature","volume":"5","author":"Yokoya","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Arad, B., Ben-Shahar, O., Timofte, R., Van Gool, L., Zhang, L., Yang, M.H., Xiong, Z., Chen, C., Shi, Z., and Liu, D. (2018, January 18\u201322). NTIRE 2018 Challenge on Spectral Reconstruction from RGB Images. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00138"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/5\/1308\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:43:17Z","timestamp":1760121797000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/5\/1308"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,26]]},"references-count":47,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["rs15051308"],"URL":"https:\/\/doi.org\/10.3390\/rs15051308","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2023,2,26]]}}}