{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T08:26:16Z","timestamp":1774513576056,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2020,4,24]],"date-time":"2020-04-24T00:00:00Z","timestamp":1587686400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Hyperspectral images reconstruction focuses on recovering the spectral information from a single RGBimage. In this paper, we propose two advanced Generative Adversarial Networks (GAN) for the heavily underconstrained inverse problem. We first propose scale attention pyramid UNet (SAPUNet), which uses U-Net with dilated convolution to extract features. We establish the feature pyramid inside the network and use the attention mechanism for feature selection. The superior performance of this model is due to the modern architecture and capturing of spatial semantics. To provide a more accurate solution, we propose another distinct architecture, named W-Net, that builds one more branch compared to U-Net to conduct boundary supervision. SAPUNet and scale attention pyramid WNet (SAPWNet) provide improvements on the Interdisciplinary Computational Vision Lab at Ben Gurion University (ICVL) datasetby 42% and 46.6%, and 45% and 50% in terms of root mean square error (RMSE) and relative RMSE, respectively. The experimental results demonstrate that our proposed models are more accurate than the state-of-the-art hyperspectral recovery methods<\/jats:p>","DOI":"10.3390\/s20082426","type":"journal-article","created":{"date-parts":[[2020,4,24]],"date-time":"2020-04-24T11:42:14Z","timestamp":1587728534000},"page":"2426","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Adversarial Networks for Scale Feature-Attention Spectral Image Reconstruction from a Single RGB"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8432-711X","authenticated-orcid":false,"given":"Pengfei","family":"Liu","sequence":"first","affiliation":[{"name":"Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China"},{"name":"Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110016, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Key Laboratory of Opto-Electronic Information Processing, Chinese Academy of Sciences, Shenyang 110016, China"},{"name":"The Key Lab of Image Understanding and Computer Vision, Shenyang 110016, China"}]},{"given":"Huaici","family":"Zhao","sequence":"additional","affiliation":[{"name":"Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China"},{"name":"Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110016, China"},{"name":"Key Laboratory of Opto-Electronic Information Processing, Chinese Academy of Sciences, Shenyang 110016, China"},{"name":"The Key Lab of Image Understanding and Computer Vision, Shenyang 110016, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Fang, B., Li, Y., Zhang, H., and Chan, J.C. (2019). Hyperspectral Images Classification Based on Dense Convolutional Networks with Spectral-Wise Attention Mechanism. Remote Sens., 11.","DOI":"10.3390\/rs11020159"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"14330","DOI":"10.1364\/OE.18.014330","article-title":"Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy","volume":"18","author":"Liang","year":"2010","journal-title":"Opt. Express"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Arad, B., and Benshahar, O. (2016). Sparse Recovery of Hyperspectral Signal from Natural RGB Images. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46478-7_2"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Xiong, Z., Shi, Z., Li, H., Wang, L., Liu, D., and Wu, F. (2017, January 22\u201329). HSCNN: CNN-Based Hyperspectral Image Recovery from Spectrally Undersampled projections. Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV), Venice, Italy.","DOI":"10.1109\/ICCVW.2017.68"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"He, Z., and Liu, L. (2018). Hyperspectral Image Super-Resolution Inspired by Deep Laplacian Pyramid Network. Remote Sens., 10.","DOI":"10.3390\/rs10121939"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Parmar, M., Lansel, S., and Wandell, B.A. (2008, January 12\u201315). Spatio-spectral reconstruction of the multispectral datacube using sparse recovery. Proceedings of the 2008 15th IEEE International Conference on Image Processing, San Diego, CA, USA.","DOI":"10.1109\/ICIP.2008.4711794"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Wang, L., Xiong, Z., Gao, D., Shi, G., and Feng, W. (2015, January 7\u201312). High-speed hyperspectral video acquisition with a dual-camera architecture. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299128"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"B44","DOI":"10.1364\/AO.47.000B44","article-title":"Single disperser design for coded aperture snapshot spectral imaging","volume":"47","author":"Wagadarikar","year":"2008","journal-title":"Appl. Opt."},{"key":"ref_9","unstructured":"Xun, C., Xin, T., Dai, Q., and Lin, S. (2011, January 20\u201325). High resolution multispectral video capture with a hybrid camera system. Proceedings of the CVPR 2011, Providence, RI, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Goel, M., Patel, S.N., Whitmire, E., Mariakakis, A., and Borriello, G. (2015, January 7\u201311). HyperCam: Hyperspectral imaging for ubiquitous computing applications. Proceedings of the 2015 ACM International Joint Conference, Osaka, Japan.","DOI":"10.1145\/2750858.2804282"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Takatani, T., Aoto, T., and Mukaigawa, Y. (2017, January 21\u201326). One-Shot Hyperspectral Imaging Using Faced Reflectors. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.288"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Oh, S.W., Brown, M.S., Pollefeys, M., and Kim, S.J. (2016, January 27\u201330). Do It Yourself Hyperspectral Imaging with Everyday Digital Cameras. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.270"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Akhtar, N., Shafait, F., and Mian, A. (2016, January 8\u201316). Hierarchical Beta Process with Gaussian Process Prior for Hyperspectral Image Super Resolution. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_7"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Nguyen, R.M.H., Prasad, D.K., and Brown, M.S. (2014, January 6\u201312). Training-Based Spectral Reconstruction from a Single RGB Image. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10584-0_13"},{"key":"ref_15","unstructured":"Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A. (2017, January 4\u20139). Improved Training of Wasserstein GANs. Proceedings of the Neural Information Processing Systems, Long Beach, LA, USA."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Jia, Y., Zheng, Y., Gu, L., Subpaasa, A., Lam, A., Sato, Y., and Sato, I. (2017, January 21\u201326). From RGB to Spectrum for Natural Scenes via Manifold-Based Mapping. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/ICCV.2017.504"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-Image Translation with Conditional Adversarial Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2423","DOI":"10.1109\/TPAMI.2011.80","article-title":"A Prism-Mask System for Multispectral Video Acquisition","volume":"33","author":"Stephen","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/s11263-013-0690-4","article-title":"Acquisition of High Spatial and Spectral Resolution Video with a Hybrid Camera System","volume":"110","author":"Ma","year":"2014","journal-title":"Int. J. Comput. Vis."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kawakami, R., Matsushita, Y., Wright, J., Benezra, M., Tai, Y., and Ikeuchi, K. (2011, January 20\u201325). High-resolution hyperspectral imaging via matrix factorization. Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995457"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wu, J., Aeschbacher, J., and Timofte, R. (2017, January 22\u201329). In Defense of Shallow Learned Spectral Reconstruction from RGB Images. Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), Venice, Italy.","DOI":"10.1109\/ICCVW.2017.63"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Timofte, R., Smet, V.D., and Gool, L.V. (2014, January 1\u20135). A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution. Proceedings of the Computer Vision\u2014ACCV 2014: 12th Asian Conference on Computer Vision, Singapore.","DOI":"10.1007\/978-3-319-16817-3_8"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Robleskelly, A. (2015). Single Image Spectral Reconstruction for Multimedia Applications. ACM Multimed., 251\u2013260.","DOI":"10.1145\/2733373.2806223"},{"key":"ref_24","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Adv. Neural Inf. Process. Syst., 25."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Jian, S. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., and Laurens, E.A. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Qiu, Z., Chen, J., Zhao, Y., Zhu, S., He, Y., and Zhang, C. (2018). Variety Identification of Single Rice Seed Using Hyperspectral Imaging Combined with Convolutional Neural Network. Appl. Sci., 8.","DOI":"10.3390\/app8020212"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Shi, Z., Chen, C., Xiong, Z., Liu, D., and Wu, F. (2018, January 18\u201322). HSCNN+: Advanced CNN-Based Hyperspectral Recovery from RGB Images. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00139"},{"key":"ref_29","unstructured":"Galliani, S., Lanaras, C., Marmanis, D., Baltsavias, E.P., and Schindler, K. (2017, January 21\u201326). Learned Spectral Super-Resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Alvarezgila, A., De Weijer, J.V., and Garrote, E. (2017, January 22\u201329). Adversarial Networks for Spatial Context-Aware Spectral Image Reconstruction from RGB. Proceedings of the EEE International Conference on Computer Vision Workshop (ICCVW 2017), Venice, Italy.","DOI":"10.1109\/ICCVW.2017.64"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, K., He, L., Ma, S., Gao, S., and Bi, D. (2018). A Sensor Image Dehazing Algorithm Based on Feature Learning. Sensors, 18.","DOI":"10.3390\/s18082606"},{"key":"ref_33","unstructured":"Arjovsky, M., and Bottou, L. (2017, January 24\u201326). Towards Principled Methods for Training Generative Adversarial Networks. Proceedings of the International Conference on Learning Representations, Toulon, France."},{"key":"ref_34","unstructured":"Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6\u201311). Wasserstein GAN. Proceedings of the The 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Nah, S., Kim, T.H., and Lee, K.M. (2017, January 21\u201326). Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.35"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 10\u201316). Identity Mappings in Deep Residual Networks. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"ref_37","unstructured":"Chen, L., Yang, Y., Wang, J., Xu, W., and Yuille, A.L. (July, January 26). Attention to Scale: Scale-Aware Semantic Image Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_38","unstructured":"Li, H., Xiong, P., An, J., and Wang, L. (2018, January 18\u201321). Pyramid Attention Network for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Li, C., and Wand, M. (2016, January 8\u201316). Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_43"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Chakrabarti, A., and Zickler, T. (2011, January 20\u201325). Statistics of real-world hyperspectral images. Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995660"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2426\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:23:41Z","timestamp":1760361821000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2426"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,24]]},"references-count":40,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["s20082426"],"URL":"https:\/\/doi.org\/10.3390\/s20082426","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,24]]}}}