{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T18:15:03Z","timestamp":1774030503141,"version":"3.50.1"},"reference-count":51,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2019,7,11]],"date-time":"2019-07-11T00:00:00Z","timestamp":1562803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Hyperspectral (HS) sensors sample reflectance spectrum in very high resolution, which allows us to examine material properties in very fine details. However, their widespread adoption has been hindered because they are very expensive. Reflectance spectra of real materials are high dimensional but sparse signals. By utilizing prior information about the statistics of real HS spectra, many previous studies have reconstructed HS spectra from multispectral (MS) signals (which can be obtained from cheaper, lower spectral resolution sensors). However, most of these techniques assume that the MS bands are known apriori and do not optimize the MS bands to produce more accurate reconstructions. In this paper, we propose a new end-to-end fully convolutional residual neural network architecture that simultaneously learns both the MS bands and the transformation to reconstruct HS spectra from MS signals by analyzing large quantity of HS data. The learned band can be implemented in hardware to obtain an MS sensor that collects data that is best to reconstruct HS spectra using the learned transformation. Using a diverse set of real-world datasets, we show how the proposed approach of optimizing MS bands along with the transformation can drastically increase the reconstruction accuracy. Additionally, we also investigate the prospects of using reconstructed HS spectra for land cover classification.<\/jats:p>","DOI":"10.3390\/rs11141648","type":"journal-article","created":{"date-parts":[[2019,7,11]],"date-time":"2019-07-11T11:28:28Z","timestamp":1562844508000},"page":"1648","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Spectral Super-Resolution with Optimized Bands"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1749-4329","authenticated-orcid":false,"given":"Utsav B.","family":"Gewali","sequence":"first","affiliation":[{"name":"Chester F. Carlson Center for Imaging Science, Rochester Institute of Technology, Rochester, NY 14623, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7694-9536","authenticated-orcid":false,"given":"Sildomar T.","family":"Monteiro","sequence":"additional","affiliation":[{"name":"Boeing Research and Technology, Huntsville, AL 35824, USA"}]},{"given":"Eli","family":"Saber","sequence":"additional","affiliation":[{"name":"Chester F. Carlson Center for Imaging Science, Rochester Institute of Technology, Rochester, NY 14623, USA"},{"name":"Department of Electrical &amp; Microelectronic Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA"}]}],"member":"1968","published-online":{"date-parts":[[2019,7,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Arad, B., and Ben-Shahar, O. (2016, January 11\u201314). Sparse recovery of hyperspectral signal from natural RGB images. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46478-7_2"},{"key":"ref_2","unstructured":"Arad, B., Ben-Shahar, O., Timofte, R., Van Gool, L., Zhang, L., and Yang, M.H. (2018, January 18). NTIRE 2018 challenge on spectral reconstruction from RGB images. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Schott, J.R. (2007). Remote Sensing: The Image Chain Approach, Oxford University Press on Demand.","DOI":"10.1093\/oso\/9780195178173.001.0001"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2435","DOI":"10.1109\/TGRS.2008.918089","article-title":"Hyperspectral subspace identification","volume":"46","author":"Nascimento","year":"2008","journal-title":"IEEE Trans. Geosci. Remote. Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MGRS.2015.2440094","article-title":"Hyperspectral pansharpening: A review","volume":"3","author":"Loncan","year":"2015","journal-title":"IEEE Geosci. Remote. Sens. Mag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1963","DOI":"10.1109\/JSTARS.2017.2655112","article-title":"Hyperspectral image superresolution by transfer learning","volume":"10","author":"Yuan","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.neucom.2017.05.024","article-title":"Hyperspectral image super-resolution using deep convolutional neural network","volume":"266","author":"Li","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1219","DOI":"10.1016\/j.neucom.2017.09.062","article-title":"Image super-resolution using a dilated convolutional neural network","volume":"275","author":"Lin","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1002\/col.10230","article-title":"The principal components of reflectances","volume":"29","author":"Fairman","year":"2004","journal-title":"Color Res. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1002\/col.20431","article-title":"Reconstruction of reflectance spectra using weighted principal component analysis","volume":"33","author":"Agahian","year":"2008","journal-title":"Color Res. Appl."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2673","DOI":"10.1364\/JOSAA.24.002673","article-title":"Regularized learning framework in the estimation of reflectance spectra from camera responses","volume":"24","author":"Heikkinen","year":"2007","journal-title":"JOSA A"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Nguyen, R.M., Prasad, D.K., and Brown, M.S. (2014, January 6\u201312). Training-based spectral reconstruction from a single RGB image. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10584-0_13"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3358","DOI":"10.1109\/TIP.2018.2820839","article-title":"Spectral Reflectance Estimation Using Gaussian Processes and Combination Kernels","volume":"27","author":"Heikkinen","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_14","unstructured":"Galliani, S., Lanaras, C., Marmanis, D., Baltsavias, E., and Schindler, K. (2017). Learned spectral super-resolution. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"J\u00e9gou, S., Drozdzal, M., Vazquez, D., Romero, A., and Bengio, Y. (2017, January 21\u201326). The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.156"},{"key":"ref_16","unstructured":"Aeschbacher, J., Wu, J., and Timofte, R. (2017, January 21\u201326). In defense of shallow learned spectral reconstruction from RGB images. Proceedings of the IEEE International Conference on Computer Vision, Honolulu, HI, USA."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Timofte, R., De Smet, V., and Van Gool, L. (2014, January 1\u20135). A+: Adjusted anchored neighborhood regression for fast super-resolution. Proceedings of the Asian Conference on Computer Vision, Singapore.","DOI":"10.1109\/ICCV.2013.241"},{"key":"ref_18","unstructured":"Can, Y.B., and Timofte, R. (2018). An efficient CNN for spectral reconstruction from RGB images. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Han, X.H., Shi, B., and Zheng, Y. (2018, January 20\u201324). Residual HSRCNN: Residual Hyper-Spectral Reconstruction CNN from an RGB Image. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8545634"},{"key":"ref_20","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 27). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gwn Lore, K., Reddy, K.K., Giering, M., and Bernal, E.A. (2019, January 15\u201321). Generative Adversarial Networks for Spectral Super-Resolution and Bidirectional RGB-To-Multispectral Mapping. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00122"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sun, T., and Kelly, K. (2009). Compressive sensing hyperspectral imager. Computational Optical Sensing and Imaging, Optical Society of America.","DOI":"10.1364\/COSI.2009.CTuA5"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1266","DOI":"10.1109\/JSTARS.2017.2787483","article-title":"Compressed sensing reconstruction of hyperspectral images based on spectral unmixing","volume":"11","author":"Wang","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2819","DOI":"10.1109\/TGRS.2014.2365534","article-title":"HYCA: A new technique for hyperspectral compressive sensing","volume":"53","author":"Plaza","year":"2015","journal-title":"IEEE Trans. Geosci. Remote. Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"942","DOI":"10.1364\/JOSAA.24.000942","article-title":"Selecting algorithms, sensors, and linear bases for optimum spectral recovery of skylight","volume":"24","author":"Valero","year":"2007","journal-title":"JOSA A"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"634","DOI":"10.1364\/AO.53.000634","article-title":"Channel selection for multispectral color imaging using binary differential evolution","volume":"53","author":"Shen","year":"2014","journal-title":"Appl. Opt."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Arad, B., and Ben-Shahar, O. (2017, January 22\u201329). Filter selection for hyperspectral estimation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.342"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, Y., Majumder, A., Zhang, H., and Gopi, M. (2018). Optimized Multi-Spectral Filter Array Based Imaging of Natural Scenes. Sensors, 18.","DOI":"10.3390\/s18041172"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Fu, Y., Zhang, T., Zheng, Y., Zhang, D., and Huang, H. (2018, January 8\u201314). Joint Camera Spectral Sensitivity Selection and Hyperspectral Image Recovery. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01219-9_48"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Nie, S., Gu, L., Zheng, Y., Lam, A., Ono, N., and Sato, I. (2018, January 18\u201322). Deeply Learned Filter Response Functions for Hyperspectral Reconstruction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00501"},{"key":"ref_31","unstructured":"Gewali, U.B., Monteiro, S.T., and Saber, E. (2018). Machine learning based hyperspectral image analysis: A survey. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Lanaras, C., Baltsavias, E., and Schindler, K. (2015, January 11\u201318). Hyperspectral super-resolution by coupled spectral unmixing. Proceedings of the IEEE International Conference on Computer Vision, Araucano Park, Las Condes, Chile.","DOI":"10.1109\/ICCV.2015.409"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1109\/TGRS.2003.819189","article-title":"Estimation of number of spectrally distinct signal sources in hyperspectral imagery","volume":"42","author":"Chang","year":"2004","journal-title":"IEEE Trans. Geosci. Remote. Sens."},{"key":"ref_34","first-page":"55","article-title":"Analysis of spectral absorption features in hyperspectral imagery","volume":"5","year":"2004","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/S0034-4257(98)00032-7","article-title":"Derivative analysis of hyperspectral data","volume":"66","author":"Tsai","year":"1998","journal-title":"Remote. Sens. Environ."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1109\/JPROC.2015.2494218","article-title":"Taking the human out of the loop: A review of Bayesian optimization","volume":"104","author":"Shahriari","year":"2016","journal-title":"Proc. IEEE"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Frazier, P.I. (2018). A tutorial on Bayesian optimization. arXiv.","DOI":"10.1287\/educ.2018.0188"},{"key":"ref_38","unstructured":"Snoek, J., Larochelle, H., and Adams, R.P. (2012, January 3\u20136). Practical Bayesian optimization of machine learning algorithms. Proceedings of the Advances in Neural Information Processing Systems (NIPS), Lake Tahoe, NT, USA."},{"key":"ref_39","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_40","unstructured":"Glorot, X., and Bengio, Y. (2010, January 13\u201315). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy."},{"key":"ref_41","first-page":"2241","article-title":"Generalized Assorted Pixel Camera: Post-Capture Control of Resolution, Dynamic Range and Spectrum","volume":"9","author":"Yasuma","year":"2008","journal-title":"IEEE Trans. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"4045","DOI":"10.3390\/rs5084045","article-title":"NASA Goddard\u2019s LiDAR, hyperspectral and thermal (G-LiHT) airborne imager","volume":"5","author":"Cook","year":"2013","journal-title":"Remote. Sens."},{"key":"ref_43","unstructured":"Pearlman, J., Carman, S., Segal, C., Jarecke, P., Clancy, P., and Browne, W. (2001, January 9\u201313). Overview of the Hyperion imaging spectrometer for the NASA EO-1 mission. Proceedings of the International Geoscience and Remote Sensing Symposium (IGARSS), Sydney, Australia."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.rse.2014.02.001","article-title":"Landsat-8: Science and product vision for terrestrial global change research","volume":"145","author":"Roy","year":"2014","journal-title":"Remote. Sens. Environ."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/0034-4257(93)90012-M","article-title":"The airborne visible\/infrared imaging spectrometer (AVIRIS)","volume":"44","author":"Vane","year":"1993","journal-title":"Remote. Sens. Environ."},{"key":"ref_46","unstructured":"Baumgardner, M.F., Biehl, L.L., and Landgrebe, D.A. (2015). 220 Band AVIRIS Hyperspectral Image Data Set: June 12, 1992 Indian Pine Test Site 3, Purdue University."},{"key":"ref_47","unstructured":"(2019, April 08). Airborne Visible\/Infrared Imaging Spectrometer (AVIRIS) Data Products Portal, Available online: https:\/\/aviris.jpl.nasa.gov\/alt_locator\/."},{"key":"ref_48","unstructured":"Aarts, E., and Korst, J. (1988). Simulated Annealing and Boltzmann Machines, John Wiley and Sons Inc."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1109\/36.581987","article-title":"Second simulation of the satellite signal in the solar spectrum, 6S: An overview","volume":"35","author":"Vermote","year":"1997","journal-title":"IEEE Trans. Geosci. Remote. Sens."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Su, J., Yi, D., Liu, C., Guo, L., and Chen, W.H. (2017). Dimension reduction aided hyperspectral image classification with a small-sized training dataset: Experimental comparisons. Sensors, 17.","DOI":"10.3390\/s17122726"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1109\/TSMCB.2006.870645","article-title":"A novel incremental principal component analysis and its application for face recognition","volume":"36","author":"Zhao","year":"2006","journal-title":"IEEE Trans. Syst. Man, Cybern. Part B (Cybern.)"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/14\/1648\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:04:30Z","timestamp":1760187870000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/14\/1648"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,11]]},"references-count":51,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2019,7]]}},"alternative-id":["rs11141648"],"URL":"https:\/\/doi.org\/10.3390\/rs11141648","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,11]]}}}