{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T09:16:36Z","timestamp":1782897396523,"version":"3.54.5"},"reference-count":46,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2018,6,27]],"date-time":"2018-06-27T00:00:00Z","timestamp":1530057600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004299","name":"Secretar\u00eda de Educaci\u00f3n Superior, Ciencia, Tecnolog\u00eda e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["2015-AR3R7694"],"award-info":[{"award-number":["2015-AR3R7694"]}],"id":[{"id":"10.13039\/501100004299","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003329","name":"Ministerio de Econom\u00eda y Competitividad","doi-asserted-by":"publisher","award":["TIN2014-56919-C3-2-R"],"award-info":[{"award-number":["TIN2014-56919-C3-2-R"]}],"id":[{"id":"10.13039\/501100003329","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003329","name":"Ministerio de Econom\u00eda y Competitividad","doi-asserted-by":"publisher","award":["TIN2017-89723-P"],"award-info":[{"award-number":["TIN2017-89723-P"]}],"id":[{"id":"10.13039\/501100003329","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Multi-spectral RGB-NIR sensors have become ubiquitous in recent years. These sensors allow the visible and near-infrared spectral bands of a given scene to be captured at the same time. With such cameras, the acquired imagery has a compromised RGB color representation due to near-infrared bands (700\u20131100 nm) cross-talking with the visible bands (400\u2013700 nm). This paper proposes two deep learning-based architectures to recover the full RGB color images, thus removing the NIR information from the visible bands. The proposed approaches directly restore the high-resolution RGB image by means of convolutional neural networks. They are evaluated with several outdoor images; both architectures reach a similar performance when evaluated in different scenarios and using different similarity metrics. Both of them improve the state of the art approaches.<\/jats:p>","DOI":"10.3390\/s18072059","type":"journal-article","created":{"date-parts":[[2018,6,27]],"date-time":"2018-06-27T11:02:05Z","timestamp":1530097325000},"page":"2059","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Wide-Band Color Imagery Restoration for RGB-NIR Single Sensor Images"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2997-2439","authenticated-orcid":false,"given":"Xavier","family":"Soria","sequence":"first","affiliation":[{"name":"Computer Vision Center, Edifici O, Campus UAB, Bellaterra, 08193 Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Angel D.","family":"Sappa","sequence":"additional","affiliation":[{"name":"Computer Vision Center, Edifici O, Campus UAB, Bellaterra, 08193 Barcelona, Spain"},{"name":"Facultad de Ingenier\u00eda en Electricidad y Computaci\u00f3n, CIDIS, Escuela Superior Polit\u00e9cnica del Litoral, ESPOL, Campus Gustavo Galindo, Km 30.5 v\u00eda Perimetral, Guayaquil 09-01-5863, Ecuador"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Riad I.","family":"Hammoud","sequence":"additional","affiliation":[{"name":"BAE Systems FAST Labs, 600 District Avenue, Burlington, MA 01803, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,6,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Moeslund, T.B. (2012). Introduction to Video and Image Processing: Building Real Systems and Applications, Springer.","DOI":"10.1007\/978-1-4471-2503-7"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Monno, Y., Kiku, D., Kikuchi, S., Tanaka, M., and Okutomi, M. (2014, January 27\u201330). Multispectral Demosaicking with Novel Guide Image Generation and Residual Interpolation. Proceedings of the IEEE International Conference on Image Processing, Paris, France.","DOI":"10.1109\/ICIP.2014.7025129"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dahl, R., Norouzi, M., and Shlens, J. (arXiv, 2017). Pixel Recursive Super Resolution, arXiv.","DOI":"10.1109\/ICCV.2017.581"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Sadeghipoor, Z., Thomas, J.B., and S\u00fcsstrunk, S. (2016, January 7\u201311). Demultiplexing Visible and Near-infrared Information in Single-sensor Multispectral Imaging. Proceedings of the Color and Imaging Conference, San Diego, CA, USA.","DOI":"10.2352\/ISSN.2169-2629.2017.32.76"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Salamati, N., Fredembach, C., and S\u00fcsstrunk, S. (2009, January 9\u201313). Material Classification Using Color and NIR Images. Proceedings of the Color and Imaging Conference, Albuquerque, NM, USA.","DOI":"10.2352\/CIC.2009.17.1.art00040"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3690","DOI":"10.3390\/s140203690","article-title":"Feature Point Descriptors: Infrared and Visible Spectra","volume":"14","author":"Ricaurte","year":"2014","journal-title":"Sensors"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Barrera, F., Lumbreras, F., and Sappa, A.D. (2012, January 25\u201327). Evaluation of Similarity Functions in Multimodal Stereo. Proceedings of the International Conference Image Analysis and Recognition, Aveiro, Portugal.","DOI":"10.1007\/978-3-642-31295-3_38"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1210","DOI":"10.1109\/TITS.2014.2354731","article-title":"Multispectral Stereo Odometry","volume":"16","author":"Mouats","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/j.isprsjprs.2010.11.001","article-title":"Support Vector Machines in Remote Sensing: A Review","volume":"66","author":"Mountrakis","year":"2011","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1007\/s11273-009-9169-z","article-title":"Multispectral and Hyperspectral Remote Sensing for Identification and Mapping of Wetland Vegetation: A Review","volume":"18","author":"Adam","year":"2010","journal-title":"Wetl. Ecol. Manag."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4985","DOI":"10.1364\/OE.22.004985","article-title":"RGB-NIR Multispectral Camera","volume":"22","author":"Chen","year":"2014","journal-title":"Opt. Express"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2791","DOI":"10.1364\/BOE.5.002791","article-title":"Single Camera Imaging System for Color and Near-Infrared Fluorescence Image Guided Surgery","volume":"5","author":"Chen","year":"2014","journal-title":"Biomed. Opt. Express"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Martinello, M., Wajs, A., Quan, S., Lee, H., Lim, C., Woo, T., Lee, W., Kim, S.S., and Lee, D. (2015, January 24\u201326). Dual Aperture Photography: Image and Depth from a Mobile Camera. Proceedings of the IEEE International Conference on Computational Photography, Houston, TX, USA.","DOI":"10.1109\/ICCPHOT.2015.7168366"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tang, H., Zhang, X., Zhuo, S., Chen, F., Kutulakos, K.N., and Shen, L. (2015, January 24\u201326). High Resolution Photography with an RGB-Infrared Camera. Proceedings of the IEEE International Conference on Computational Photography (ICCP), Houston, TX, USA.","DOI":"10.1109\/ICCPHOT.2015.7168367"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"De la Prieta, F., Vale, Z., Antunes, L., Pinto, T., Campbell, A.T., Juli\u00e1n, V., Neves, A.J., and Moreno, M.N. (2017). RGBN Multispectral Images: A Novel Color Restoration Approach. Proceedings of the Trends in Cyber-Physical Multi-Agent Systems. The PAAMS Collection 15th International Conference, Porto, Portugal, 21\u201323 June 2017, Springer International Publishing.","DOI":"10.1007\/978-3-319-61578-3"},{"key":"ref_16","unstructured":"Soria, X., Sappa, A.D., and Akbarinia, A. (December, January 28). Multispectral Single-Sensor RGB-NIR Imaging: New Challenges and Opportunities. Proceedings of the IEEE Seventh International Conference on Image Processing Theory, Tools and Applications, Montreal, QC, Canada."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Park, C., and Kang, M.G. (2016). Color Restoration of RGBN Multispectral Filter Array Sensor Images Based on Spectral Decomposition. Sensors, 16.","DOI":"10.3390\/s16050719"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Park, C.H., Oh, H.M., and Kang, M.G. (2015, January 11\u201314). Color Restoration for Infrared Cutoff Filter Removed RGBN. Proceedings of the Multispectral Filter Array Image Sensor, Berlin, Germany.","DOI":"10.5220\/0005263600300037"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1611","DOI":"10.1109\/TIP.2017.2781303","article-title":"Convolutional Sparse Coding for RGB + NIR Imaging","volume":"27","author":"Hu","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Monno, Y., Tanaka, M., and Okutomi, M. (2015, January 7\u201313). N-to-sRGB Mapping for Single-Sensor Multispectral Imaging. Proceedings of the IEEE International Conference on Computer Vision Workshops, Santiago, Chile.","DOI":"10.1109\/ICCVW.2015.18"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2352\/ISSN.2470-1173.2016.18.DPMI-256","article-title":"Single-Sensor RGB and NIR Image Acquisition: Toward Optimal Performance by Taking Account of CFA Pattern, Demosaicking, and Color Correction","volume":"2016","author":"Teranaka","year":"2016","journal-title":"Electron. Imaging"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep Learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_23","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Hayat, K. (arXiv, 2017). Super-Resolution via Deep Learning, arXiv.","DOI":"10.1016\/j.dsp.2018.07.005"},{"key":"ref_25","unstructured":"Dong, W., Wang, P., Yin, W., Shi, G., Wu, F., and Lu, X. (arXiv, 2018). Denoising Prior Driven Deep Neural Network for Image Restoration, arXiv."},{"key":"ref_26","first-page":"3371","article-title":"Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion","volume":"11","author":"Vincent","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Burger, H.C., Schuler, C.J., and Harmeling, S. (2012, January 16\u201321). Image Denoising: Can Plain Neural Networks Compete with BM3D?. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6247952"},{"key":"ref_28","unstructured":"Jain, V., and Seung, S. (2008, January 8\u201310). Natural Image Denoising with Convolutional Networks. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-Based Learning Applied to Document Recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image Super-Resolution Using Deep Convolutional Networks","volume":"38","author":"Dong","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Dong, C., Loy, C.C., He, K., and Tang, X. (2014, January 6\u201312). Learning a Deep Convolutional Network for Image Super-Resolution. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10593-2_13"},{"key":"ref_32","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th International Conference on Machine Learning (ICML), Haifa, Israel."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ledig, C., Theis, L., Husz\u00e1r, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (arXiv, 2016). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, arXiv.","DOI":"10.1109\/CVPR.2017.19"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Hore, A., and Ziou, D. (2010, January 23\u201326). Image Quality Metrics: PSNR vs. SSIM. Proceedings of the 20th international conference on Pattern recognition, Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.579"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Klette, R. (2014). Concise Computer Vision, Springer.","DOI":"10.1007\/978-1-4471-6320-6"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1016\/j.protcy.2012.05.033","article-title":"A Survey on Various Edge Detector Techniques","volume":"4","author":"Bhardwaj","year":"2012","journal-title":"Proc. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1561\/0600000017","article-title":"Local Invariant Feature Detectors: a Survey","volume":"3","author":"Tuytelaars","year":"2008","journal-title":"Found. Trends Compu. Gr. Vis."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., Krishnan, D., Taylor, G.W., and Fergus, R. (2010, January 13\u201318). Deconvolutional Networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539957"},{"key":"ref_39","unstructured":"Murugan, P., and Durairaj, S. (arXiv Preprint, 2017). Regularization and Optimization Strategies in Deep Convolutional Neural Network, arXiv Preprint."},{"key":"ref_40","unstructured":"Kingma, D., and Ba, J. (arXiv Preprint, 2014). Adam: A Method for Stochastic Optimization, arXiv Preprint."},{"key":"ref_41","unstructured":"Evangelidis, G. (2013). IAT: A Matlab Toolbox for Image Alignment, MathWorks."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image Quality Assessment: From Error Visibility to Structural Similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_43","unstructured":"Wang, Z., Simoncelli, E.P., and Bovik, A.C. (2003, January 9\u201312). Multiscale Structural Similarity for Image Quality Assessment. Proceedings of the IEEE Conference Record of the Thirty-Seventh Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1002\/col.1049","article-title":"The Development of the CIE 2000 Colour-Difference Formula: CIEDE2000","volume":"26","author":"Luo","year":"2001","journal-title":"Color Res. Appl."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1002\/col.20070","article-title":"The CIEDE2000 Color-Difference Formula: Implementation Notes, Supplementary Test Data, and Mathematical Observations","volume":"30","author":"Sharma","year":"2005","journal-title":"Color Res. Appl."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1002\/1520-6378(200102)26:1<76::AID-COL8>3.0.CO;2-3","article-title":"A Study of Digital Camera Colorimetric Characterisation Based on Polynomial Modelling","volume":"26","author":"Hong","year":"2001","journal-title":"Color Res. Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/7\/2059\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:10:28Z","timestamp":1760195428000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/7\/2059"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,27]]},"references-count":46,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2018,7]]}},"alternative-id":["s18072059"],"URL":"https:\/\/doi.org\/10.3390\/s18072059","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,6,27]]}}}