{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:35:18Z","timestamp":1760240118172,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2019,3,12]],"date-time":"2019-03-12T00:00:00Z","timestamp":1552348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Recently, several red-green-blue near-infrared (RGB-NIR) multispectral filter arrays (MFAs), which include near infrared (NIR) pixels, have been proposed. For extremely low light scenes, the RGB-NIR MFA sensor has been extended to receive NIR light, by adding NIR pixels to supplement for the insufficient visible band light energy. However, the resolution reconstruction of the RGB-NIR MFA, using demosaicing and color restoration methods, is based on the correlation between the NIR pixels and the pixels of other colors; this does not improve the RGB channel sensitivity with respect to the NIR channel sensitivity. In this paper, we propose a color restored image post-processing method to improve the sensitivity and resolution of an RGB-NIR MFA. Although several linear regression based color channel reconstruction methods have taken advantage of the high sensitivity NIR channel, it is difficult to accurately estimate the linear coefficients because of the high level of noise in the color channels under extremely low light conditions. The proposed method solves this problem in three steps: guided filtering, based on the linear similarity between the NIR and color channels, edge preserving smoothing to improve the accuracy of linear coefficient estimation, and residual compensation for lost spatial resolution information. The results show that the proposed method is effective, while maintaining the NIR pixel resolution characteristics, and improving the sensitivity in terms of the signal-to-noise ratio by approximately 13 dB.<\/jats:p>","DOI":"10.3390\/s19051256","type":"journal-article","created":{"date-parts":[[2019,3,13]],"date-time":"2019-03-13T04:07:37Z","timestamp":1552450057000},"page":"1256","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Sensitivity Improvement of Extremely Low Light Scenes with RGB-NIR Multispectral Filter Array Sensor"],"prefix":"10.3390","volume":"19","author":[{"given":"Seunghoon","family":"Jee","sequence":"first","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei Road, Seodaemun-gu, Seoul 03722, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Moon Gi","family":"Kang","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Yonsei University, 50 Yonsei Road, Seodaemun-gu, Seoul 03722, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,3,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1080\/014311698215748","article-title":"Review article Multisensor image fusion in remote sensing: Concepts, methods and applications","volume":"19","author":"Pohl","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TGRS.2010.2051674","article-title":"A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement","volume":"49","author":"Choi","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hao, X., Chen, H., Yao, C., Yang, N., Bi, H., and Wang, C. (2010, January 23\u201325). A near-infrared imaging method for capturing the interior of a vehicle through windshield. Proceedings of the 2010 IEEE Southwest Symposium on Image Analysis & Interpretation (SSIAI), Austin, TX, USA.","DOI":"10.1109\/SSIAI.2010.5483907"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hertel, D., Marechal, H., Tefera, D.A., Fan, W., and Hicks, R. (2009, January 3\u20135). A low-cost VIS-NIR true color night vision video system based on a wide dynamic range CMOS imager. Proceedings of the 2009 IEEE Intelligent Vehicles Symposium, Xi\u2019an, China.","DOI":"10.1109\/IVS.2009.5164290"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2127","DOI":"10.1109\/TIP.2009.2023153","article-title":"Personal Authentication Using Hand Vein Triangulation and Knuckle Shape","volume":"18","author":"Kumar","year":"2009","journal-title":"IEEE Trans. Image Process."},{"key":"ref_6","unstructured":"Yi, D., Liu, R., Chu, R., Lei, Z., and Li, S.Z. (2007, January 27\u201329). Face Matching Between Near Infrared and Visible Light Images. Proceedings of the International Conference on Biometrics, Seoul, Korea."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1109\/TPAMI.2007.1014","article-title":"Illumination invariant face recognition using near-infrared images","volume":"29","author":"Li","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Salamati, N., Fredembach, C., and S\u00fcsstrunk, S. (2009). Material classification using color and NIR images. Color and Imaging Conference, Society for Imaging Science and Technology.","DOI":"10.2352\/CIC.2009.17.1.art00040"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Fredembach, C., and Susstrunk, S. (2009). Illuminant estimation and detection using near-infrared. Digital Photography V, International Society for Optics and Photonics.","DOI":"10.1117\/12.806431"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Schaul, L., Fredembach, C., and Susstrunk, S. (2009, January 7\u201310). Color image dehazing using the near-infrared. Proceedings of the 2009 16th IEEE International Conference on Image Processing (ICIP), Cairo, Egypt.","DOI":"10.1109\/ICIP.2009.5413700"},{"key":"ref_11","unstructured":"Bayer, B.E. (1976). Color Imaging Array. (No. 3,971,065), U.S. Patent."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1185","DOI":"10.1109\/TIP.2007.894236","article-title":"Multispectral Bilateral Video Fusion","volume":"16","author":"Bennett","year":"2007","journal-title":"IEEE Trans. Image Process."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Feng, C., Zhuo, S., Zhang, X., Shen, L., and Susstrunk, S. (2013, January 15\u201318). Near-infrared guided color image dehazing. Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, Australia.","DOI":"10.1109\/ICIP.2013.6738487"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1145\/1531326.1531402","article-title":"Dark flash photography","volume":"28","author":"Krishnan","year":"2009","journal-title":"ACM Trans. Graph."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wu, C., Samadani, R., and Gunawardane, P. (2011, January 11\u201314). Same frame rate IR to enhance visible video conference lighting. Proceedings of the 2011 18th IEEE International Conference on Image Processing, Brussels, Belgium.","DOI":"10.1109\/ICIP.2011.6115734"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yan, Q., Shen, X., Xu, L., Zhuo, S., Zhang, X., Shen, L., and Jia, J. (2013, January 1\u20138). Cross-Field Joint Image Restoration via Scale Map. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, Australia.","DOI":"10.1109\/ICCV.2013.194"},{"key":"ref_17","unstructured":"Zhang, X., Sim, T., and Miao, X. (2008, January 23\u201328). Enhancing photographs with Near Infra-Red images. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhuo, S., Zhang, X., Miao, X., and Sim, T. (2010, January 26\u201329). Enhancing low light images using near infrared flash images. Proceedings of the 2010 IEEE International Conference on Image, Hong Kong, China.","DOI":"10.1109\/ICIP.2010.5652900"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"215","DOI":"10.2197\/ipsjtcva.2.215","article-title":"Image Enhancement of Low-light Scenes with Near-infrared Flash Images","volume":"2","author":"Matsui","year":"2010","journal-title":"IPSJ Trans. Comput. Vis. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tian, Q., Lansel, S., Farrell, J.E., and Wandell, B.A. (2014). Automating the design of image processing pipelines for novel color filter arrays: Local, linear, learned (L3) method. Digital Photography X, International Society for Optics and Photonics.","DOI":"10.1117\/12.2042565"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Condat, L. (2009, January 7\u201310). A generic variational approach for demosaicking from an arbitrary color filter array. Proceedings of the 2009 16th IEEE International Conference on Image Processing (ICIP), Cairo, Egypt.","DOI":"10.1109\/ICIP.2009.5413388"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Gu, J., Wolfe, P.J., and Hirakawa, K. (2010, January 26\u201329). Filterbank-based universal demosaicking. Proceedings of the 2010 IEEE International Conference on Image Processing, Hong Kong, China.","DOI":"10.1109\/ICIP.2010.5649949"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"030501","DOI":"10.1117\/1.JEI.23.3.030501","article-title":"Generalized color interpolation scheme based on intermediate quincuncial pattern","volume":"23","author":"Park","year":"2014","journal-title":"J. Electron. Imaging"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/TIP.2004.823821","article-title":"Gradient-Based Multiresolution Image Fusion","volume":"13","author":"Petrovic","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1145\/1015706.1015778","article-title":"Flash photography enhancement via intrinsic relighting","volume":"23","author":"Eisemann","year":"2004","journal-title":"ACM Trans. Graph."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1145\/1015706.1015777","article-title":"Digital photography with flash and no-flash image pairs","volume":"23","author":"Petschnigg","year":"2004","journal-title":"ACM Trans. Graph."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Park, C., and Kang, M. (2016). Color Restoration of RGBN Multispectral Filter Array Sensor Images Based on Spectral Decomposition. Sensors, 16.","DOI":"10.3390\/s16050719"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.inffus.2016.05.004","article-title":"Pixel-level image fusion: A survey of the state of the art","volume":"33","author":"Li","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1397","DOI":"10.1109\/TPAMI.2012.213","article-title":"Guided image filtering","volume":"6","author":"He","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1109\/JSEN.2018.2876774","article-title":"Single-Sensor RGB-NIR Imaging: High-Quality System Design and Prototype Implementation","volume":"19","author":"Monno","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3586","DOI":"10.1109\/TIP.2015.2448356","article-title":"Enhancing Color Images of Extremely Low Light Scenes Based on RGB\/NIR Images Acquisition With Different Exposure Times","volume":"24","author":"Sugimura","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_32","unstructured":"Montgomery, D.C., Peck, E.A., and Vining, G.G. (2012). Simple linear regression. Introduction to Linear Regression Analysis, John Wiley & Sons. [5th ed.]."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2729","DOI":"10.4236\/cs.2016.79235","article-title":"Metaheuristic Based Noise Identification and Image Denoising Using Adaptive Block Selection Based Filtering","volume":"7","author":"Devi","year":"2016","journal-title":"Circuits Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/5\/1256\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:38:16Z","timestamp":1760186296000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/5\/1256"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,12]]},"references-count":33,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2019,3]]}},"alternative-id":["s19051256"],"URL":"https:\/\/doi.org\/10.3390\/s19051256","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,3,12]]}}}