{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T10:52:30Z","timestamp":1761648750135,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2020,5,11]],"date-time":"2020-05-11T00:00:00Z","timestamp":1589155200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"SK Hynix","award":["2018-0403"],"award-info":[{"award-number":["2018-0403"]}]},{"DOI":"10.13039\/501100002631","name":"Gachon University","doi-asserted-by":"publisher","award":["GCU-2019-0774"],"award-info":[{"award-number":["GCU-2019-0774"]}],"id":[{"id":"10.13039\/501100002631","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>A few approaches have studied image fusion using color-plus-mono dual cameras to improve the image quality in low-light shooting. Among them, the color transfer approach, which transfers the color information of a color image to a mono image, is considered to be promising for obtaining improved images with less noise and more detail. However, the color transfer algorithms rely heavily on appropriate color hints from a given color image. Unreliable color hints caused by errors in stereo matching of a color-plus-mono image pair can generate various visual artifacts in the final fused image. This study proposes a novel color transfer method that seeks reliable color hints from a color image and colorizes a corresponding mono image with reliable color hints that are based on a deep learning model. Specifically, a color-hint-based mask generation algorithm is developed to obtain reliable color hints. It removes unreliable color pixels using a reliability map computed by the binocular just-noticeable-difference model. In addition, a deep colorization network that utilizes structural information is proposed for solving the color bleeding artifact problem. The experimental results demonstrate that the proposed method provides better results than the existing image fusion algorithms for dual cameras.<\/jats:p>","DOI":"10.3390\/s20092743","type":"journal-article","created":{"date-parts":[[2020,5,11]],"date-time":"2020-05-11T12:26:30Z","timestamp":1589199990000},"page":"2743","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Deep Color Transfer for Color-Plus-Mono Dual Cameras"],"prefix":"10.3390","volume":"20","author":[{"given":"Hae Woong","family":"Jang","sequence":"first","affiliation":[{"name":"College of Information Technology Convergence, Gachon University, Seongnam 1342, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6173-0857","authenticated-orcid":false,"given":"Yong Ju","family":"Jung","sequence":"additional","affiliation":[{"name":"College of Information Technology Convergence, Gachon University, Seongnam 1342, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2020,5,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chakrabarti, A., Freeman, W.T., and Zickler, T. (2014, January 2\u20134). Rethinking color cameras. Proceedings of the 2014 IEEE International Conference on Computational Photography (ICCP), Santa Clara, CA, USA.","DOI":"10.1109\/ICCPHOT.2014.6831801"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1201","DOI":"10.1109\/TCSVT.2015.2430632","article-title":"Critical binocular asymmetry measure for the perceptual quality assessment of synthesized stereo 3D images in view synthesis","volume":"26","author":"Jung","year":"2016","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_3","unstructured":"Trinidad, M.C., Brualla, R.M., Kainz, F., and Kontkanen, J. (November, January 27). Multi-View Image Fusion. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jeon, H.-G., Lee, J.-Y., Im, S., Ha, H., and So Kweon, I. (2016, January 27\u201330). Stereo matching with color and monochrome cameras in low-light conditions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.443"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.neucom.2019.04.007","article-title":"Shoot high-quality color images using dual-lens system with monochrome and color cameras","volume":"352","author":"Dong","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_6","unstructured":"Dong, X., Li, W., Wang, X., and Wang, Y. (February, January 27). Learning a deep convolutional network for colorization in monochrome-color dual-lens system. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Welsh, T., Ashikhmin, M., and Mueller, K. (2002, January 23\u201326). Transferring color to greyscale images. Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, San Antonio, TX, USA.","DOI":"10.1145\/566570.566576"},{"key":"ref_8","unstructured":"Cohen-Or, D., Irony, R., and Lischinski, D. (July, January 29). Colorization by example. Proceedings of the Eurographics Symposium on Rendering Techniques, Konstanz, Germany."},{"key":"ref_9","unstructured":"Gupta, R.K., Chia, A.Y.-S., Rajan, D., Ng, E.S., and Huang, Z. (November, January 29). Image colorization using similar images. Proceedings of the ACM International Conference on Multimedia, Nara, Japan."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"47:1","DOI":"10.1145\/3197517.3201365","article-title":"Deep exemplar-based colorization","volume":"37","author":"He","year":"2018","journal-title":"ACM Trans. Graph."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"13:1","DOI":"10.1145\/3292482","article-title":"Progressive color transfer with dense semantic correspondences","volume":"38","author":"He","year":"2019","journal-title":"ACM Trans. Graph."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1145\/1015706.1015780","article-title":"Colorization using optimization","volume":"23","author":"Levin","year":"2004","journal-title":"ACM Trans. Graph."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"119:1","DOI":"10.1145\/3072959.3073703","article-title":"Real-time user-guided image colorization with learned deep priors","volume":"36","author":"Zhang","year":"2017","journal-title":"ACM Trans. Graph."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, B., He, M., Liao, J., Sander, P.V., Yuan, L., Bermak, A., and Chen, D. (2019, January 16\u201320). Deep exemplar-based video colorization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00824"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"12029","DOI":"10.1364\/OE.25.012029","article-title":"Enhancement of low light level images using color-plus-mono dual camera","volume":"25","author":"Jung","year":"2017","journal-title":"Opt. Express"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1391","DOI":"10.1109\/TGRS.2005.846874","article-title":"A comparative analysis of image fusion methods","volume":"43","author":"Wang","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1007\/s10851-010-0251-1","article-title":"A first-order primal-dual algorithm for convex problems with applications to imaging","volume":"40","author":"Chambolle","year":"2011","journal-title":"J. Math. Imaging Vis."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_19","unstructured":"International Telecommunications Union (2020, May 08). Recommendation ITU-R BT.601-7. Available online: https:\/\/www.itu.int\/dms_pubrec\/itu-r\/rec\/bt\/R-REC-BT.601-7-201103-I!!PDF-E.pdf."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1109\/LSP.2010.2090041","article-title":"Binocular just-noticeable-difference model for stereoscopic images","volume":"18","author":"Zhao","year":"2011","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A computational approach to edge detection","volume":"8","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/TCI.2016.2644865","article-title":"Loss functions for image restoration with neural networks","volume":"3","author":"Zhao","year":"2017","journal-title":"IEEE Trans. Comput. Imaging"},{"key":"ref_23","unstructured":"Wang, Z., Simoncelli, E.P., and Bovik, A.C. (2003, January 9\u201312). Multiscale structural similarity for image quality assessment. Proceedings of the Thirty-Seventh Asilomar Conference on Signals, Systems & Computers, Pacific Grove, CA, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2663","DOI":"10.1109\/TMI.2018.2845918","article-title":"H-DenseUNet: Hybrid densely connected UNet for liver and tumor segmentation from CT volumes","volume":"37","author":"Li","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"\u00c7i\u00e7ek, \u00d6., Abdulkadir, A., Lienkamp, S.S., Brox, T., and Ronneberger, O. (2016, January 17\u201321). 3D U-Net: Learning dense volumetric segmentation from sparse annotation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2016, Athens, Greece.","DOI":"10.1007\/978-3-319-46723-8_49"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1109\/TPAMI.2016.2572683","article-title":"Fully convolutional networks for semantic segmentation","volume":"39","author":"Shelhamer","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_29","unstructured":"Yu, F., and Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv, Available online: https:\/\/arxiv.org\/abs\/1511.07122."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_31","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv, Available online: https:\/\/arxiv.org\/abs\/1412.6980."},{"key":"ref_32","first-page":"281","article-title":"Random search for hyper-parameter optimization","volume":"13","author":"Bergstra","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1109\/TPAMI.2017.2723009","article-title":"Places: A 10 million image database for scene recognition","volume":"40","author":"Zhou","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1582","DOI":"10.1109\/TPAMI.2008.221","article-title":"Evaluation of stereo matching costs on images with radiometric differences","volume":"31","author":"Scharstein","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1002\/col.1049","article-title":"The development of the CIE 2000 colour-difference formula: CIEDE2000","volume":"26","author":"Luo","year":"2001","journal-title":"Color Res. Appl."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"5032","DOI":"10.1109\/TIP.2017.2713942","article-title":"Learning the Image Processing Pipeline","volume":"26","author":"Jiang","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_37","unstructured":"Pixabay (2019, December 09). Celebration Dahlia Flower Color. Available online: https:\/\/pixabay.com\/photos\/celebration-dahlia-flower-color-21891\/."},{"key":"ref_38","unstructured":"Pixabay (2019, December 09). Sweet Peppers Paprika Gun Orange. Available online: https:\/\/pixabay.com\/photos\/sweet-peppers-paprika-g%C3%BCn-orange-499075\/."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"745","DOI":"10.1109\/TCSVT.2013.2291281","article-title":"Visual comfort amelioration technique for stereoscopic images: Disparity remapping to mitigate global and local discomfort causes","volume":"24","author":"Sohn","year":"2014","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1778765.1778812","article-title":"Nonlinear disparity mapping for stereoscopic 3D","volume":"29","author":"Lang","year":"2010","journal-title":"ACM Trans. Graph."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1109\/TPAMI.2007.1166","article-title":"Stereo processing by semiglobal matching and mutual information","volume":"30","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"64270","DOI":"10.1109\/ACCESS.2018.2877890","article-title":"Benchmark Analysis of Representative Deep Neural Network Architectures","volume":"6","author":"Bianco","year":"2018","journal-title":"IEEE Access"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/9\/2743\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:27:49Z","timestamp":1760174869000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/9\/2743"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,11]]},"references-count":42,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2020,5]]}},"alternative-id":["s20092743"],"URL":"https:\/\/doi.org\/10.3390\/s20092743","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,5,11]]}}}