{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:03:34Z","timestamp":1760238214280,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2020,7,13]],"date-time":"2020-07-13T00:00:00Z","timestamp":1594598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41571436"],"award-info":[{"award-number":["41571436"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2017YFB1302400"],"award-info":[{"award-number":["2017YFB1302400"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Hubei Province Science and Technology Support Program, China","award":["2015BAA027"],"award-info":[{"award-number":["2015BAA027"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Multi-focus image fusion has become a very practical image processing task. It uses multiple images focused on various depth planes to create an all-in-focus image. Although extensive studies have been produced, the performance of existing methods is still limited by the inaccurate detection of the focus regions for fusion. Therefore, in this paper, we proposed a novel U-shape network which can generate an accurate decision map for the multi-focus image fusion. The Siamese encoder of our U-shape network can preserve the low-level cues with rich spatial details and high-level semantic information from the source images separately. Moreover, we introduce the ResBlocks to expand the receptive field, which can enhance the ability of our network to distinguish between focus and defocus regions. Moreover, in the bridge stage between the encoder and decoder, the spatial pyramid pooling is adopted as a global perception fusion module to capture sufficient context information for the learning of the decision map. Finally, we use a hybrid loss that combines the binary cross-entropy loss and the structural similarity loss for supervision. Extensive experiments have demonstrated that the proposed method can achieve the state-of-the-art performance.<\/jats:p>","DOI":"10.3390\/s20143901","type":"journal-article","created":{"date-parts":[[2020,7,14]],"date-time":"2020-07-14T09:30:49Z","timestamp":1594719049000},"page":"3901","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["A Novel Multi-Focus Image Fusion Network with U-Shape Structure"],"prefix":"10.3390","volume":"20","author":[{"given":"Tao","family":"Pan","sequence":"first","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430000, China"}]},{"given":"Jiaqin","family":"Jiang","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430000, China"}]},{"given":"Jian","family":"Yao","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430000, China"},{"name":"School of Artificial Intelligence, The Open University of Guangdong, Guangzhou 510000, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1971-1972","authenticated-orcid":false,"given":"Bin","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Electronic Information, Micro-Nano technology College, Qingdao University, Qingdao 266071, China"}]},{"given":"Bin","family":"Tan","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430000, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,13]]},"reference":[{"key":"ref_1","unstructured":"Stathaki, T. (2011). Image Fusion: Algorithms and Applications, Elsevier."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.inffus.2006.04.001","article-title":"Image fusion: Advances in the state of the art","volume":"2","author":"Goshtasby","year":"2007","journal-title":"Inf. Fusion"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1109\/TCOM.1983.1095851","article-title":"The Laplacian Pyramid as a Compact Image Code","volume":"31","author":"Burt","year":"1983","journal-title":"IEEE Trans. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/0167-8655(89)90003-2","article-title":"Image fusion by a ratio of low-pass pyramid","volume":"9","author":"Toet","year":"1989","journal-title":"Pattern Recognit. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"884","DOI":"10.1109\/TIM.2009.2026612","article-title":"Multifocus image fusion and restoration with sparse representation","volume":"59","author":"Yang","year":"2009","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1006\/gmip.1995.1022","article-title":"Multisensor Image Fusion Using the Wavelet Transform","volume":"57","author":"Li","year":"1995","journal-title":"Graph. Model. Image Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.inffus.2005.09.006","article-title":"Pixel- and region-based image fusion with complex wavelets","volume":"8","author":"Lewis","year":"2007","journal-title":"Inf. Fusion"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.inffus.2006.02.001","article-title":"Remote sensing image fusion using the curvelet transform","volume":"8","author":"Nencini","year":"2007","journal-title":"Inf. Fusion"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1334","DOI":"10.1016\/j.sigpro.2009.01.012","article-title":"Multifocus image fusion using the nonsubsampled contourlet transform","volume":"89","author":"Zhang","year":"2009","journal-title":"Signal Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/S1566-2535(01)00038-0","article-title":"Combination of images with diverse focuses using the spatial frequency","volume":"2","author":"Li","year":"2001","journal-title":"Inf. Fusion"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"8861","DOI":"10.1016\/j.eswa.2010.06.011","article-title":"Fusion of multi-focus images using differential evolution algorithm","volume":"37","author":"Aslantas","year":"2010","journal-title":"Expert Syst. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.inffus.2012.01.007","article-title":"Multi-focus image fusion using a morphology-based focus measure in a quad-tree structure","volume":"14","author":"De","year":"2013","journal-title":"Inf. Fusion"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.inffus.2014.05.003","article-title":"Quadtree-based multi-focus image fusion using a weighted focus-measure","volume":"22","author":"Bai","year":"2015","journal-title":"Inf. Fusion"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1016\/j.imavis.2007.10.012","article-title":"Multifocus image fusion using region segmentation and spatial frequency","volume":"26","author":"Li","year":"2008","journal-title":"Image Vis. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1948","DOI":"10.1016\/j.patrec.2006.05.004","article-title":"A region-based multi-sensor image fusion scheme using pulse-coupled neural network","volume":"27","author":"Li","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2864","DOI":"10.1109\/TIP.2013.2244222","article-title":"Image Fusion With Guided Filtering","volume":"22","author":"Li","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.inffus.2014.05.004","article-title":"Multi-focus image fusion with dense SIFT","volume":"23","author":"Liu","year":"2015","journal-title":"Inf. Fusion"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.inffus.2013.11.005","article-title":"Multi-scale weighted gradient-based fusion for multi-focus images","volume":"20","author":"Zhou","year":"2014","journal-title":"Inf. Fusion"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.inffus.2016.12.001","article-title":"Multi-focus image fusion with a deep convolutional neural network","volume":"36","author":"Liu","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.ins.2017.12.043","article-title":"Pixel convolutional neural network for multi-focus image fusion","volume":"433","author":"Tang","year":"2018","journal-title":"Inf. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1775","DOI":"10.1162\/neco_a_01098","article-title":"Fully convolutional network-based multifocus image fusion","volume":"30","author":"Guo","year":"2018","journal-title":"Neural Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Amin-Naji, M., Aghagolzadeh, A., and Ezoji, M. (2018, January 7\u201319). Fully convolutional networks for multi-focus image fusion. Proceedings of the 9th International Symposium on Telecommunications (IST), Tehran, Iran.","DOI":"10.1109\/ISTEL.2018.8660989"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Prabhakar, K.R., Srikar, V.S., and Babu, R.V. (2017, January 22\u201329). DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.505"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1102","DOI":"10.1109\/TCSVT.2018.2821177","article-title":"Multi-focus image fusion with a natural enhancement via a joint multi-level deeply supervised convolutional neural network","volume":"29","author":"Zhao","year":"2018","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_25","unstructured":"Yan, X., Gilani, S.Z., Qin, H., and Mian, A. (2018). Unsupervised deep multi-focus image fusion. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"15750","DOI":"10.1109\/ACCESS.2017.2735019","article-title":"Image segmentation-based multi-focus image fusion through multi-scale convolutional neural network","volume":"5","author":"Du","year":"2017","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1982","DOI":"10.1109\/TMM.2019.2895292","article-title":"Fusegan: Learning to fuse multi-focus image via conditional generative adversarial network","volume":"21","author":"Guo","year":"2019","journal-title":"IEEE Trans. Multimed."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.inffus.2018.01.009","article-title":"Multi-focus image fusion using content adaptive blurring","volume":"45","author":"Farid","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/s10479-005-5724-z","article-title":"A tutorial on the cross-entropy method","volume":"134","author":"Kroese","year":"2005","journal-title":"Ann. Oper. Res."},{"key":"ref_33","unstructured":"Wang, Z., Simoncelli, E.P., and Bovik, A.C. (2003, January 9\u201312). Multiscale structural similarity for image quality assessment. Proceedings of the Thrity-Seventh Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.inffus.2014.10.004","article-title":"Multi-focus image fusion using dictionary-based sparse representation","volume":"25","author":"Nejati","year":"2015","journal-title":"Inf. Fusion"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1007\/s11263-014-0733-5","article-title":"The pascal visual object classes challenge: A retrospective","volume":"111","author":"Everingham","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_36","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Ketkar, N. (2017). Introduction to pytorch. Deep Learning with Python, Springer.","DOI":"10.1007\/978-1-4842-2766-4"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1066","DOI":"10.1049\/el:20081754","article-title":"Comments on \u2018Information measure for performance of image fusion\u2019","volume":"44","author":"Hossny","year":"2008","journal-title":"Electron. Lett."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1049\/el:20000267","article-title":"Objective image fusion performance measure","volume":"36","author":"Xydeas","year":"2000","journal-title":"Electron. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.inffus.2006.09.001","article-title":"A novel similarity based quality metric for image fusion","volume":"9","author":"Yang","year":"2008","journal-title":"Inf. Fusion"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1016\/j.imavis.2007.12.002","article-title":"A new automated quality assessment algorithm for image fusion","volume":"27","author":"Chen","year":"2009","journal-title":"Image Vis. Comput."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/14\/3901\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:51:03Z","timestamp":1760176263000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/14\/3901"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,13]]},"references-count":41,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2020,7]]}},"alternative-id":["s20143901"],"URL":"https:\/\/doi.org\/10.3390\/s20143901","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,7,13]]}}}