{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T19:14:54Z","timestamp":1776885294579,"version":"3.51.2"},"reference-count":44,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2019,3,22]],"date-time":"2019-03-22T00:00:00Z","timestamp":1553212800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61525305;61625304"],"award-info":[{"award-number":["61525305;61625304"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007219","name":"Shanghai Natural Science Foundation","doi-asserted-by":"publisher","award":["17ZR1409700;18ZR1415300"],"award-info":[{"award-number":["17ZR1409700;18ZR1415300"]}],"id":[{"id":"10.13039\/100007219","id-type":"DOI","asserted-by":"publisher"}]},{"name":"basic research project of Shanghai Municipal Science and Technology Commission","award":["16JC1400900"],"award-info":[{"award-number":["16JC1400900"]}]},{"DOI":"10.13039\/501100007129","name":"Shandong Provincial Natural Science Foundation","doi-asserted-by":"publisher","award":["ZR2017BF021"],"award-info":[{"award-number":["ZR2017BF021"]}],"id":[{"id":"10.13039\/501100007129","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Multi-focus image fusion is a technique for obtaining an all-in-focus image in which all objects are in focus to extend the limited depth of field (DoF) of an imaging system. Different from traditional RGB-based methods, this paper presents a new multi-focus image fusion method assisted by depth sensing. In this work, a depth sensor is used together with a colour camera to capture images of a scene. A graph-based segmentation algorithm is used to segment the depth map from the depth sensor, and the segmented regions are used to guide a focus algorithm to locate in-focus image blocks from among multi-focus source images to construct the reference all-in-focus image. Five test scenes and six evaluation metrics were used to compare the proposed method and representative state-of-the-art algorithms. Experimental results quantitatively demonstrate that this method outperforms existing methods in both speed and quality (in terms of comprehensive fusion metrics). The generated images can potentially be used as reference all-in-focus images.<\/jats:p>","DOI":"10.3390\/s19061409","type":"journal-article","created":{"date-parts":[[2019,3,25]],"date-time":"2019-03-25T06:56:52Z","timestamp":1553497012000},"page":"1409","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Construction of All-in-Focus Images Assisted by Depth Sensing"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7948-4236","authenticated-orcid":false,"given":"Hang","family":"Liu","sequence":"first","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, No. 99 Shangda Road BaoShan District, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2243-5908","authenticated-orcid":false,"given":"Hengyu","family":"Li","sequence":"additional","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, No. 99 Shangda Road BaoShan District, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jun","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, No. 99 Shangda Road BaoShan District, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shaorong","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, No. 99 Shangda Road BaoShan District, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Mechatronic Engineering and Automation, Shanghai University, No. 99 Shangda Road BaoShan District, Shanghai 200444, China"},{"name":"Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,3,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.inffus.2016.05.004","article-title":"Pixel-level image fusion: A survey of the state of the art","volume":"33","author":"Li","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Motta, D., De Matos, L., De Souza, A.C., Marcato, R., Paiva, A., and De Carvalho, L.A.V. (2015, January 13\u201317). All-in-focus imaging technique used to improve 3D retinal fundus image reconstruction. Proceedings of the 30th Annual ACM Symposium on Applied Computing, Salamanca, Spain.","DOI":"10.1145\/2695664.2695845"},{"key":"ref_3","first-page":"21","article-title":"Real-time precise 3D measurement of micro transparent objects using All-In-Focus imaging system","volume":"7","author":"Nguyen","year":"2012","journal-title":"J. Micro-Bio Robot."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1364\/OE.19.000353","article-title":"Extended depth-of-field microscopic imaging with a variable focus microscope objective","volume":"19","author":"Liu","year":"2011","journal-title":"Opt. Express"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"972","DOI":"10.1109\/TPAMI.2011.168","article-title":"The Light Field Camera: Extended Depth of Field, Aliasing, and Superresolution","volume":"34","author":"Bishop","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","first-page":"341","article-title":"4D Frequency Analysis of Computational Cameras for Depth of Field Extension","volume":"28","author":"Gario","year":"2009","journal-title":"ACM Trans. Graph."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1109\/TVCG.2015.2391861","article-title":"Extended Depth-of-Field Projector by Fast Focal Sweep Projection","volume":"21","author":"Iwai","year":"2015","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1109\/TPAMI.2010.66","article-title":"Flexible Depth of Field Photography","volume":"33","author":"Kuthirummal","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1859","DOI":"10.1364\/AO.34.001859","article-title":"Extended depth of field through wave-front coding","volume":"34","author":"Dowski","year":"1995","journal-title":"Appl. Opt."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.optcom.2006.11.021","article-title":"Optimized phase pupil masks for extended depth of field","volume":"272","author":"Yang","year":"2007","journal-title":"Opt. Commun."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1171","DOI":"10.1364\/OL.33.001171","article-title":"Optimized sinusoidal phase mask to extend the depth of field of an incoherent imaging system","volume":"33","author":"Zhao","year":"2008","journal-title":"Opt. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1145\/1778765.1778768","article-title":"Diffusion coded photography for extended depth of field","volume":"29","author":"Cossairt","year":"2010","journal-title":"ACM Trans. Graph."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1002","DOI":"10.1016\/j.patcog.2012.09.012","article-title":"Multi-focus image fusion based on the neighbor distance","volume":"46","author":"Zhao","year":"2013","journal-title":"Pattern Recognit."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.inffus.2010.03.002","article-title":"Performance comparison of different multi-resolution transforms for image fusion","volume":"12","author":"Li","year":"2011","journal-title":"Inf. Fusion"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1001","DOI":"10.1016\/j.patrec.2013.03.003","article-title":"Multifocus image fusion based on robust principal component analysis","volume":"34","author":"Wan","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2898","DOI":"10.1109\/TIP.2012.2183140","article-title":"Image fusion using higher order singular value decomposition","volume":"21","author":"Liang","year":"2012","journal-title":"Image Process. IEEE Trans."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2864","DOI":"10.1109\/TIP.2013.2244222","article-title":"Image fusion with guided filtering","volume":"22","author":"Li","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.inffus.2011.07.001","article-title":"Image matting for fusion of multi-focus images in dynamic scenes","volume":"14","author":"Li","year":"2013","journal-title":"Inf. Fusion"},{"key":"ref_19","first-page":"227","article-title":"Everywhere-in-focus image fusion using controlablle cameras","volume":"2905","author":"Mckee","year":"1996","journal-title":"Proc. SPIE Int. Soc. Opt. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1007\/s00138-003-0125-2","article-title":"Microscopic vision system with all-in-focus and depth images","volume":"15","author":"Ohba","year":"2003","journal-title":"Mach. Vis. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.1016\/j.patrec.2007.01.013","article-title":"Multi-focus image fusion using pulse coupled neural network","volume":"28","author":"Huang","year":"2007","journal-title":"Pattern Recognit. Lett."},{"key":"ref_22","first-page":"1508","article-title":"Image Fusion Algorithm Based on Spatial Frequency-Motivated Pulse Coupled Neural Networks in Nonsubsampled Contourlet Transform Domain","volume":"34","author":"Yan","year":"2008","journal-title":"Acta Autom. Sin."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.inffus.2016.12.001","article-title":"Multi-focus image fusion with a deep convolutional neural network","volume":"36","author":"Liu","year":"2017","journal-title":"Inf. Fusion"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.inffus.2014.05.004","article-title":"Multi-focus image fusion with dense SIFT","volume":"23","author":"Liu","year":"2015","journal-title":"Inf. Fusion"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1109\/TPAMI.2011.109","article-title":"Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision: A Comparative Study","volume":"34","author":"Liu","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1109\/TMM.2018.2863604","article-title":"Context-Aware Three-Dimensional Mean-Shift With Occlusion Handling for Robust Object Tracking in RGB-D Videos","volume":"21","author":"Liu","year":"2019","journal-title":"IEEE Trans. Multimed."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1016\/j.patcog.2018.11.028","article-title":"Simultaneous color-depth super-resolution with conditional generative adversarial networks","volume":"88","author":"Zhao","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.cviu.2017.12.001","article-title":"A Novel perspective invariant feature transform for RGB-D images","volume":"167","author":"Yu","year":"2018","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1007\/s00138-017-0831-9","article-title":"Image-guided ToF depth upsampling: A survey","volume":"28","author":"Eichhardt","year":"2017","journal-title":"Mach. Vis. Appl."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yang, Q., Yang, R., Davis, J., and Nist\u00e9r, D. (2007, January 17\u201322). Spatial-Depth Super Resolution for Range Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.","DOI":"10.1109\/CVPR.2007.383211"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/s11265-013-0821-8","article-title":"A New High Resolution Depth Map Estimation System Using Stereo Vision and Kinect Depth Sensing","volume":"79","author":"Zhang","year":"2015","journal-title":"J. Signal Process. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.3390\/s120201437","article-title":"Accuracy and resolution of kinect depth data for indoor mapping applications","volume":"12","author":"Khoshelham","year":"2012","journal-title":"Sensors"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1318","DOI":"10.1109\/TCYB.2013.2265378","article-title":"Enhanced computer vision with microsoft kinect sensor: A review","volume":"43","author":"Han","year":"2013","journal-title":"IEEE Trans. Cybern."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1007\/s11036-013-0458-7","article-title":"Real-Time Refinement of Kinect Depth Maps Using Multi-Resolution Anisotropic Diffusion","volume":"19","author":"Vijayanagar","year":"2014","journal-title":"Mob. Netw. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"629","DOI":"10.1109\/34.56205","article-title":"Scale-space and edge detection using anisotropic diffusion","volume":"12","author":"Perona","year":"1990","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1023\/B:VISI.0000022288.19776.77","article-title":"Efficient Graph-Based Image Segmentation","volume":"59","author":"Felzenszwalb","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1002\/jemt.20118","article-title":"Autofocusing in computer microscopy: selecting the optimal focus algorithm","volume":"65","author":"Sun","year":"2004","journal-title":"Microsc. Res. Tech."},{"key":"ref_38","unstructured":"Li, H., Manjunath, B.S., and Mitra, S.K. (2002, January 13\u201316). Multisensor Image Fusion Using the Wavelet Transform. Proceedings of the 1994 International Conference on Image Processing, ICIP-94, Austin, TX, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1334","DOI":"10.1016\/j.sigpro.2009.01.012","article-title":"Multifocus image fusion using the nonsubsampled contourlet transform","volume":"89","author":"Zhang","year":"2009","journal-title":"Signal Process."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1066","DOI":"10.1049\/el:20081754","article-title":"Comments on \u2018Information measure for performance of image fusion\u2019","volume":"44","author":"Hossny","year":"2008","journal-title":"Electron. Lett."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/j.physd.2004.11.001","article-title":"A nonlinear correlation measure for multivariable data set","volume":"200","author":"Wang","year":"2005","journal-title":"Phys. Nonlinear Phenom."},{"key":"ref_42","first-page":"308","article-title":"Objective image fusion performance measure","volume":"36","author":"Xydeas","year":"2000","journal-title":"Mil. Tech. Cour."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.inffus.2006.09.001","article-title":"A novel similarity based quality metric for image fusion","volume":"9","author":"Yang","year":"2008","journal-title":"Inf. Fusion"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1016\/j.imavis.2007.12.002","article-title":"A New Automated Quality Assessment Algorithm for Image Fusion","volume":"27","author":"Chen","year":"2009","journal-title":"Image Vis. Comput."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/6\/1409\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:39:53Z","timestamp":1760186393000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/6\/1409"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,22]]},"references-count":44,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2019,3]]}},"alternative-id":["s19061409"],"URL":"https:\/\/doi.org\/10.3390\/s19061409","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,3,22]]}}}