{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,25]],"date-time":"2026-01-25T15:29:32Z","timestamp":1769354972457,"version":"3.49.0"},"reference-count":40,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,2,13]],"date-time":"2023-02-13T00:00:00Z","timestamp":1676246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Australian Government Research Training Program"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Light field reconstruction and synthesis algorithms are essential for improving the lower spatial resolution for hand-held plenoptic cameras. Previous light field synthesis algorithms produce blurred regions around depth discontinuities, especially for stereo-based algorithms, where no information is available to fill the occluded areas in the light field image. In this paper, we propose a light field synthesis algorithm that uses the focal stack images and the all-in-focus image to synthesize a 9 \u00d7 9 sub-aperture view light field image. Our approach uses depth from defocus to estimate a depth map. Then, we use the depth map and the all-in-focus image to synthesize the sub-aperture views, and their corresponding depth maps by mimicking the apparent shifting of the central image according to the depth values. We handle the occluded regions in the synthesized sub-aperture views by filling them with the information recovered from the focal stack images. We also show that, if the depth levels in the image are known, we can synthesize a high-accuracy light field image with just five focal stack images. The accuracy of our approach is compared with three state-of-the-art algorithms: one non-learning and two CNN-based approaches, and the results show that our algorithm outperforms all three in terms of PSNR and SSIM metrics.<\/jats:p>","DOI":"10.3390\/s23042119","type":"journal-article","created":{"date-parts":[[2023,2,14]],"date-time":"2023-02-14T01:41:06Z","timestamp":1676338866000},"page":"2119","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Light Field View Synthesis Using the Focal Stack and All-in-Focus Image"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3559-7885","authenticated-orcid":false,"given":"Rishabh","family":"Sharma","sequence":"first","affiliation":[{"name":"School of Professional Practice and Leadership, University of Technology Sydney, Ultimo, NSW 2007, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2794-3178","authenticated-orcid":false,"given":"Stuart","family":"Perry","sequence":"additional","affiliation":[{"name":"School of Professional Practice and Leadership, University of Technology Sydney, Ultimo, NSW 2007, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1632-8062","authenticated-orcid":false,"given":"Eva","family":"Cheng","sequence":"additional","affiliation":[{"name":"School of Professional Practice and Leadership, University of Technology Sydney, Ultimo, NSW 2007, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,13]]},"reference":[{"key":"ref_1","unstructured":"Ng, R. (2006). Digital Light Field Photography, Stanford University California."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"926","DOI":"10.1109\/JSTSP.2017.2747126","article-title":"Light field image processing: An overview","volume":"11","author":"Wu","year":"2017","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1145\/1073204.1073259","article-title":"High performance imaging using large camera arrays","volume":"Volume 24","author":"Wilburn","year":"2005","journal-title":"Proceedings of the ACM Transactions on Graphics (TOG)"},{"key":"ref_4","first-page":"1","article-title":"Light field photography with a hand-held plenoptic camera","volume":"2","author":"Ng","year":"2005","journal-title":"Comput. Sci. Tech. Rep. CSTR"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sharma, R., Perry, S., and Cheng, E. (2022). Noise-Resilient Depth Estimation for Light Field Images Using Focal Stack and FFT Analysis. Sensors, 22.","DOI":"10.3390\/s22051993"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kolmogorov, V., and Zabih, R. (2002, January 28\u201331). Multi-camera scene reconstruction via graph cuts. Proceedings of the European Conference on Computer Vision, Copenhagen, Denmark.","DOI":"10.1007\/3-540-47977-5_6"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Bleyer, M., Rother, C., and Kohli, P. (2010, January 13\u201318). Surface stereo with soft segmentation. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539783"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.cviu.2015.12.007","article-title":"Robust depth estimation for light field via spinning parallelogram operator","volume":"145","author":"Zhang","year":"2016","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1023\/A:1008175127327","article-title":"Depth from Defocus vs. Stereo: How Different Really Are They?","volume":"39","author":"Schechner","year":"2000","journal-title":"Int. J. Comput. Vis."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"739","DOI":"10.1109\/TCSVT.2016.2555778","article-title":"Light-Field Depth Estimation via Epipolar Plane Image Analysis and Locally Linear Embedding","volume":"27","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/j.cviu.2004.06.001","article-title":"Extracting layers and analyzing their specular properties using epipolar-plane-image analysis","volume":"97","author":"Criminisi","year":"2005","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wanner, S., and Goldluecke, B. (2012, January 16\u201321). Globally Consistent Depth Labeling of 4D Light Fields. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6247656"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2682631","article-title":"Light field reconstruction using sparsity in the continuous fourier domain","volume":"34","author":"Shi","year":"2014","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Levin, A., and Durand, F. (2010, January 13\u201318). Linear view synthesis using a dimensionality gap light field prior. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539854"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wu, G., Zhao, M., Wang, L., Dai, Q., Chai, T., and Liu, Y. (2017, January 21\u201326). Light field reconstruction using deep convolutional network on EPI. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.178"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1109\/TPAMI.2013.147","article-title":"Variational light field analysis for disparity estimation and super-resolution","volume":"36","author":"Wanner","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","unstructured":"Zhang, Z., Liu, Y., and Dai, Q. (2015, January 7\u201312). Light field from micro-baseline image pair. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chao, C.H., Liu, C.L., and Chen, H.H. (2021, January 19\u201322). Robust light field synthesis from stereo images with left-right geometric consistency. Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA.","DOI":"10.1109\/ICIP42928.2021.9506423"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1109\/TIP.2006.884938","article-title":"Reconstructing dense light field from array of multifocus images for novel view synthesis","volume":"16","author":"Kubota","year":"2006","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","unstructured":"Mousnier, A., Vural, E., and Guillemot, C. (2015). Partial light field tomographic reconstruction from a fixed-camera focal stack. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2980179.2980251","article-title":"Learning-based view synthesis for light field cameras","volume":"35","author":"Kalantari","year":"2016","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Srinivasan, P.P., Wang, T., Sreelal, A., Ramamoorthi, R., and Ng, R. (2017, January 22\u201329). Learning to synthesize a 4D RGBD light field from a single image. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.246"},{"key":"ref_23","unstructured":"Siu, W.C., and Hung, K.W. (2012, January 3\u20136). Review of image interpolation and super-resolution. Proceedings of the 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, Hollywood, CA, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, S.E., and Williams, L. (1993, January 2\u20136). View interpolation for image synthesis. Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, Anaheim, CA, USA.","DOI":"10.1145\/166117.166153"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1007\/s10851-016-0658-4","article-title":"Lightfield recovery from its focal stack","volume":"56","author":"Magdaleno","year":"2016","journal-title":"J. Math. Imaging Vis."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Gul, M.S.K., Mukati, M.U., B\u00e4tz, M., Forchhammer, S., and Keinert, J. (2021, January 19\u201322). Light-field view synthesis using a convolutional block attention module. Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA.","DOI":"10.1109\/ICIP42928.2021.9506586"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"106992","DOI":"10.1016\/j.optlaseng.2022.106992","article-title":"Dense view synthesis for three-dimensional light-field displays based on position-guiding convolutional neural network","volume":"153","author":"Wang","year":"2022","journal-title":"Opt. Lasers Eng."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Mayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., and Brox, T. (2016, January 27\u201330). A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.438"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1681","DOI":"10.1109\/TPAMI.2018.2845393","article-title":"Light field reconstruction using convolutional network on EPI and extended applications","volume":"41","author":"Wu","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yeung, H.W.F., Hou, J., Chen, J., Chung, Y.Y., and Chen, X. (2018, January 8\u201314). Fast light field reconstruction with deep coarse-to-fine modeling of spatial-angular clues. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01231-1_9"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhou, T., Tucker, R., Flynn, J., Fyffe, G., and Snavely, N. (2018). Stereo magnification: Learning view synthesis using multiplane images. arXiv.","DOI":"10.1145\/3197517.3201323"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"104447","DOI":"10.1016\/j.imavis.2022.104447","article-title":"Robust depth estimation on real-world light field images using Gaussian belief propagation","volume":"122","author":"Zhao","year":"2022","journal-title":"Image Vis. Comput."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Rogge, S., Schiopu, I., and Munteanu, A. (2020). Depth estimation for light-field images using stereo matching and convolutional neural networks. Sensors, 20.","DOI":"10.3390\/s20216188"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1873","DOI":"10.1109\/TIP.2017.2666041","article-title":"Robust and dense depth estimation for light field images","volume":"26","author":"Navarro","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Tao, M.W., Hadap, S., Malik, J., and Ramamoorthi, R. (2013, January 1\u20138). Depth from combining defocus and correspondence using light-field cameras. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.89"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Honauer, K., Johannsen, O., Kondermann, D., and Goldluecke, B. (2016, January 20\u201324). A dataset and evaluation methodology for depth estimation on 4d light fields. Proceedings of the Asian Conference on Computer Vision, Taipei, Taiwan.","DOI":"10.1007\/978-3-319-54187-7_2"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Fachada, S., Bonatto, D., Schenkel, A., and Lafruit, G. (2018, January 3\u20135). Depth image based view synthesis with multiple reference views for virtual reality. Proceedings of the 2018-3DTV-Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), Helsinki, Finland.","DOI":"10.1109\/3DTV.2018.8478484"},{"key":"ref_39","unstructured":"(2022, October 09). Panasonic Camera Focus Stacking. Available online: https:\/\/www.panasonic.com\/uk\/consumer\/cameras-camcorders\/lumix-expert-advice-learn\/lumix-expert-advice\/in-camera-focus-stacking.html."},{"key":"ref_40","unstructured":"(2022, October 09). Olympus Camera Focus Stacking. Available online: https:\/\/learnandsupport.getolympus.com\/learn-center\/photography-tips\/macro\/autumn-mushrooms-focus-stacking-bracketing."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/2119\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:34:11Z","timestamp":1760121251000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/2119"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,13]]},"references-count":40,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23042119"],"URL":"https:\/\/doi.org\/10.3390\/s23042119","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,13]]}}}