{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,18]],"date-time":"2026-02-18T23:46:53Z","timestamp":1771458413937,"version":"3.50.1"},"reference-count":50,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2020,10,30]],"date-time":"2020-10-30T00:00:00Z","timestamp":1604016000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003130","name":"Fonds Wetenschappelijk Onderzoek","doi-asserted-by":"publisher","award":["1S83118N"],"award-info":[{"award-number":["1S83118N"]}],"id":[{"id":"10.13039\/501100003130","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The paper presents a novel depth-estimation method for light-field (LF) images based on innovative multi-stereo matching and machine-learning techniques. In the first stage, a novel block-based stereo matching algorithm is employed to compute the initial estimation. The proposed algorithm is specifically designed to operate on any pair of sub-aperture images (SAIs) in the LF image and to compute the pair\u2019s corresponding disparity map. For the central SAI, a disparity fusion technique is proposed to compute the initial disparity map based on all available pairwise disparities. In the second stage, a novel pixel-wise deep-learning (DL)-based method for residual error prediction is employed to further refine the disparity estimation. A novel neural network architecture is proposed based on a new structure of layers. The proposed DL-based method is employed to predict the residual error of the initial estimation and to refine the final disparity map. The experimental results demonstrate the superiority of the proposed framework and reveal that the proposed method achieves an average improvement of 15.65% in root mean squared error (RMSE), 43.62% in mean absolute error (MAE), and 5.03% in structural similarity index (SSIM) over machine-learning-based state-of-the-art methods.<\/jats:p>","DOI":"10.3390\/s20216188","type":"journal-article","created":{"date-parts":[[2020,10,30]],"date-time":"2020-10-30T09:29:32Z","timestamp":1604050172000},"page":"6188","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Depth Estimation for Light-Field Images Using Stereo Matching and Convolutional Neural Networks"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5734-9583","authenticated-orcid":false,"given":"S\u00e9gol\u00e8ne","family":"Rogge","sequence":"first","affiliation":[{"name":"Department of Electronics and Informatics, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2202-1163","authenticated-orcid":false,"given":"Ionut","family":"Schiopu","sequence":"additional","affiliation":[{"name":"Department of Electronics and Informatics, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7290-0428","authenticated-orcid":false,"given":"Adrian","family":"Munteanu","sequence":"additional","affiliation":[{"name":"Department of Electronics and Informatics, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Lin, H., Chen, C., Bing Kang, S., and Yu, J. (2015, January 7\u201313). Depth recovery from light field using focal stack symmetry. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.394"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Wang, T.-C., Efros, A.A., and Ramamoorthi, R. (2015, January 7\u201313). Occlusion-aware depth estimation using light-field cameras. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.398"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2170","DOI":"10.1109\/TPAMI.2016.2515615","article-title":"Depth estimation with occlusion modeling using light-field cameras","volume":"38","author":"Wang","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jeon, H.-G., Park, J., Choe, G., Park, J., Bok, Y., Tai, Y.-W., and Kweon, I.S. (2015). Accurate depth map estimation from a lenslet light field camera. Comput. Vision Pattern Recognit., 1547\u20131555.","DOI":"10.1109\/CVPR.2015.7298762"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1109\/TPAMI.2018.2794979","article-title":"Depth from a light field image with learning-based matching costs","volume":"41","author":"Jeon","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1145\/1073204.1073256","article-title":"Fourier slice photography","volume":"24","author":"Ng","year":"2005","journal-title":"ACM Trans. Graph."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Dansereau, D.G., Pizarro, O., and Williams, S.B. (2013). Decoding, calibration and rectification for lenselet-based plenoptic cameras. Comput. Vis. Pattern Recognit., 1027\u20131034.","DOI":"10.1109\/CVPR.2013.137"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1109\/TPAMI.2016.2541145","article-title":"Geometric calibration of microlens-based light field cameras using line features","volume":"39","author":"Bok","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1145\/2601097.2601125","article-title":"How do people edit light fields","volume":"33","author":"Jarabo","year":"2014","journal-title":"ACM Trans. Graph."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1007\/978-3-319-10593-2_7","article-title":"Consistent matting for light field images","volume":"Volume 8692","author":"Cho","year":"2014","journal-title":"European Conference on Computer Vision, Proceedings of the ECCV 2014: Computer Vision\u2014ECCV 2014, Zurich, Switzerland, 6\u201312 September 2014"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Galdi, C., Chiesa, V., Busch, C., Correia, P., Dugelay, J., and Guillemot, C. (2019). Light Fields for Face Analysis. Sensors, 19.","DOI":"10.3390\/s19122687"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Farhood, H., Perry, S., Cheng, E., and Kim, J. (2020). Enhanced 3D Point Cloud from a Light Field Image. Remote Sens., 12.","DOI":"10.3390\/rs12071125"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Tao, M.W., Srinivasan, P.P., Malik, J., Rusinkiewicz, S., and Ramamoorthi, R. (2015). Depth from shading, defocus, and correspondence using light-field angular coherence. Comput. Vision Pattern Recognit., 1940\u20131948.","DOI":"10.1109\/CVPR.2015.7298804"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"546","DOI":"10.1109\/TPAMI.2016.2554121","article-title":"Shape estimation from shading, defocus, and correspondence using light-field angular coherence","volume":"39","author":"Tao","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Schindler, G., and Dellaert, F. (2012). 4D Cities: Analyzing, Visualizing, and Interacting with Historical Urban Photo Collections. J. Multimedia, 7.","DOI":"10.4304\/jmm.7.2.124-131"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Doulamis, A., Doulamis, N., Ioannidis, C., Chrysouli, C., Nikos, G., Dimitropoulos, K., Potsiou, C., Stathopoulou, E., and Ioannides, M. (September, January 31). 5D Modelling: An Efficient Approach for Creating Spatiotemporal Predictive 3D Maps of Large-Scale Cultural Resources. Proceedings of the ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, Taipei, Taiwan.","DOI":"10.5194\/isprsannals-II-5-W3-61-2015"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bonatto, D., Rogge, S., Schenkel, A., Ercek, R., and Lafruit, G. (2016, January 13\u201314). Explorations for real-time point cloud rendering of natural scenes in virtual reality. Proceedings of the International Conference on 3D Imaging, Li\u00e8ge, Belgium.","DOI":"10.1109\/IC3D.2016.7823453"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ioannides, M., Martins, J., \u017darni\u0107, R., and Lim, V. (2018). 4D Modelling in Cultural Heritage. Advances in Digital Cultural Heritage, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-75789-6"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Isteni\u010d, K., Gracias, N., Arnaubec, A., Escart\u00edn, J., and Garcia, R. (2019). Scale Accuracy Evaluation of Image-Based 3D Reconstruction Strategies Using Laser Photogrammetry. Remote Sens., 11.","DOI":"10.3390\/rs11182093"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bellia-Munzon, G., Martinez, J., Toselli, L., Peirano, M., Sanjurjo, D., Vallee, M., and Martinez-Ferro, M. (2020). From bench to bedside: 3D reconstruction and printing as a valuable tool for the chest wall surgeon. J. Pediatr. Surg., in press.","DOI":"10.1016\/j.jpedsurg.2020.07.010"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.autcon.2019.02.012","article-title":"A digital construction framework integrating building information modeling and reverse engineering technologies for renovation projects","volume":"102","author":"Ding","year":"2019","journal-title":"Autom. Construct."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3586","DOI":"10.1109\/TIP.2018.2814217","article-title":"Benchmark data set and method for depth estimation from light field images","volume":"27","author":"Feng","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Shin, C., Jeon, H., Yoon, Y., Kweon, I.S., and Kim, S.J. (2018, January 18\u201323). EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth From Light Field Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00499"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Rogge, S., Ceulemans, B., Bols\u00e9e, Q., and Munteanu, A. (2018, January 3\u20137). Multi-stereo matching for light field camera arrays. Proceedings of the IEEE European Signal Processing Conference, Rome, Italy.","DOI":"10.23919\/EUSIPCO.2018.8553075"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1049\/el.2019.2073","article-title":"Deep-learning based depth estimation for light field images","volume":"55","author":"Schiopu","year":"2019","journal-title":"Electron. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1032","DOI":"10.1049\/el.2018.0889","article-title":"Residual-error prediction based on deep learning for lossless image compression","volume":"54","author":"Schiopu","year":"2018","journal-title":"IET Electron. Lett."},{"key":"ref_27","first-page":"1829","article-title":"Deep-Learning based Lossless Image Coding","volume":"30","author":"Schiopu","year":"2019","journal-title":"IEEE Trans. Circ. Syst. Video Technol."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Tao, M., Hadap, S., Malik, J., and Ramamoorthi, R. (2013, January 1\u20138). Depth from combining defocus and correspondence using light-field cameras. Proceedings of the International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.89"},{"key":"ref_29","unstructured":"Tao, M., Ramamoorthi, R., Malik, J., and Efros, A.A. (2015). Unified Multi-Cue Depth Estimation from Light-Field Images: Correspondence, Defocus, Shading and Specularity, University of California. Technical Report No. UCB\/EECS-2015-174."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"888","DOI":"10.1137\/140984269","article-title":"Reliable Multiscale and Multiwindow Stereo Matching","volume":"8","author":"Buades","year":"2015","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1873","DOI":"10.1109\/TIP.2017.2666041","article-title":"Robust and dense depth estimation for light field images","volume":"26","author":"Navarro","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2484","DOI":"10.1109\/TPAMI.2017.2746858","article-title":"Robust Light Field Depth Estimation Using Occlusion-Noise Aware Data Costs","volume":"40","author":"Williem","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1109\/TPAMI.2018.2809502","article-title":"Empirical Bayesian Light-Field Stereo Matching by Robust Pseudo Random Field Modeling","volume":"41","author":"Huang","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wanner, S., and Goldluecke, B. (2012, January 16\u201321). Globally consistent depth labeling of 4D light fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6247656"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.cviu.2015.12.007","article-title":"Robust depth estimation for light field via spinning parallelogram operator","volume":"145","author":"Zhang","year":"2016","journal-title":"Comput. Vis. Image Understand."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4232","DOI":"10.1109\/TIP.2020.2970814","article-title":"Fast Depth Estimation for Light Field Cameras","volume":"29","author":"Mishiba","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Spyropoulos, A., Komodakis, N., and Mordohai, P. (2014, January 23\u201328). Learning to Detect Ground Control Points for Improving the Accuracy of Stereo Matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.210"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kim, S., Min, D., Ham, B., Kim, S., and Sohn, K. (2017, January 17\u201320). Deep stereo confidence prediction for depth estimation. Proceedings of the IEEE International Conference on Image Processing, Beijing, China.","DOI":"10.1109\/ICIP.2017.8296430"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Joung, S., Kim, S., Ham, B., and Sohn, K. (2017, January 17\u201320). Unsupervised stereo matching using correspondence consistency. Proceedings of the IEEE International Conference on Image Processing, Beijing, China.","DOI":"10.1109\/ICIP.2017.8296736"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1299","DOI":"10.1109\/TIP.2018.2878325","article-title":"Unified Confidence Estimation Networks for Robust Stereo Matching","volume":"28","author":"Kim","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ma, H., Qian, Z., Mu, T., and Shi, S. (2019). Fast and Accurate 3D Measurement Based on Light-Field Camera and Deep Learning. Sensors, 19.","DOI":"10.3390\/s19204399"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1109\/TPAMI.2003.1206509","article-title":"Stereo matching using belief propagation","volume":"25","author":"Sun","year":"2003","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Honauer, K., Johannsen, O., Kondermann, D., and Goldluecke, B. (2016, January 20\u201324). A dataset and evaluation methodology for depth estimation on 4D light fields. Proceedings of the Asian Conference on Computer Vision, Taipei, Taiwan.","DOI":"10.1007\/978-3-319-54187-7_2"},{"key":"ref_44","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_45","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Favaro, P. (2010). Recovering thin structures via nonlocal-means regularization with application to depth from defocus. Comput. Vis. Pattern Recognit., 1133\u20131140.","DOI":"10.1109\/CVPR.2010.5540089"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/s11263-007-0052-1","article-title":"Nonlocal image and movie denoising","volume":"76","author":"Buades","year":"2008","journal-title":"Int. J. Comput. Vis."},{"key":"ref_48","unstructured":"Kwon, H., Tai, Y.-W., and Lin, S. (2015). Data-driven depth map refinement via multi-scale sparse representation. Comput. Vis. Pattern Recognit., 159\u2013167."},{"key":"ref_49","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6188\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:26:55Z","timestamp":1760178415000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6188"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,30]]},"references-count":50,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["s20216188"],"URL":"https:\/\/doi.org\/10.3390\/s20216188","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,30]]}}}