{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:02:31Z","timestamp":1760230951617,"version":"build-2065373602"},"reference-count":37,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2022,8,21]],"date-time":"2022-08-21T00:00:00Z","timestamp":1661040000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["61702384","61502357"],"award-info":[{"award-number":["61702384","61502357"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Light field (LF) image depth estimation is a critical technique for LF-related applications such as 3D reconstruction, target detection, and tracking. The refocusing property of LF images provide rich information for depth estimations; however, it is still challenging in cases of occlusion regions, edge regions, noise interference, etc. The epipolar plane image (EPI) of LF can effectively deal with the depth estimation because of its characteristics of multidirectionality and pixel consistency\u2014in which the LF depth estimations are converted to calculate the EPI slope. This paper proposed an EPI LF depth estimation algorithm based on a directional relationship model and attention mechanism. Unlike the subaperture LF depth estimation method, the proposed method takes EPIs as input images. Specifically, a directional relationship model was used to extract direction features of the horizontal and vertical EPIs, respectively. Then, a multiviewpoint attention mechanism combining channel attention and spatial attention is used to give more weight to the EPI slope information. Subsequently, multiple residual modules are used to eliminate the redundant features that interfere with the EPI slope information\u2014in which a small stride convolution operation is used to avoid losing key EPI slope information. The experimental results revealed that the proposed algorithm outperformed the compared algorithms in terms of accuracy.<\/jats:p>","DOI":"10.3390\/s22166291","type":"journal-article","created":{"date-parts":[[2022,8,22]],"date-time":"2022-08-22T01:56:40Z","timestamp":1661133400000},"page":"6291","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["EPI Light Field Depth Estimation Based on a Directional Relationship Model and Multiviewpoint Attention Mechanism"],"prefix":"10.3390","volume":"22","author":[{"given":"Ming","family":"Gao","sequence":"first","affiliation":[{"name":"School of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huiping","family":"Deng","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sen","family":"Xiang","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jin","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zeyang","family":"He","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1109\/34.121783","article-title":"Single Lens Stereo with a Plenoptic Camera","volume":"14","author":"Adelson","year":"1992","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_2","unstructured":"Ren, N. (2006). Digital Light Field Photography. [Ph.D. Thesis, Stanford University]."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Levoy, M., and Hanrahan, P. (1996, January 4\u20139). Light field rendering. Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, USA.","DOI":"10.1145\/237170.237199"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., and Urtasun, R. (2016, January 27\u201330). Monocular 3d object detection for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.236"},{"key":"ref_5","unstructured":"Feng, M. (2019). Deep Learning Based 3D Perception and Recognition for Robot Vision. [Ph.D. Thesis, Hunan University]."},{"key":"ref_6","unstructured":"Sajjan, S., Moore, M., Pan, M., Nagaraja, G., Lee, J., Zeng, A., and Song, S. (June, January 31). Clear Grasp: 3D Shape Estimation of Transparent Objects for Manipulation. Proceedings of the IEEE International Conference on Robotics and Automation, Online."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shin, C., Jeon, H., Yoon, Y., Kweon, I.S., and Kim, S.J. (2018, January 18\u201323). Epinet: A fully-convolutional neural network using epipolar geometry for depth from light field images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00499"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Tsai, Y., Liu, Y., Ouhyoung, M., and Chuang, Y.Y. (2020, January 7\u201312). Attention-based view selection networks for light-field disparity estimation. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i07.6888"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Tao, M., Hadap, S., Malik, J., and Ramamoorthi, R. (2013, January 1\u20138). Depth from combining defocus and correspondence using light-field cameras. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.89"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tao, M., Srinivasan, P., Malik, J., Rusinkiewicz, S., and Ramamoorthi, R. (2015, January 7\u201312). Depth from shading, defocus, and correspondence using lightfield angular coherence. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298804"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wang, T., Efros, A., and Ramamoorthi, R. (2015, January 7\u201313). Occlusion-aware depth estimation using light-field cameras. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.398"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Williem, W., and Park, I. (2016, January 27\u201330). Robust light field depth estimation for noisy scene with occlusion. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.476"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1007\/BF00128525","article-title":"Epipolar-plane image analysis: An approach to determining structure from motion","volume":"1","author":"Bolles","year":"1987","journal-title":"Int. J. Comput. Vis."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Johannsen, O., Sulc, A., and Goldluecke, B. (2016, January 27\u201330). What sparse light field coding reveals about scene structure. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.355"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1109\/TPAMI.2013.147","article-title":"Variational light field analysis for disparity estimation and super-resolution","volume":"36","author":"Wanner","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Jeon, H., Park, J., Choe, G., Park, J., Bok, Y., Tai, Y., and Kweon, I.S. (2015, January 7\u201312). Accurate depth map estimation from a lenslet light field camera. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298762"},{"key":"ref_17","first-page":"2484","article-title":"Robust light field depth estimation using occlusion-noise aware data costs","volume":"40","author":"Park","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhou, W., Liang, L., Zhang, H., Lumsdaine, A., and Lin, L. (2018, January 20\u201324). Scale and orientation aware epi-patch learning for light field depth estimation. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8545490"},{"key":"ref_19","unstructured":"Li, K., Zhang, J., Sun, R., Zhang, X., and Gao, J. (2020). Epi-based oriented relation networks for light field depth estimation. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Luo, Y., Zhou, W., Fang, J., Liang, L., Zhang, H., and Dai, G. (2017). Epi-patch based convolutional neural network for depth estimation on 4d light field. International Conference on Neural Information Processing, Springer.","DOI":"10.1007\/978-3-319-70090-8_65"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Leistner, T., Schilling, H., Mackowiak, R., Gumhold, S., and Rother, C. (2019, January 16\u201319). Learning to think outside the box: Wide-baseline light field depth estimation with EPI-shift. Proceedings of the 2019 International Conference on 3D Vision (3DV), Qu\u00e9bec City, QC, Canada.","DOI":"10.1109\/3DV.2019.00036"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.cviu.2015.12.007","article-title":"Robust depth estimation for light field via spinning parallelogram operator","volume":"145","author":"Zhang","year":"2016","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1016\/j.patcog.2017.09.010","article-title":"Occlusion-aware depth estimation for light field using multi-orientation EPIs","volume":"74","author":"Sheng","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Heber, S., Yu, W., and Pock, T. (2017, January 22\u201329). Neural epi-volume networks for shape from light field. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.247"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chen, J., Zhang, S., and Lin, Y. (2021, January 2\u20139). Attention-based multi-level fusion network for light field depth estimation. Proceedings of the AAAI Conference on Artificial Intelligence, Online.","DOI":"10.1609\/aaai.v35i2.16185"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Huang, Z., Hu, X., Xue, Z., Xu, W., and Yue, T. (2021, January 11\u201317). Fast Light-Field Disparity Estimation with Multi-Disparity-Scale Cost Aggregation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00626"},{"key":"ref_27","first-page":"12","article-title":"Anti-highlighting method for optical field depth estimation","volume":"25","author":"Wang","year":"2020","journal-title":"Chin. J. Image Graph."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5867","DOI":"10.1109\/TIP.2019.2923323","article-title":"A framework for learning depth from a flexible subset of dense and sparse light field view","volume":"28","author":"Shi","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., and Brox, T. (2017, January 21\u201326). Flownet 2.0: Evolution of optical flow estimation with deep networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.179"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2288","DOI":"10.1109\/TIP.2021.3051761","article-title":"A Lightweight Depth Estimation Network for Wide-Baseline Light Fields","volume":"30","author":"Li","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, Y., Wang, L., Wu, G., Yang, J., An, W., Yu, J., and Guo, Y. (2022). Disentangling Light Fields for Super-Resolution and Disparity Estimation. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1109\/TPAMI.2022.3152488"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, Y., Wang, L., Liang, Z., Yang, J., An, W., and Guo, Y. (2022). Occlusion-Aware Cost Constructor for Light Field Depth Estimation. arXiv.","DOI":"10.1109\/CVPR52688.2022.01919"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Heber, S., and Pock, T. (2016, January 27\u201330). Convolutional networks for shape from light field. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.407"},{"key":"ref_34","first-page":"5","article-title":"U-shaped Networks for Shape from Light Field","volume":"3","author":"Heber","year":"2016","journal-title":"BMVC British Machine Vision Conference 2016."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zhou, W., Zhou, E., Yan, Y., Lin, L., and Lumsdaine, A. (2019, January 22\u201325). Learning depth cues from focal stack for light field depth estimation. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, China.","DOI":"10.1109\/ICIP.2019.8804270"},{"key":"ref_36","unstructured":"Honauer, K., Johannsen, O., Kondermann, D., and Goldluecke, B. (2016). A dataset and evaluation methodology for depth estimation on 4D light fields. Asian Conference on Computer Vision, Springer."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Johannsen, O., Honauer, K., Goldluecke, B., Alperovich, A., Battisti, F., Bok, Y., Brizzi, M., Carli, M., Choe, G., and Diebold, M. (2017, January 21\u201326). A Taxonomy and Evaluation of Dense Light Field Depth Estimation Algorithms. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.226"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6291\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:13:13Z","timestamp":1760141593000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6291"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,21]]},"references-count":37,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["s22166291"],"URL":"https:\/\/doi.org\/10.3390\/s22166291","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,8,21]]}}}