{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T11:31:34Z","timestamp":1768908694040,"version":"3.49.0"},"reference-count":46,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T00:00:00Z","timestamp":1657843200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["NRF-2018R1D1A3B07044041"],"award-info":[{"award-number":["NRF-2018R1D1A3B07044041"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["NRF-2020R1A2C1101258"],"award-info":[{"award-number":["NRF-2020R1A2C1101258"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study proposes a robust depth map framework based on a convolutional neural network (CNN) to calculate disparities using multi-direction epipolar plane images (EPIs). A combination of three-dimensional (3D) and two-dimensional (2D) CNN-based deep learning networks is used to extract the features from each input stream separately. The 3D convolutional blocks are adapted according to the disparity of different directions of epipolar images, and 2D-CNNs are employed to minimize data loss. Finally, the multi-stream networks are merged to restore the depth information. A fully convolutional approach is scalable, which can handle any size of input and is less prone to overfitting. However, there is some noise in the direction of the edge. A weighted median filtering (WMF) is used to acquire the boundary information and improve the accuracy of the results to overcome this issue. Experimental results indicate that the suggested deep learning network architecture outperforms other architectures in terms of depth estimation accuracy.<\/jats:p>","DOI":"10.3390\/s22145288","type":"journal-article","created":{"date-parts":[[2022,7,18]],"date-time":"2022-07-18T01:53:22Z","timestamp":1658109202000},"page":"5288","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Depth Estimation for Integral Imaging Microscopy Using a 3D\u20132D CNN with a Weighted Median Filter"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7001-1667","authenticated-orcid":false,"given":"Shariar Md","family":"Imtiaz","sequence":"first","affiliation":[{"name":"School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0334-504X","authenticated-orcid":false,"given":"Ki-Chul","family":"Kwon","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6000-9128","authenticated-orcid":false,"given":"Md. Biddut","family":"Hossain","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Korea"}]},{"given":"Md. Shahinur","family":"Alam","sequence":"additional","affiliation":[{"name":"VL2 Center, Gallaudet University, 800 Florida Avenue NE, Washington, DC 20002, USA"}]},{"given":"Seok-Hee","family":"Jeon","sequence":"additional","affiliation":[{"name":"Department of Electronics Engineering, Incheon National University, 119 Academy-ro, Yeonsu-gu, Incheon-si 22012, Gyeonggi-do, Korea"}]},{"given":"Nam","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, Chungbuk National University, Cheongju-si 28644, Chungcheongbuk-do, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,15]]},"reference":[{"key":"ref_1","first-page":"821","article-title":"Reversible prints giving the sensation of relief","volume":"7","author":"Lippmann","year":"1908","journal-title":"J. Phys. Arch."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1598","DOI":"10.1364\/AO.36.001598","article-title":"Real-time pickup method for a three-dimensional image based on integral photography","volume":"36","author":"Okano","year":"1997","journal-title":"Appl. Opt."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1364\/AOP.10.000512","article-title":"Fundamentals of 3D imaging and displays: A tutorial on integral imaging, light-field, and plenoptic systems","volume":"10","author":"Javidi","year":"2018","journal-title":"Adv. Opt. Photonics"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Alam, S., Kwon, K.-C., Erdenebat, M.-U., Lim, Y.-T., Imtiaz, S., Sufian, M.A., Jeon, S.-H., and Kim, N. (2020, January 3\u20135). Resolution enhancement of an integral imaging microscopy using generative adversarial network. Proceedings of the 14th Pacific Rim Conference on Lasers and Electro-Optics (CLEO PR 2020), Sydney, Australia.","DOI":"10.1364\/CLEOPR.2020.C3G_4"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"12096","DOI":"10.1364\/OE.14.012096","article-title":"Three-dimensional identification of biological microorganism using integral imaging","volume":"14","author":"Javidi","year":"2006","journal-title":"Opt. Express"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"546","DOI":"10.1364\/AO.52.000546","article-title":"Advances in three-dimensional integral imaging: Sensing, display, and applications","volume":"52","author":"Xiao","year":"2013","journal-title":"Appl. Opt."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/B978-0-12-385981-5.00005-7","article-title":"Electron holography for electric and magnetic field measurements and its application for nanophysics","volume":"Volume 169","author":"Nepijko","year":"2011","journal-title":"Advances in Imaging and Electron Physics"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1230","DOI":"10.1364\/OL.29.001230","article-title":"Three-dimensional integral imaging of micro-objects","volume":"29","author":"Jang","year":"2004","journal-title":"Opt. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Chen, C., Lu, Y., and Su, M. (2010, January 15\u201319). Light field based digital refocusing using a DSLR camera with a pinhole array mask. Proceedings of the 2010 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Dallas, TX, USA.","DOI":"10.1109\/ICASSP.2010.5495014"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"19253","DOI":"10.1364\/OE.17.019253","article-title":"Resolution-enhanced integral imaging microscopy that uses lens array shifting","volume":"17","author":"Lim","year":"2009","journal-title":"Opt. Express"},{"key":"ref_11","unstructured":"(2022, June 03). Industrial Microscope OLYMPUS Stream. Available online: https:\/\/www.olympus-ims.com\/en\/microscope\/stream2\/."},{"key":"ref_12","unstructured":"(2022, June 03). Stereo Microscopes. Available online: https:\/\/www.olympus-lifescience.com\/en\/microscopes\/stereo\/."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Levoy, M., Ng, R., Adams, A., Footer, M., and Horowitz, M. (August, January 30). Light field microscopy. Proceedings of the ACM SIGGRAPH 2006 papers (SIGGRAPH \u201906), San Diego, CA, USA.","DOI":"10.1145\/1179352.1141976"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"5079","DOI":"10.1364\/OL.436201","article-title":"High-quality 3d display system for an integral imaging microscope using a simplified direction-inversed computation based on user interaction","volume":"46","author":"Kwon","year":"2021","journal-title":"Opt. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Farhood, H., Perry, S., Cheng, E., and Kim, J. (2020). Enhanced 3D point cloud from a light field image. Remote Sens., 12.","DOI":"10.3390\/rs12071125"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"889","DOI":"10.1109\/JDT.2014.2361147","article-title":"Augmented reality 3d displays with micro integral imaging","volume":"11","author":"Wang","year":"2015","journal-title":"J. Disp. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"5806","DOI":"10.1364\/AO.43.005806","article-title":"Integral imaging with improved depth of field by use of amplitude-modulated microlens arrays","volume":"43","author":"Javidi","year":"2004","journal-title":"Appl. Opt."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2072","DOI":"10.1364\/OE.24.002072","article-title":"Integral imaging microscopy with enhanced depth-of-field using a spatial multiplexing","volume":"24","author":"Kwon","year":"2016","journal-title":"Opt. Express"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"34","DOI":"10.3807\/COPP.2017.1.1.034","article-title":"Resolution of temporal-multiplexing and spatial-multiplexing stereoscopic televisions","volume":"1","author":"Kim","year":"2017","journal-title":"Curr. Opt. Photonics"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3209","DOI":"10.1364\/OL.42.003209","article-title":"Enhanced depth-of-field of an integral imaging microscope using a bifocal holographic optical element-micro lens array","volume":"42","author":"Kwon","year":"2017","journal-title":"Opt. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"30503","DOI":"10.1364\/OE.25.030503","article-title":"Enhancement of the depth-of-field of integral imaging microscope by using switchable bifocal liquid-crystalline polymer micro lens array","volume":"25","author":"Kwon","year":"2017","journal-title":"Opt. Express"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Alam, M.S., Kwon, K.-C., Erdenebat, M.-U., Abbass, M.Y., Alam, M.A., and Kim, N. (2021). Super-resolution enhancement method based on generative adversarial network for integral imaging microscopy. Sensors, 21.","DOI":"10.3390\/s21062164"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yang, Q., Tan, K.-H., Culbertson, B., and Apostolopoulos, J. (2010, January 4\u20136). Fusion of active and passive sensors for fast 3D capture. Proceedings of the 2010 IEEE International Workshop on Multimedia Signal Processing (MMSP), Saint-Malo, France.","DOI":"10.1109\/MMSP.2010.5661996"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Honauer, K., Johannsen, O., Kondermann, D., and Goldluecke, B. (2016, January 20\u201324). A dataset and evaluation methodology for depth estimation on 4d light fields. Proceedings of the 13th Asian Conference on Computer Vision (ACCV), Taipei, Taiwan.","DOI":"10.1007\/978-3-319-54187-7_2"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wang, Q., Zhang, L., Bertinetto, L., Hu, W., and Torr, P.H.S. (2019, January 15\u201320). Fast online object tracking and segmentation: A unifying approach. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00142"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Kim, S., Jun, D., Kim, B.-G., Lee, H., and Rhee, E. (2021). Single image super-resolution method using cnn-based lightweight neural networks. Appl. Sci., 11.","DOI":"10.3390\/app11031092"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shin, C., Jeon, H.-G., Yoon, Y., Kweon, I.S., and Kim, S.J. (2018, January 18\u201323). EPINET: A fully-convolutional neural network using epipolar geometry for depth from light field images. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00499"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Heber, S., Yu, W., and Pock, T. (2017, January 22\u201329). Neural epi-volume networks for shape from light field. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.247"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Rogge, S., Schiopu, I., and Munteanu, A. (2020). Depth estimation for light-field images using stereo matching and convolutional neural networks. Sensors, 20.","DOI":"10.3390\/s20216188"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Han, L., Huang, X., Shi, Z., and Zheng, S. (2021). Depth estimation from light field geometry using convolutional neural networks. Sensors, 21.","DOI":"10.3390\/s21186061"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wu, G., Zhao, M., Wang, L., Dai, Q., Chai, T., and Liu, Y. (2017, January 21\u201326). Light field reconstruction using deep convolutional network on EPI. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.178"},{"key":"ref_32","unstructured":"Li, K., Zhang, J., Sun, R., Zhang, X., and Gao, J. (2020). EPI-based oriented relation networks for light field depth estimation. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Luo, Y., Zhou, W., Fang, J., Liang, L., Zhang, H., and Dai, G. (2017, January 14\u201318). EPI-patch based convolutional neural network for depth estimation on 4D light field. Proceedings of the International Conference on Neural Information Processing, Guangzhou, China.","DOI":"10.1007\/978-3-319-70090-8_65"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5867","DOI":"10.1109\/TIP.2019.2923323","article-title":"A framework for learning depth from a flexible subset of dense and sparse light field views","volume":"28","author":"Shi","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., and Brox, T. (2017, January 21\u201326). FlowNet 2.0: Evolution of optical flow estimation with deep networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.179"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3586","DOI":"10.1109\/TIP.2018.2814217","article-title":"Benchmark data set and method for depth estimation from light field images","volume":"27","author":"Feng","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1364\/JOSAA.388555","article-title":"Light-field-depth-estimation network based on epipolar geometry and image segmentation","volume":"37","author":"Wang","year":"2020","journal-title":"J. Opt. Soc. Am. A"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Li, Y., Zhang, L., Wang, Q., and Lafruit, G. (2020, January 4\u20138). MANET: Multi-scale aggregated network for light field depth estimation. Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053532"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Faluvegi, A., Bolsee, Q., Nedevschi, S., Dadarlat, V.-T., and Munteanu, A. (2019, January 11\u201312). A 3D convolutional neural network for light field depth estimation. Proceedings of the 2019 International Conference on 3D Immersion (IC3D), Brussels, Belgium.","DOI":"10.1109\/IC3D48390.2019.8975996"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Leistner, T., Schilling, H., Mackowiak, R., Gumhold, S., and Rother, C. (2019, January 16\u201319). Learning to think outside the box: Wide-baseline light field depth estimation with EPI-shift. Proceedings of the 2019 International Conference on 3D Vision (3DV), Qu\u00e9bec City, QC, Canada.","DOI":"10.1109\/3DV.2019.00036"},{"key":"ref_41","first-page":"524","article-title":"Identification and correction of microlens-array error in an integral-imaging-microscopy system","volume":"5","author":"Imtiaz","year":"2021","journal-title":"Curr. Opt. Photonics"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/JPHOT.2020.3010319","article-title":"Advanced three-dimensional visualization system for an integral imaging microscope using a fully convolutional depth estimation network","volume":"12","author":"Kwon","year":"2020","journal-title":"IEEE Photonics J."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"4450","DOI":"10.1364\/AO.53.004450","article-title":"Real-time interactive display for integral imaging microscopy","volume":"53","author":"Kwon","year":"2014","journal-title":"Appl. Opt."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Ma, Z., He, K., Wei, Y., Sun, J., and Wu, E. (2013, January 1\u20138). Constant time weighted median filtering for stereo matching and beyond. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, Australia.","DOI":"10.1109\/ICCV.2013.13"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"15347","DOI":"10.1109\/ACCESS.2020.2966794","article-title":"Cross fusion-based low dynamic and saturated image enhancement for infrared search and tracking systems","volume":"8","author":"Kim","year":"2020","journal-title":"IEEE Access"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5288\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:50:59Z","timestamp":1760140259000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5288"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,15]]},"references-count":46,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22145288"],"URL":"https:\/\/doi.org\/10.3390\/s22145288","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,15]]}}}