{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T18:08:18Z","timestamp":1772042898371,"version":"3.50.1"},"reference-count":53,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T00:00:00Z","timestamp":1711584000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"KIT-Publication Fund of the Karlsruhe Institute of Technology"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In this contribution we evaluate the 3D geometry reconstructed by Neural Radiance Fields (NeRFs) of an object\u2019s occluded parts behind obstacles through a point cloud comparison in 3D space against traditional Multi-View Stereo (MVS), addressing the accuracy and completeness. The key challenge lies in recovering the underlying geometry, completing the occluded parts of the object and investigating if NeRFs can compete against traditional MVS for scenarios where the latter falls short. In addition, we introduce a new \u201cobSTaclE, occLusion and visibiLity constrAints\u201d dataset named STELLA concerning transparent and non-transparent obstacles in real-world scenarios since there is no existing dataset dedicated to this problem setting to date. Considering that the density field represents the 3D geometry of NeRFs and is solely position-dependent, we propose an effective approach for extracting the geometry in the form of a point cloud. We voxelize the whole density field and apply a 3D density-gradient based Canny edge detection filter to better represent the object\u2019s geometric features. The qualitative and quantitative results demonstrate NeRFs\u2019 ability to capture geometric details of the occluded parts in all scenarios, thus outperforming in completeness, as our voxel-based point cloud extraction approach achieves point coverage up to 93%. However, MVS remains a more accurate image-based 3D reconstruction method, deviating from the ground truth 2.26 mm and 3.36 mm for each obstacle scenario respectively.<\/jats:p>","DOI":"10.3390\/rs16071188","type":"journal-article","created":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T12:09:40Z","timestamp":1711627780000},"page":"1188","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Vision through Obstacles\u20143D Geometric Reconstruction and Evaluation of Neural Radiance Fields (NeRFs)"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-8062-6623","authenticated-orcid":false,"given":"Ivana","family":"Petrovska","sequence":"first","affiliation":[{"name":"Institute of Photogrammetry and Remote Sensing (IPF), Karlsruhe Institute of Technology (KIT), 76131 Karlsruhe, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4322-3074","authenticated-orcid":false,"given":"Boris","family":"Jutzi","sequence":"additional","affiliation":[{"name":"Institute of Photogrammetry and Remote Sensing (IPF), Karlsruhe Institute of Technology (KIT), 76131 Karlsruhe, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Dumic, E., and da Silva Cruz, L.A. (2023). Subjective Quality Assessment of V-PCC-Compressed Dynamic Point Clouds Degraded by Packet Losses. Sensors, 23.","DOI":"10.3390\/s23125623"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3603534","article-title":"Point cloud quality assessment: Dataset construction and learning-based no-reference metric","volume":"19","author":"Liu","year":"2023","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sch\u00f6nberger, J.L., Zheng, E., Frahm, J.M., and Pollefeys, M. (2016, January 11\u201314). Pixelwise view selection for unstructured multi-view stereo. Proceedings of the Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Proceedings, Part III 14.","DOI":"10.1007\/978-3-319-46487-9_31"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Schonberger, J.L., and Frahm, J.M. (2016, January 27\u201330). Structure-from-motion revisited. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.445"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Stathopoulou, E.K., Battisti, R., Cernea, D., Remondino, F., and Georgopoulos, A. (2021). Semantically Derived Geometric Constraints for MVS Reconstruction of Textureless Areas. Remote Sens., 13.","DOI":"10.3390\/rs13061053"},{"key":"ref_6","unstructured":"Sitzmann, V., Zollh\u00f6fer, M., and Wetzstein, G. (2019, January 8\u201314). Scene representation networks: Continuous 3D-structure-aware neural scene representations. Proceedings of the Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1145\/3503250","article-title":"Nerf: Representing scenes as neural radiance fields for view synthesis","volume":"65","author":"Mildenhall","year":"2021","journal-title":"Commun. ACM"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yan, Z., Li, C., and Lee, G.H. (2023, January 17\u201324). Nerf-ds: Neural radiance fields for dynamic specular objects. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00801"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.neucom.2023.01.062","article-title":"MG-MVSNet: Multiple granularities feature fusion network for multi-view stereo","volume":"528","author":"Zhang","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"391","DOI":"10.5194\/isprs-archives-XLIII-B2-2021-391-2021","article-title":"Enhancing Geometric Edge Details in MVS Reconstruction","volume":"43","author":"Stathopoulou","year":"2021","journal-title":"Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci."},{"key":"ref_11","unstructured":"Zhu, Q., Min, C., Wei, Z., Chen, Y., and Wang, G. (2021). Deep learning for multi-view stereo via plane sweep: A survey. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Zhu, J., and Lin, L. (2023, January 17\u201324). Multi-View Stereo Representation Revist: Region-Aware MVSNet. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01667"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Peng, R., Hu, Y., and Wang, R. (2023, January 17\u201324). GeoMVSNet: Learning Multi-View Stereo With Geometry Perception. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.02060"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Yamashita, K., Enyo, Y., Nobuhara, S., and Nishino, K. (2023, January 2\u20137). nLMVS-Net: Deep Non-Lambertian Multi-View Stereo. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV56688.2023.00305"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1007\/s00138-023-01380-8","article-title":"PM-MVS: PatchMatch multi-view stereo","volume":"34","author":"Ito","year":"2023","journal-title":"Mach. Vis. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Barron, J.T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., and Srinivasan, P.P. (2021, January 11\u201317). Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00580"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., and Hedman, P. (2022, January 18\u201324). Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00539"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Martin-Brualla, R., Radwan, N., Sajjadi, M.S., Barron, J.T., Dosovitskiy, A., and Duckworth, D. (2021, January 20\u201325). Nerf in the wild: Neural radiance fields for unconstrained photo collections. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00713"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Jiang, Y., Hedman, P., Mildenhall, B., Xu, D., Barron, J.T., Wang, Z., and Xue, T. (2023, January 17\u201324). AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00013"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, X., Kundu, A., Funkhouser, T., Guibas, L., Su, H., and Genova, K. (2023, January 17\u201324). Nerflets: Local radiance fields for efficient structure-aware 3d scene representation from 2d supervision. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00800"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, Z., M\u00fcller, T., Evans, A., Taylor, R.H., Unberath, M., Liu, M.Y., and Lin, C.H. (2023, January 17\u201324). Neuralangelo: High-Fidelity Neural Surface Reconstruction. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00817"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Hu, B., Huang, J., Liu, Y., Tai, Y.W., and Tang, C.K. (2023, January 17\u201324). NeRF-RPN: A general framework for object detection in NeRFs. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.02253"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Xu, Q., Xu, Z., Philip, J., Bi, S., Shu, Z., Sunkavalli, K., and Neumann, U. (2022, January 18\u201324). Point-nerf: Point-based neural radiance fields. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00536"},{"key":"ref_24","unstructured":"Zimny, D., Trzci\u0144ski, T., and Spurek, P. (2022). Points2nerf: Generating neural radiance fields from 3D point cloud. arXiv."},{"key":"ref_25","first-page":"1","article-title":"Instant neural graphics primitives with a multiresolution hash encoding","volume":"41","author":"Evans","year":"2022","journal-title":"ACM Trans. Graph. (ToG)"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Guo, Y.C., Kang, D., Bao, L., He, Y., and Zhang, S.H. (2022). NeRFReN: Neural Radiance Fields with Reflections. arXiv.","DOI":"10.1109\/CVPR52688.2022.01786"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Tong, J., Muthu, S., Maken, F.A., Nguyen, C., and Li, H. (2023, January 17\u201324). Seeing Through the Glass: Neural 3D Reconstruction of Object Inside a Transparent Container. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01208"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, D., Zhang, T., and S\u00fcsstrunk, S. (2023). NEMTO: Neural Environment Matting for Novel View and Relighting Synthesis of Transparent Objects. arXiv.","DOI":"10.1109\/ICCV51070.2023.00036"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Li, Z., Long, X., Wang, Y., Cao, T., Wang, W., Luo, F., and Xiao, C. (2023). NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing. arXiv.","DOI":"10.1109\/ICCV51070.2023.01700"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhan, X., Pan, X., Dai, B., Liu, Z., Lin, D., and Loy, C.C. (2020, January 13\u201319). Self-supervised scene de-occlusion. Proceedings of the IEEE\/CVF conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00384"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhou, Q., Wang, S., Wang, Y., Huang, Z., and Wang, X. (2021, January 20\u201325). Human de-occlusion: Invisible perception and recovery for humans. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00369"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1109\/TMM.2019.2929005","article-title":"Widerperson: A diverse dataset for dense pedestrian detection in the wild","volume":"22","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Multimed."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhuo, J., Chen, Z., Lai, J., and Wang, G. (2018, January 23\u201327). Occluded person re-identification. Proceedings of the 2018 IEEE International Conference on Multimedia and Expo (ICME), San Diego, CA, USA.","DOI":"10.1109\/ICME.2018.8486568"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1294","DOI":"10.1109\/TMM.2022.3141267","article-title":"Learning disentangled representation implicitly via transformer for occluded person re-identification","volume":"25","author":"Jia","year":"2022","journal-title":"IEEE Trans. Multimed."},{"key":"ref_35","unstructured":"Ouyang, W., and Wang, X. (2012, January 16\u201321). A discriminative deep model for pedestrian detection with occlusion handling. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lee, H., and Park, J. (2022, January 19\u201320). Instance-wise occlusion and depth orders in natural scenes. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.02053"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hodan, T., Haluza, P., Obdr\u017e\u00e1lek, \u0160., Matas, J., Lourakis, M., and Zabulis, X. (2017, January 24\u201331). T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects. Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA.","DOI":"10.1109\/WACV.2017.103"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Tyree, S., Tremblay, J., To, T., Cheng, J., Mosier, T., Smith, J., and Birchfield, S. (2022, January 23\u201327). 6-DoF pose estimation of household objects for robotic manipulation: An accessible dataset and benchmark. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9981838"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1016\/j.biosystemseng.2021.06.001","article-title":"Image-based size estimation of broccoli heads under varying degrees of occlusion","volume":"208","author":"Blok","year":"2021","journal-title":"Biosyst. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kaskman, R., Zakharov, S., Shugurov, I., and Ilic, S. (2019, January 27\u201328). Homebreweddb: RGB-D dataset for 6D pose estimation of 3D objects. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00338"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Koch, T., Liebel, L., Fraundorfer, F., and Korner, M. (2018, January 8\u201314). Evaluation of cnn-based single-image depth estimation methods. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.","DOI":"10.1007\/978-3-030-11015-4_25"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Tancik, M., Weber, E., Ng, E., Li, R., Yi, B., Wang, T., Kristoffersen, A., Austin, J., Salahi, K., and Ahuja, A. (2023, January 6\u201310). Nerfstudio: A Modular Framework for Neural Radiance Field Development. Proceedings of the ACM SIGGRAPH 2023 Conference Proceedings, Los Angeles, CA, USA.","DOI":"10.1145\/3588432.3591516"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Remondino, F., Karami, A., Yan, Z., Mazzacca, G., Rigon, S., and Qin, R. (2023). A critical analysis of nerf-based 3d reconstruction. Remote Sens., 15.","DOI":"10.3390\/rs15143585"},{"key":"ref_44","unstructured":"Jiang, H., Li, R., Sun, H., Tai, Y.W., and Tang, C.K. (2023). Registering Neural Radiance Fields as 3D Density Images. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"153","DOI":"10.5194\/isprs-archives-XLVIII-1-W3-2023-153-2023","article-title":"Geometric Accuracy Analysis between Neural Radiance Fields (NeRFs) and Terrestrial laser scanning (TLS)","volume":"48","author":"Petrovska","year":"2023","journal-title":"Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Oechsle, M., Peng, S., and Geiger, A. (2021, January 11\u201317). Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00554"},{"key":"ref_47","first-page":"71","article-title":"3D Density-Gradient Based Edge Detection on Neural Radiance Fields (NeRFS) for Geometric Reconstruction","volume":"48","author":"Jutzi","year":"2023","journal-title":"Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Ni, H., Lin, X., Ning, X., and Zhang, J. (2016). Edge Detection and Feature Line Tracing in 3D-Point Clouds by Analyzing Geometric Properties of Neighborhoods. Remote Sens., 8.","DOI":"10.3390\/rs8090710"},{"key":"ref_49","first-page":"5","article-title":"Methods of Image Edge Detection: A Review","volume":"4","author":"Mutneja","year":"2015","journal-title":"J. Electr. Electron. Syst."},{"key":"ref_50","first-page":"586","article-title":"Method for registration of 3-D shapes","volume":"Volume 1611","author":"Besl","year":"1992","journal-title":"Proceedings of the Sensor Fusion IV: Control Paradigms and Data Structures"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"5481","DOI":"10.5194\/gmd-15-5481-2022","article-title":"Root-mean-square error (RMSE) or mean absolute error (MAE): When to use them or not","volume":"15","author":"Hodson","year":"2022","journal-title":"Geosci. Model Dev."},{"key":"ref_52","unstructured":"P\u00fcschel, J. (2011). Vergleich eines 3D-Modells zwischen Bundler und Breuckmann. [Bachelor\u2019s Thesis, Institute for Photogrammetry and Remote Sensing, Karlsruhe Institute of Technology\u2014KIT]."},{"key":"ref_53","unstructured":"J\u00e4ger, M., Landgraf, S., and Jutzi, B. (2023). Density Uncertainty Quantification with NeRF-Ensembles: Impact of Data and Scene Constraints. arXiv."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/7\/1188\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:20:19Z","timestamp":1760106019000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/7\/1188"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,28]]},"references-count":53,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["rs16071188"],"URL":"https:\/\/doi.org\/10.3390\/rs16071188","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,28]]}}}