{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,21]],"date-time":"2025-12-21T06:23:25Z","timestamp":1766298205549,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,4]],"date-time":"2021-02-04T00:00:00Z","timestamp":1612396800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100012774","name":"Innovationsfonden","doi-asserted-by":"publisher","award":["8057-00011B"],"award-info":[{"award-number":["8057-00011B"]}],"id":[{"id":"10.13039\/100012774","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>When 3D scanning objects, the objective is usually to obtain a continuous surface. However, most surface scanning methods, such as structured light scanning, yield a point cloud. Obtaining a continuous surface from a point cloud requires a subsequent surface reconstruction step, which is directly affected by any error from the computation of the point cloud. In this work, we propose a one-step approach in which we compute the surface directly from structured light images. Our method minimizes the least-squares error between photographs and renderings of a triangle mesh, where the vertex positions of the mesh are the parameters of the minimization problem. To ensure fast iterations during optimization, we use differentiable rendering, which computes images and gradients in a single pass. We present simulation experiments demonstrating that our method for computing a triangle mesh has several advantages over approaches that rely on an intermediate point cloud. Our method can produce accurate reconstructions when initializing the optimization from a sphere. We also show that our method is good at reconstructing sharp edges and that it is robust with respect to image noise. In addition, our method can improve the output from other reconstruction algorithms if we use these for initialization.<\/jats:p>","DOI":"10.3390\/s21041068","type":"journal-article","created":{"date-parts":[[2021,2,4]],"date-time":"2021-02-04T21:29:27Z","timestamp":1612474167000},"page":"1068","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Surface Reconstruction from Structured Light Images Using Differentiable Rendering"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0820-5871","authenticated-orcid":false,"given":"Janus N\u00f8rtoft","family":"Jensen","sequence":"first","affiliation":[{"name":"DTU Compute, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9956-9226","authenticated-orcid":false,"given":"Morten","family":"Hannemose","sequence":"additional","affiliation":[{"name":"DTU Compute, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2583-0660","authenticated-orcid":false,"given":"J. Andreas","family":"B\u00e6rentzen","sequence":"additional","affiliation":[{"name":"DTU Compute, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4964-0645","authenticated-orcid":false,"given":"Jakob","family":"Wilm","sequence":"additional","affiliation":[{"name":"SDU Robotics, University of Southern Denmark, 5230 Odense, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0603-3669","authenticated-orcid":false,"given":"Jeppe Revall","family":"Frisvad","sequence":"additional","affiliation":[{"name":"DTU Compute, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0068-8170","authenticated-orcid":false,"given":"Anders Bjorholm","family":"Dahl","sequence":"additional","affiliation":[{"name":"DTU Compute, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.patrec.2014.03.023","article-title":"3D reconstruction methods for digital preservation of cultural heritage: A survey","volume":"50","author":"Gomes","year":"2014","journal-title":"Pattern Recognit. Lett."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Nocerino, E., Stathopoulou, E.K., Rigon, S., and Remondino, F. (2020). Surface reconstruction assessment in photogrammetric applications. Sensors, 20.","DOI":"10.3390\/s20205863"},{"key":"ref_3","unstructured":"Turk, G. (2020, December 21). The Stanford Bunny. Available online: https:\/\/www.cc.gatech.edu\/~turk\/bunny\/bunny.html."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/cgf.12802","article-title":"A survey of surface reconstruction from point clouds","volume":"36","author":"Berger","year":"2017","journal-title":"Comput. Graph. Forum"},{"key":"ref_5","unstructured":"Horn, B.K.P. (1970). Shape from Shading: A Method for Obtaining the Shape of a Smooth Opaque Object from One View. [Ph.D. Thesis, Massachusetts Institute of Technology]. MAC TR-79."},{"key":"ref_6","unstructured":"Winston, P.H. (1975). Obtaining Shape from Shading Information. The Psychology of Computer Vision, McGraw-Hill. Chapter 4."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/S0010-4485(96)00056-5","article-title":"Three-dimensional object reconstruction from two-dimensional images","volume":"29","author":"Rockwood","year":"1997","journal-title":"Comput.-Aided Des."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1117\/12.410896","article-title":"Image-based multiresolution shape recovery by surface deformation","volume":"4309","author":"Zhang","year":"2000","journal-title":"Proc. SPIE"},{"key":"ref_9","unstructured":"Inagaki, A., Tagawa, N., Minagawa, A., and Moriya, T. (2001, January 7\u201310). Computation of shape and reflectance of 3D object using 3-D object using moir\u00e9 phase and reflection model. Proceedings of the International Conference on Image Processing (ICIP 2001), Thessaloniki, Greece."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Naganuma, S., Tagawa, N., and Minagawa, A. (2003, January 9\u201312). Estimation of 3D shape and reflectance using multiple moir\u00e9 images and shading model. Proceedings of the ACM Symposium on Applied Computing (SAC 2003), Melbourne, FL, USA.","DOI":"10.1145\/952532.952717"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Naganuma, S., Tagawa, N., Minagawa, A., and Moriya, T. (2004, January 26). Simultaneous determination of object shape and color by Moire analysis using a reflection model. Proceedings of the International Conference on Pattern Recognition (ICPR 2004), Cambridge, UK.","DOI":"10.1109\/ICPR.2004.1334503"},{"key":"ref_12","unstructured":"Isidoro, J., and Sclaroff, S. (2002, January 19\u201321). Stochastic mesh-based multiview reconstruction. Proceedings of the First International Symposium on 3D Data Processing Visualization and Transmission, Padova, Italy."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Isidoro, J., and Sclaroff, S. (2003, January 13\u201316). Stochastic Refinement of the Visual Hull to Satisfy Photometric and Silhouette Consistency Constraints. Proceedings of the International Conference on Computer Vision (ICCV 2003), Nice, France.","DOI":"10.1109\/ICCV.2003.1238645"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Yu, T., Xu, N., and Ahuja, N. (2004, January 11\u201314). Shape and view independent reflectance map from multiple views. Proceedings of the European Conference on Computer Vision (ECCV 2004), Prague, Czech Republic.","DOI":"10.1007\/978-3-540-24673-2_48"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/s11263-006-9373-8","article-title":"Shape and view independent reflectance map from multiple views","volume":"73","author":"Yu","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wu, C., Wilburn, B., Matsushita, Y., and Theobalt, C. (2011, January 20\u201325). High-quality shape from multi-view stereo and shading under general illumination. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR 2011), Providence, RI, USA.","DOI":"10.1109\/CVPR.2011.5995388"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ha, H., Oh, T.H., and Kweon, I.S. (2015, January 19\u201322). A multi-view structured-light system for highly accurate 3D modeling. Proceedings of the International Conference on 3D Vision (3DV 2015), Lyon, France.","DOI":"10.1109\/3DV.2015.21"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"228:1","DOI":"10.1145\/3355089.3356510","article-title":"Reparameterizing discontinuous integrands for differentiable rendering","volume":"38","author":"Loubet","year":"2019","journal-title":"ACM Trans. Graph."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, S., Li, T., Chen, W., and Li, H. (November, January 27). Soft rasterizer: A differentiable renderer for image-based 3D reasoning. Proceedings of the International Conference on Computer Vision (ICCV 2019), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00780"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Cazals, F., and Giesen, J. (2006). Delaunay triangulation based surface reconstruction. Effective Computational Geometry for Curves and Surfaces, Springer.","DOI":"10.1007\/978-3-540-33259-6_6"},{"key":"ref_21","unstructured":"Kazhdan, M., Bolitho, M., and Hoppe, H. (2006, January 26\u201328). Poisson surface reconstruction. Proceedings of the Fourth Eurographics Symposium on Geometry Processing (SGP 2006), Cagliari, Italy."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1111\/cgf.14077","article-title":"Poisson Surface Reconstruction with Envelope Constraints","volume":"39","author":"Kazhdan","year":"2020","journal-title":"Comput. Graph. Forum"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1145\/37402.37422","article-title":"Marching cubes: A high resolution 3D surface construction algorithm","volume":"21","author":"Lorensen","year":"1987","journal-title":"ACM Siggraph Comput. Graph."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2487228.2487237","article-title":"Screened poisson surface reconstruction","volume":"32","author":"Kazhdan","year":"2013","journal-title":"ACM Trans. Graph."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1109\/TVCG.2004.1272725","article-title":"Fuzzy vector median-based surface smoothing","volume":"10","author":"Shen","year":"2004","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1145\/1122501.1122504","article-title":"A Bayesian method for probable surface reconstruction and decimation","volume":"25","author":"Diebel","year":"2006","journal-title":"ACM Trans. Graph."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1521","DOI":"10.1109\/TVCG.2010.264","article-title":"Bilateral normal filtering for mesh denoising","volume":"17","author":"Zheng","year":"2010","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.cagd.2017.02.011","article-title":"Robust mesh denoising via vertex pre-filtering and L1-median normal filtering","volume":"54","author":"Lu","year":"2017","journal-title":"Comput. Aided Geom. Des."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1111\/cgf.13386","article-title":"State of the Art on 3D Reconstruction with RGB-D Cameras","volume":"37","author":"Stotko","year":"2018","journal-title":"Comput. Graph. Forum"},{"key":"ref_30","first-page":"9609","article-title":"Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer","volume":"32","author":"Chen","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1007\/s11042-020-09722-8","article-title":"Single image 3D object reconstruction based on deep learning: A review","volume":"80","author":"Fu","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_32","unstructured":"Nocedal, J., and Wright, S. (2006). Numerical Optimization, Springer Science & Business Media."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1117\/12.287750","article-title":"White light heterodyne principle for 3D-measurement","volume":"Volume 3100","author":"Loffeld","year":"1997","journal-title":"Sensors, Sensor Systems, and Sensor Data Processing"},{"key":"ref_34","unstructured":"Zhou, Q. (2020, December 21). PyMesh\u2014Geometry Processing Library for Python. Available online: https:\/\/github.com\/PyMesh\/PyMesh."},{"key":"ref_35","unstructured":"Adobe Systems Incorporated (2019). Digital Negative (DNG) Specification, Adobe Inc.. Version 1.5.0.0."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Xu, J., Ma, Y., He, S., and Zhu, J. (2019). 3D-GIoU: 3D Generalized Intersection over Union for Object Detection in Point Cloud. Sensors, 19.","DOI":"10.3390\/s19194093"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Jack, D., Pontes, J.K., Sridharan, S., Fookes, C., Shirazi, S., Maire, F., and Eriksson, A. (2018, January 2\u20136). Learning free-form deformations for 3D object reconstruction. Proceedings of the Asian Conference on Computer Vision (ACCV 2018), Perth, Australia.","DOI":"10.1007\/978-3-030-20890-5_21"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Varol, G., Ceylan, D., Russell, B., Yang, J., Yumer, E., Laptev, I., and Schmid, C. (2018, January 8\u201314). BodyNet: Volumetric inference of 3D human body shapes. Proceedings of the European Conference on Computer Vision (ECCV 2018), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_2"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Shin, D., Fowlkes, C.C., and Hoiem, D. (2018, January 18\u201322). Pixels, voxels, and views: A study of shape representations for single view 3D object shape prediction. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00323"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kato, H., Ushiku, Y., and Harada, T. (2018, January 18\u201322). Neural 3D mesh renderer. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00411"},{"key":"ref_41","unstructured":"(2020, December 21). Steve. Dandelion Vase. Available online: www.thingiverse.com\/thing:157102."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1111\/j.1467-8659.2009.01388.x","article-title":"Feature preserving point set surfaces based on non-linear kernel regression","volume":"28","author":"Guennebaud","year":"2009","journal-title":"Comput. Graph. Forum"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1145\/1141911.1141977","article-title":"Fast separation of direct and global components of a scene using high frequency illumination","volume":"25","author":"Nayar","year":"2006","journal-title":"ACM Trans. Graph."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1068\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:20:09Z","timestamp":1760160009000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1068"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,4]]},"references-count":43,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21041068"],"URL":"https:\/\/doi.org\/10.3390\/s21041068","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,2,4]]}}}