{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T17:47:59Z","timestamp":1780940879924,"version":"3.54.1"},"reference-count":57,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2020,7,3]],"date-time":"2020-07-03T00:00:00Z","timestamp":1593734400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Single-shot 3D imaging and shape reconstruction has seen a surge of interest due to the ever-increasing evolution in sensing technologies. In this paper, a robust single-shot 3D shape reconstruction technique integrating the structured light technique with the deep convolutional neural networks (CNNs) is proposed. The input of the technique is a single fringe-pattern image, and the output is the corresponding depth map for 3D shape reconstruction. The essential training and validation datasets with high-quality 3D ground-truth labels are prepared by using a multi-frequency fringe projection profilometry technique. Unlike the conventional 3D shape reconstruction methods which involve complex algorithms and intensive computation to determine phase distributions or pixel disparities as well as depth map, the proposed approach uses an end-to-end network architecture to directly carry out the transformation of a 2D image to its corresponding 3D depth map without extra processing. In the approach, three CNN-based models are adopted for comparison. Furthermore, an accurate structured-light-based 3D imaging dataset used in this paper is made publicly available. Experiments have been conducted to demonstrate the validity and robustness of the proposed technique. It is capable of satisfying various 3D shape reconstruction demands in scientific research and engineering applications.<\/jats:p>","DOI":"10.3390\/s20133718","type":"journal-article","created":{"date-parts":[[2020,7,3]],"date-time":"2020-07-03T06:51:20Z","timestamp":1593759080000},"page":"3718","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":137,"title":["Single-Shot 3D Shape Reconstruction Using Structured Light and Deep Convolutional Neural Networks"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5154-0125","authenticated-orcid":false,"given":"Hieu","family":"Nguyen","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, The Catholic University of America, Washington, DC 20064, USA"},{"name":"Neuroimaging Research Branch, National Institute on Drug Abuse, National Institutes of Health, Baltimore, MD 21224, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuzeng","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Jinan University, Jinan 250022, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhaoyang","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, The Catholic University of America, Washington, DC 20064, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.optlaseng.2009.03.012","article-title":"Dynamic 3-D shape measurement method: A review","volume":"48","author":"Su","year":"2010","journal-title":"Opt. Lasers Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1364\/AOP.3.000128","article-title":"Structured-light 3D surface imaging: A tutorial","volume":"2","author":"Geng","year":"2011","journal-title":"Adv Opt. Photonics"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.optlaseng.2018.02.017","article-title":"High-speed 3D shape measurement with structured light methods: A review","volume":"106","author":"Zhang","year":"2018","journal-title":"Opt. Lasers Eng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/j.aei.2018.05.005","article-title":"A review of 3D reconstruction techniques in civil engineering and their applications","volume":"38","author":"Ma","year":"2018","journal-title":"Adv. Eng. Inf."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Br\u00e4uer-Burchardt, C., Heinze, M., Schmidt, I., K\u00fchmstedt, P., and Notni, G. (2016). Underwater 3D Surface Measurement Using Fringe Projection Based Scanning Devices. Sensors, 16.","DOI":"10.3390\/s16010013"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Du, H., Chen, X., Xi, J., Yu, C., and Zhao, B. (2017). Development and Verification of a Novel Robot-Integrated Fringe Projection 3D Scanning System for Large-Scale Metrology. Sensors, 17.","DOI":"10.3390\/s17122886"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liberadzki, P., Adamczyk, M., Witkowski, M., and Sitnik, R. (2018). Structured-Light-Based System for Shape Measurement of the Human Body in Motion. Sensors, 18.","DOI":"10.3390\/s18092827"},{"key":"ref_8","unstructured":"Cheng, X., Liu, X., Li, Z., Zhong, K., Han, L., He, W., Gan, W., Xi, G., Wang, C., and Shi, Y. (2019). Development and Verification of a Novel Robot-Integrated Fringe Projection 3D Scanning System for Large-Scale Metrology. Sensors, 19."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wu, H., Yu, S., and Yu, X. (2020). 3D Measurement of Human Chest and Abdomen Surface Based on 3D Fourier Transform and Time Phase Unwrapping. Sensors, 20.","DOI":"10.3390\/s20041091"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2018","DOI":"10.1016\/j.optlaseng.2018.04.019","article-title":"Phase shifting algorithms for fringe projection profilometry: A review","volume":"109","author":"Zuo","year":"2018","journal-title":"Opt. Lasers Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.optlaseng.2018.03.003","article-title":"Absolute phase retrieval methods for digital fringe projection profilometry: A review","volume":"107","author":"Zhang","year":"2018","journal-title":"Opt. Lasers Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"28549","DOI":"10.1364\/OE.24.028549","article-title":"Accurate and fast 3D surface measurement with temporal-spatial binary encoding structured illumination","volume":"25","author":"Zhu","year":"2016","journal-title":"Opt. Express"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"20324","DOI":"10.1364\/OE.24.020324","article-title":"Structured light field 3D imaging","volume":"24","author":"Cai","year":"2016","journal-title":"Opt. Express"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, X., He, D., Hu, H., and Liu, L. (2019). Fast 3D Surface Measurement with Wrapped Phase and Pseudorandom Image. Sensors, 19.","DOI":"10.3390\/s19194185"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.optlaseng.2016.04.009","article-title":"Lens distortion elimination for improving measurement accuracy of fringe projection profilometry","volume":"86","author":"Li","year":"2016","journal-title":"Opt. Lasers Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5219","DOI":"10.1364\/AO.55.005219","article-title":"Single-shot absolute 3D shape measurement with Fourier transform profilometry","volume":"55","author":"Li","year":"2016","journal-title":"Appl. Opt."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.optlaseng.2017.10.013","article-title":"Micro Fourier Transform Profilometry (\u03bcFTP): 3D shape measurement at 10,000 frames per second","volume":"102","author":"Zuo","year":"2018","journal-title":"Opt. Lasers Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/j.optlaseng.2009.09.001","article-title":"Fringe projection techniques: Whither we are?","volume":"48","author":"Gorthi","year":"2010","journal-title":"Opt. Lasers Eng."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully Convolutional Networks for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Intentional Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","unstructured":"Eigen, D., Puhrsch, C., and Fergus, R. (2014, January 8\u201311). Depth Map Prediction from a Single Image Using a Multi-scale Deep Network. Proceedings of the International Conference on Neural Information Processing Systems (NIPS), Montreal, QC, Canada."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, F., Shen, C., and Lin, G. (2015, January 7\u201312). Deep convolutional neural fields for depth estimation from a single image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299152"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Choy, C.B., Xu, D., Gwak, J., Chen, K., and Savarese, S. (2016, January 8\u201316). 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46484-8_38"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Dou, P., Shah, S., and Kakadiaris, I. (2017, January 21\u201326). End-to-end 3D face reconstruction with deep neural network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.164"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Paschalidou, D., Ulusoy, A., Schmitt, C., Gool, L., and Geiger, A. (2018, January 18\u201323). RayNet: Learning Volumetric 3D Reconstruction With Ray Potentials. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00410"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1016\/j.optlaseng.2019.04.020","article-title":"Micro deep learning profilometry for high-speed 3D surface imaging","volume":"121","author":"Feng","year":"2019","journal-title":"Opt. Lasers Eng."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"025001","DOI":"10.1117\/1.AP.1.2.025001","article-title":"Fringe pattern analysis using deep learning","volume":"1","author":"Feng","year":"2019","journal-title":"Adv. Photonics"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"20175","DOI":"10.1038\/s41598-019-56222-3","article-title":"Temporal phase unwrapping using deep learning","volume":"9","author":"Yin","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"17091","DOI":"10.1364\/OE.27.017091","article-title":"Deep neural networks for single shot structured light profilometry","volume":"27","author":"Jeught","year":"2019","journal-title":"Opt. Express"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3338","DOI":"10.1364\/AO.58.003338","article-title":"Batch denoising of ESPI fringe patterns based on convolutional neural network","volume":"58","author":"Hao","year":"2019","journal-title":"Appl. Opt."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"28929","DOI":"10.1364\/OE.27.028929","article-title":"Label enhanced and patch based deep learning for phase retrieval from single frame fringe pattern in fringe projection 3D measurement","volume":"27","author":"Shi","year":"2019","journal-title":"Opt. Express"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"9405","DOI":"10.1364\/OE.387215","article-title":"Dynamic 3-D measurement based on fringe-to-fringe transformation using deep learning","volume":"28","author":"Yu","year":"2020","journal-title":"Opt. Express"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.optlaseng.2018.08.018","article-title":"Rapid tracking of extrinsic projector parameters in fringe projection using machine learning","volume":"114","author":"Stavroulakis","year":"2019","journal-title":"Opt. Lasers Eng."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"6179","DOI":"10.1109\/TII.2019.2913853","article-title":"Fringe Pattern Improvement and Super-Resolution Using Deep Learning in Digital Holography","volume":"15","author":"Ren","year":"2019","journal-title":"IEEE Trans. Ind."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.optcom.2018.12.058","article-title":"Fringe pattern denoising based on deep learning","volume":"437","author":"Yan","year":"2019","journal-title":"Opt. Commun."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lin, B., Fu, S., Zhang, C., Wang, F., Xie, S., Zhao, Z., and Li, Y. (2019). Optical fringe patterns filtering based on multi-stage convolution neural network. arXiv.","DOI":"10.1016\/j.optlaseng.2019.105853"},{"key":"ref_38","unstructured":"Figueroa, A., and Rivera, M. (2019). Deep neural network for fringe pattern filtering and normalization. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1992","DOI":"10.1364\/OL.35.001992","article-title":"Generic gamma correction for accuracy enhancement in fringe-projection profilometry","volume":"25","author":"Hoang","year":"2010","journal-title":"Opt. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Nguyen, H., Wang, Z., and Quisberth, J. (2015, January 8\u201311). Accuracy Comparison of Fringe Projection Technique and 3D Digital Image Correlation Technique. Proceedings of the Conference Proceedings of the Society for Experimental Mechanics Series (SEM), Costa Mesa, CA, USA.","DOI":"10.1007\/978-3-319-22446-6_25"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"A9","DOI":"10.1364\/AO.54.0000A9","article-title":"Real-time, high-accuracy 3D imaging and shape measurement","volume":"54","author":"Nguyen","year":"2015","journal-title":"Appl. Opt."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"7100","DOI":"10.1364\/AO.58.007100","article-title":"Real-time 3D shape measurement using 3LCD projection and deep machine learning","volume":"58","author":"Nguyen","year":"2019","journal-title":"Appl. Opt."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"056009","DOI":"10.1117\/1.JBO.23.5.056009","article-title":"Demonstration of a laparoscopic structured-illumination three-dimensional imaging system for guiding reconstructive bowel anastomosis","volume":"23","author":"Le","year":"2018","journal-title":"J. Biomed. Opt."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1016\/j.optlaseng.2009.06.005","article-title":"Some practical considerations in fringe projection profilometry","volume":"48","author":"Wang","year":"2010","journal-title":"Opt. Lasers Eng."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"2438","DOI":"10.1364\/OL.32.002438","article-title":"Three-dimensional shape measurement with an arbitrarily arranged fringe projection profilometry system","volume":"32","author":"Du","year":"2007","journal-title":"Opt. Lett."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"3192","DOI":"10.1364\/OL.35.003192","article-title":"Flexible calibration technique for fringe-projection-based three-dimensional imaging","volume":"35","author":"Vo","year":"2010","journal-title":"Opt. Lett."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"16926","DOI":"10.1364\/OE.20.016926","article-title":"Hyper-accurate flexible calibration technique for fringe-projection-based three-dimensional imaging","volume":"20","author":"Vo","year":"2012","journal-title":"Opt. Express"},{"key":"ref_48","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, The MIT Press."},{"key":"ref_49","unstructured":"(2020, June 22). Single-Shot 3D Shape Reconstruction Data Sets. Available online: https:\/\/figshare.com\/articles\/Single-Shot_Fringe_Projection_Dataset\/7636697."},{"key":"ref_50","unstructured":"Kingma, D., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.optlaseng.2014.04.002","article-title":"Digital image correlation in experimental mechanics and image registration in computer vision: Similarities, differences and complements","volume":"65","author":"Wang","year":"2015","journal-title":"Opt. Lasers Eng."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"9030","DOI":"10.1364\/AO.56.009030","article-title":"3D shape, deformation, and vibration measurements using infrared Kinect sensors and digital image correlation","volume":"56","author":"Nguyen","year":"2017","journal-title":"Appl. Opt."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"2188","DOI":"10.1364\/AO.57.002188","article-title":"Three-dimensional facial digitization using advanced digital image correlation","volume":"57","author":"Nguyen","year":"2018","journal-title":"Appl. Opt."},{"key":"ref_54","unstructured":"(2020, June 22). Amazon Web Services. Available online: https:\/\/aws.amazon.com."},{"key":"ref_55","unstructured":"(2020, June 22). Google Cloud: Cloud Computing Services. Available online: https:\/\/cloud.google.com."},{"key":"ref_56","unstructured":"(2020, June 22). Microsoft Azure: Cloud Computing Services. Available online: https:\/\/azure.microsoft.com\/en-us."},{"key":"ref_57","unstructured":"(2020, June 22). IBM Cloud. Available online: https:\/\/www.ibm.com\/cloud."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/13\/3718\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:46:41Z","timestamp":1760176001000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/13\/3718"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,3]]},"references-count":57,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2020,7]]}},"alternative-id":["s20133718"],"URL":"https:\/\/doi.org\/10.3390\/s20133718","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,3]]}}}