{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T03:07:22Z","timestamp":1776308842367,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T00:00:00Z","timestamp":1616025600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Science Foundation of Shandong Province of China","award":["ZR2018ZB0852"],"award-info":[{"award-number":["ZR2018ZB0852"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents a multi-spectral photometric stereo (MPS) method based on image in-painting, which can reconstruct the shape using a multi-spectral image with a laser line. One of the difficulties in multi-spectral photometric stereo is to extract the laser line because the required illumination for MPS, e.g., red, green, and blue light, may pollute the laser color. Unlike previous methods, through the improvement of the network proposed by Isola, a Generative Adversarial Network based on image in-painting was proposed, to separate a multi-spectral image with a laser line into a clean laser image and an uncorrupted multi-spectral image without the laser line. Then these results were substituted into the method proposed by Fan to obtain high-precision 3D reconstruction results. To make the proposed method applicable to real-world objects, a rendered image dataset obtained using the rendering models in ShapeNet has been used for training the network. Evaluation using the rendered images and real-world images shows the superiority of the proposed approach over several previous methods.<\/jats:p>","DOI":"10.3390\/s21062131","type":"journal-article","created":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T22:19:36Z","timestamp":1616105976000},"page":"2131","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Three-Dimensional Reconstruction with a Laser Line Based on Image In-Painting and Multi-Spectral Photometric Stereo"],"prefix":"10.3390","volume":"21","author":[{"given":"Liang","family":"Lu","sequence":"first","affiliation":[{"name":"College of Information Science and Engineering, Ocean University of China, Qingdao 266100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongbao","family":"Zhu","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Ocean University of China, Qingdao 266100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junyu","family":"Dong","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Ocean University of China, Qingdao 266100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yakun","family":"Ju","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Ocean University of China, Qingdao 266100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huiyu","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Informatics, University of Leicester, Leicester LE1 7RH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zeng, H., Chen, Y., Zhang, Z., Wang, C., and Li, J. (August, January 28). Reconstruction of 3D Zebra Crossings from Mobile Laser Scanning Point Clouds. Proceedings of the IGARSS 2019\u20142019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan.","DOI":"10.1109\/IGARSS.2019.8899336"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3117","DOI":"10.1109\/JSTARS.2019.2918937","article-title":"Automatic 3-D Reconstruction of Indoor Environment with Mobile Laser Scanning Point Clouds","volume":"12","author":"Cui","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Liu, T., Wang, N., Fu, Q., Zhang, Y., and Wang, M. (2019, January 4\u20137). Research on 3D Reconstruction Method Based on Laser Rotation Scanning. Proceedings of the 2019 IEEE International Conference on Mechatronics and Automation (ICMA), Tianjin, China.","DOI":"10.1109\/ICMA.2019.8816561"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1117\/12.7972479","article-title":"Photometric method for determining surface orientation from multiple images","volume":"19","author":"Woodham","year":"1980","journal-title":"Opt. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"878","DOI":"10.1109\/TPAMI.2007.70752","article-title":"Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors","volume":"30","author":"Torresani","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, X., Wu, Q., and Wang, S. (2018, January 8\u201311). Research on 3D Reconstruction Based on Multiple Views. Proceedings of the 2018 13th International Conference on Computer Science & Education (ICCSE), Colombo, Sri Lanka.","DOI":"10.1109\/ICCSE.2018.8468705"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lu, L., Qi, L., Luo, Y., Jiao, H., and Dong, J. (2018). Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo. Sensors, 18.","DOI":"10.3390\/s18030764"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"30804","DOI":"10.1109\/ACCESS.2018.2840138","article-title":"Demultiplexing Colored Images for Multispectral Photometric Stereo via Deep Neural Networks","volume":"6","author":"Ju","year":"2018","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"0031","DOI":"10.1016\/j.patcog.2019.107162","article-title":"A dual-cue network for multispectral photometric stereo","volume":"100","author":"Ju","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.neucom.2019.09.084","article-title":"MPS-Net: Learning to recover surface normal for multispectral photometric stereo","volume":"375","author":"Ju","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1007\/BF01427152","article-title":"Three-dimensional shape from color photometric stereo","volume":"13","author":"Christensen","year":"1994","journal-title":"Int. J. Comput. Vis."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Ikeda, O., and Duan, Y. (2008, January 7\u20139). Color Photometric Stereo for Albedo and Shape Reconstruction. Proceedings of the 2008 IEEE Workshop on Applications of Computer Vision, Copper Mountain, CO, USA.","DOI":"10.1109\/WACV.2008.4544015"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ikeda, O. (2009, January 7\u20138). Color photometric stereo for directional diffuse object. Proceedings of the 2009 Workshop on Applications of Computer Vision (WACV), Snowbird, UT, USA.","DOI":"10.1109\/WACV.2009.5403079"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Anderson, R., Stenger, B., and Cipolla, R. (2011, January 6\u201313). Color photometric stereo for multicolored surfaces. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126495"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"103102","DOI":"10.1117\/1.OE.56.10.103102","article-title":"Deviation correction method for close-range photometric stereo with nonuniform illumination","volume":"56","author":"Fan","year":"2017","journal-title":"Opt. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"113101","DOI":"10.1117\/1.OE.56.11.113101","article-title":"Refractive laser triangulation and photometric stereo in underwater environment","volume":"56","author":"Fan","year":"2017","journal-title":"Opt. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-Image Translation with Conditional Adversarial Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_18","unstructured":"(2018, September 20). Available online: https:\/\/www.2grobotics.com\/."},{"key":"ref_19","first-page":"132","article-title":"Laser Line Detection with Sub-Pixel Accuracy","volume":"20","author":"Martens","year":"2014","journal-title":"Elektron. Elektrotechnika"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chmelarova, N., Chmelar, P., Beran, L., and Rejfek, L. (2016, January 19\u201320). Improving precision of laser line detection in 3D range scanning systems. Proceedings of the 2016 26th International Conference Radioelektronika (RADIOELEKTRONIKA), Kosice, Slovakia.","DOI":"10.1109\/RADIOELEK.2016.7477409"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, Y., Zhou, J., Huang, F., and Liu, L. (2017). Sub-Pixel Extraction of Laser Stripe Center Using an Improved Gray-Gravity Method. Sensors, 17.","DOI":"10.3390\/s17040814"},{"key":"ref_22","unstructured":"Ta, H.N., Kim, D., and Lee, S. (2011, January 26\u201329). A novel laser line detection algorithm for robot application. Proceedings of the 2011 11th International Conference on Control, Automation and Systems, Gyeonggi-do, Korea."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Song, W., Hu, X., Fu, J., Zhou, Q., Zhou, T., and Si, P. (2016, January 1\u20133). The method of hybrid-laser image spot extracts based on HSV space SVD for power transmission line detection. Proceedings of the 2016 IEEE International Conference on Information and Automation (ICIA), Ningbo, China.","DOI":"10.1109\/ICInfA.2016.7832031"},{"key":"ref_24","first-page":"1654","article-title":"The Laser Line Detection for Autonomous Mapping Based on Color Segmentation","volume":"7","author":"Chmelar","year":"2013","journal-title":"Int. J. Mech. Mater. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Liu, H., Jiang, B., Xiao, Y., and Yang, C. (November, January 27). Coherent Semantic Attention for Image In-painting. Proceedings of the International Conference on Computer Vision (ICCV 2019), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00427"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hong, X., Xiong, P., Ji, R., and Fan, H. (2019). Deep Fusion Network for Image Completion. arXiv.","DOI":"10.1145\/3343031.3351002"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., and Huang, T.S. (2018, January 18\u201323). Generative Image In-painting with Contextual Attention. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00577"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., and Huang, T. (November, January 27). Free-Form Image In-painting with Gated Convolution. Proceedings of the International Conference on Computer Vision (ICCV 2019), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00457"},{"key":"ref_29","unstructured":"Nazeri, K., Ng, E., Joseph, T., Qureshi, F.Z., and Ebrahimi, M. (2019). EdgeConnect: Generative Image In-painting with Adversarial Edge Learning. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zeng, Y., Lin, Z., Yang, J., Zhang, J., Shechtman, E., and Lu, H. (2020). High-Resolution Image In-painting with Iterative Confidence Feedback and Guided Upsampling. The 16th European Conference on Computer Vision (ECCV 2020). arXiv.","DOI":"10.1007\/978-3-030-58529-7_1"},{"key":"ref_31","first-page":"1","article-title":"Image completion using planar structure guidance","volume":"33","author":"Huang","year":"2014","journal-title":"ACM Trans. Graph."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yang, J., Qi, Z., and Shi, Y. (2020). Learning to Incorporate Structure Knowledge for Image In-painting. The thirty-fourth AAAI Conference on Artifieial Intelligence (AAAI 2020). arXiv.","DOI":"10.1609\/aaai.v34i07.6951"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Liu, H., Jiang, B., Song, Y., Huang, W., and Yang, C. (arXiv, 2020). Rethinking Image In-Painting via a Mutual Encoder-Decoder with Feature Equalizations. The 16th European Conference on Computer Vision (ECCV 2020), arXiv.","DOI":"10.1007\/978-3-030-58536-5_43"},{"key":"ref_34","unstructured":"Ren, J., Xu, L., Yan, Q., and Sun, W. Shepard Convolutional Neural Networks. Proceedings of the 28th International Conference on Neural Information Processing Systems\u2014Volume 1 (NIPS\u201915)."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, J., Wang, N., Zhang, L., Du, B., and Tao, D. (2020, January 16). Recurrent Feature Reasoning for Image In-Painting. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00778"},{"key":"ref_36","unstructured":"Kingma, D.P., and Ba, J. (2015, January 5\u20138). Adam: A method for stochastic optimization. Proceedings of the International Conference Learn, Represent (ICLR), San Diego, CA, USA."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1109\/TIP.2004.833105","article-title":"Region Filling and Object Removal by Exemplar-Based Image Inpainting","volume":"13","author":"Criminisi","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_38","unstructured":"Criminisi, A., Perez, P., and Toyama, K. (2003, January 18\u201320). Object removal by exemplar-based inpainting. Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lu, C., Feng, J., Lin, Z., and Yan, S. (2018, January 13\u201319). Exact Low Tubal Rank Tensor Recovery from Gaussian Measurements. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/347"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2131\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:37:33Z","timestamp":1760161053000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2131"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,18]]},"references-count":39,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["s21062131"],"URL":"https:\/\/doi.org\/10.3390\/s21062131","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,18]]}}}