{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T05:39:19Z","timestamp":1768282759597,"version":"3.49.0"},"reference-count":70,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2022,11,16]],"date-time":"2022-11-16T00:00:00Z","timestamp":1668556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Qii.AI Inc.","award":["NFREF-2018-00623"],"award-info":[{"award-number":["NFREF-2018-00623"]}]},{"name":"CGQ Inc.","award":["NFREF-2018-00623"],"award-info":[{"award-number":["NFREF-2018-00623"]}]},{"name":"NSERC Discovery Grant and the Tri-Council New Frontiers in Research Fund","award":["NFREF-2018-00623"],"award-info":[{"award-number":["NFREF-2018-00623"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Damage assessment of concrete structures is necessary to prevent disasters and ensure the safety of infrastructure such as buildings, sidewalks, dams, and bridges. Cracks are among the most prominent damage types in such structures. In this paper, a solution is proposed for identifying and modeling cracks in concrete structures using a stereo camera. First, crack pixels are identified using deep learning-based semantic segmentation networks trained on a custom dataset. Various techniques for improving the accuracy of these networks are implemented and evaluated. Second, modifications are applied to the stereo camera\u2019s calibration model to ensure accurate estimation of the systematic errors and the orientations of the cameras. Finally, two 3D reconstruction methods are proposed, one of which is based on detecting the dominant structural plane surrounding the crack, while the second method focuses on stereo inference. The experiments performed on close-range images of complex and challenging scenes show that structural cracks can be identified with a precision of 96% and recall of 85%. In addition, an accurate 3D replica of cracks can be produced with an accuracy higher than 1 mm, from which the cracks\u2019 size and other geometric features can be deduced.<\/jats:p>","DOI":"10.3390\/rs14225793","type":"journal-article","created":{"date-parts":[[2022,11,17]],"date-time":"2022-11-17T03:27:44Z","timestamp":1668655664000},"page":"5793","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Semantic Segmentation and 3D Reconstruction of Concrete Cracks"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4093-2473","authenticated-orcid":false,"given":"Parnia","family":"Shokri","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, University of Calgary, Calgary, AB T2N 1N4, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7371-5635","authenticated-orcid":false,"given":"Mozhdeh","family":"Shahbazi","sequence":"additional","affiliation":[{"name":"Department of Geomatics Engineering, University of Calgary, Calgary, AB T2N 1N4, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8518-3448","authenticated-orcid":false,"given":"John","family":"Nielsen","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, University of Calgary, Calgary, AB T2N 1N4, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,16]]},"reference":[{"key":"ref_1","first-page":"79","article-title":"Effects of the December 26, 2004 Sumatra earthquake and tsunami on physical infrastructure","volume":"42","author":"Saatcioglu","year":"2005","journal-title":"ISET J. Earthq. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1139\/l03-045","article-title":"Cost optimization of concrete bridge infrastructure","volume":"30","author":"Hassanain","year":"2003","journal-title":"Can. J. Civ. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1016\/j.autcon.2006.05.003","article-title":"Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel","volume":"16","author":"Yu","year":"2007","journal-title":"Autom. Constr."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1016\/j.autcon.2009.04.003","article-title":"Bridge inspection robot system with machine vision","volume":"18","author":"Oh","year":"2009","journal-title":"Autom. Constr."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.autcon.2015.02.003","article-title":"Past, present and future of robotic tunnel inspection","volume":"59","author":"Montero","year":"2015","journal-title":"Autom. Constr."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1007\/s42405-018-0120-5","article-title":"Robust Concrete Crack Detection Using Deep Learning-Based Semantic Segmentation","volume":"20","author":"Lee","year":"2019","journal-title":"Int. J. Aeronaut. Space Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/s00138-011-0394-0","article-title":"An innovative methodology for detection and quantification of cracks through incorporation of depth perception","volume":"24","author":"Jahanshahi","year":"2013","journal-title":"Mach. Vis. Appl."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kim, B., and Cho, S. (2018). Automated vision-based detection of cracks on concrete surfaces using a deep learning technique. Sensors, 18.","DOI":"10.3390\/s18103452"},{"key":"ref_9","unstructured":"Fan, Z., Wu, Y., Lu, J., and Li, W. (2018). Automatic pavement crack detection based on structured prediction with the convolutional neural network. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"8","DOI":"10.56748\/ejse.15199","article-title":"A Comprehensive Review of Spalling and Fire Performance of Concrete Members","volume":"15","author":"Hedayati","year":"2015","journal-title":"Electron. J. Struct. Eng."},{"key":"ref_11","unstructured":"Greening, N., and Landgren, R. (1966). Surface Discoloration of Concrete Flatwork, Portland Cement Association, Research and Development Laboratories. Number 203."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"035019","DOI":"10.1088\/0964-1726\/22\/3\/035019","article-title":"A new methodology for non-contact accurate crack width measurement through photogrammetry for automated structural safety evaluation","volume":"22","author":"Jahanshahi","year":"2013","journal-title":"Smart Mater. Struct."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"989354","DOI":"10.1155\/2011\/989354","article-title":"Automatic road pavement assessment with image processing: Review and comparison","volume":"2011","author":"Chambon","year":"2011","journal-title":"Int. J. Geophys."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1007\/s12205-015-0461-6","article-title":"A stereovision-based crack width detection approach for concrete surface assessment","volume":"20","author":"Shan","year":"2016","journal-title":"KSCE J. Civ. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3434","DOI":"10.1109\/TITS.2016.2552248","article-title":"Automatic road crack detection using random structured forests","volume":"17","author":"Shi","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"053011","DOI":"10.1117\/1.JEI.27.5.053011","article-title":"Crack detection based on the mesoscale geometric features for visual concrete bridge inspection","volume":"27","author":"Fan","year":"2018","journal-title":"J. Electron. Imaging"},{"key":"ref_17","unstructured":"Hoskere, V., Narazaki, Y., Hoang, T., and Spencer, B. (2018). Vision-based structural inspection using multiscale deep convolutional neural networks. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2369","DOI":"10.1109\/TMI.2016.2546227","article-title":"Segmenting retinal blood vessels with deep neural networks","volume":"35","author":"Liskowski","year":"2016","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1111\/mice.12263","article-title":"Deep learning-based crack damage detection using convolutional neural networks","volume":"32","author":"Cha","year":"2017","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"101182","DOI":"10.1016\/j.aei.2020.101182","article-title":"Survey on performance of deep learning models for detecting road damages using multiple dashcam image resources","volume":"46","author":"Cao","year":"2020","journal-title":"Adv. Eng. Inform."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"114892","DOI":"10.1109\/ACCESS.2020.3003638","article-title":"Automated pavement crack segmentation using u-net-based convolutional neural network","volume":"8","author":"Lau","year":"2020","journal-title":"IEEE Access"},{"key":"ref_24","unstructured":"Lin, F., Yang, J., Shu, J., and Scherer, R.J. (2021). Crack Semantic Segmentation using the U-Net with Full Attention Strategy. arXiv."},{"key":"ref_25","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017). Attention is all you need. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hsiel, Y.A., and Tsai, Y.C.J. (2021, January 19\u201322). Dau-net: Dense attention u-net for pavement crack segmentation. Proceedings of the 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), Indianapolis, IN, USA.","DOI":"10.1109\/ITSC48978.2021.9564806"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Song, C., Wu, L., Chen, Z., Zhou, H., Lin, P., Cheng, S., and Wu, Z. (2019). Pixel-level crack detection in images using SegNet. Multi-Disciplinary Trends in Artificial Intelligence. MIWAI 2019, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-030-33709-4_22"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A Computational Approach to Edge Detection","volume":"PAMI-8","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1109\/4.996","article-title":"Design of an image edge detection filter using the Sobel operator","volume":"23","author":"Kanopoulos","year":"1988","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_30","first-page":"100144","article-title":"Pavement crack detection and recognition using the architecture of segNet","volume":"18","author":"Chen","year":"2020","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"8016","DOI":"10.1109\/TIE.2019.2945265","article-title":"SDDNet: Real-time crack segmentation","volume":"67","author":"Choi","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_32","unstructured":"Ozgenel, C.F. (2019). Concrete Crack Segmentation Dataset. Mendeley Data."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Taylor, L., and Nitschke, G. (2018, January 18\u201321). Improving deep learning with generic data augmentation. Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence (SSCI), Bangalore, India.","DOI":"10.1109\/SSCI.2018.8628742"},{"key":"ref_34","unstructured":"Simard, P.Y., Steinkraus, D., and Platt, J.C. (2003, January 6). Best practices for convolutional neural networks applied to visual document analysis. Proceedings of the Seventh International Conference on Document Analysis and Recognition, Edinburgh, UK."},{"key":"ref_35","unstructured":"Bowles, C., Chen, L., Guerrero, R., Bentley, P., Gunn, R., Hammers, A., Dickie, D.A., Hern\u00e1ndez, M.V., Wardlaw, J., and Rueckert, D. (2018). GAN augmentation: Augmenting training data using generative adversarial networks. arXiv."},{"key":"ref_36","unstructured":"Neff, T., Payer, C., Stern, D., and Urschler, M. (2017, January 10\u201312). Generative adversarial network based synthesis for supervised medical image segmentation. Proceedings of the OAGM&ARW Joint Workshop 2017, Vienna, Austria."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"103118","DOI":"10.1016\/j.autcon.2020.103118","article-title":"Image segmentation of underfloor scenes using a mask regions convolutional neural network with two-stage transfer learning","volume":"113","author":"Atkinson","year":"2020","journal-title":"Autom. Constr."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Stan, S., and Rostami, M. (2021, January 2\u20139). Unsupervised model adaptation for continual semantic segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.","DOI":"10.1609\/aaai.v35i3.16362"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Huang, J., Lu, S., Guan, D., and Zhang, X. (2020). Contextual-relation consistent domain adaptation for semantic segmentation. Computer Vision\u2014ECCV 2020. ECCV 2020, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-030-58555-6_42"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Liu, Y., Zhang, W., and Wang, J. (2021, January 19\u201325). Source-free domain adaptation for semantic segmentation. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00127"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/j.neunet.2018.07.011","article-title":"A systematic study of the class imbalance problem in convolutional neural networks","volume":"106","author":"Buda","year":"2018","journal-title":"Neural Netw."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1260\/1369-4332.17.3.303","article-title":"Achievements and challenges in machine vision-based inspection of large concrete structures","volume":"17","author":"Koch","year":"2014","journal-title":"Adv. Struct. Eng."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Kerle, N., Nex, F., Gerke, M., Duarte, D., and Vetrivel, A. (2020). UAV-Based Structural Damage Mapping: A Review. ISPRS Int. J. Geo-Inf., 9.","DOI":"10.3390\/ijgi9010014"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Kim, H., Lee, J., Ahn, E., Cho, S., Shin, M., and Sim, S.H. (2017). Concrete crack identification using a UAV incorporating hybrid image processing. Sensors, 17.","DOI":"10.3390\/s17092052"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"04014120","DOI":"10.1061\/(ASCE)CP.1943-5487.0000454","article-title":"Multistep explicit stereo camera calibration approach to improve euclidean accuracy of large-scale 3D reconstruction","volume":"30","author":"Fathi","year":"2016","journal-title":"J. Comput. Civ. Eng."},{"key":"ref_46","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks. Communications of the ACM, Association for Computing Machinery."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., and Fergus, R. (2014). Visualizing and understanding convolutional networks. Computer Vision\u2014ECCV 2014, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Noh, H., Hong, S., and Han, B. (2015, January 7\u201313). Learning deconvolution network for semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.178"},{"key":"ref_49","unstructured":"Wu, R., Yan, S., Shan, Y., Dang, Q., and Sun, G. (2015). Deep image: Scaling up image recognition. arXiv."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Liu, Y., Ren, Q., Geng, J., Ding, M., and Li, J. (2018). Efficient patch-wise semantic segmentation for large-scale remote sensing images. Sensors, 18.","DOI":"10.3390\/s18103232"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.autcon.2018.11.028","article-title":"Autonomous concrete crack detection using deep fully convolutional neural network","volume":"99","author":"Dung","year":"2019","journal-title":"Autom. Constr."},{"key":"ref_52","unstructured":"Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv."},{"key":"ref_53","unstructured":"Karras, T., Aila, T., Laine, S., and Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv."},{"key":"ref_54","unstructured":"Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., and Kainz, B. (2018). Attention u-net: Learning where to look for the pancreas. arXiv."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"639930","DOI":"10.3389\/fgene.2021.639930","article-title":"MSU-net: Multi-scale U-net for 2D medical image segmentation","volume":"12","author":"Su","year":"2021","journal-title":"Front. Genet."},{"key":"ref_56","unstructured":"Caruana, R. (1995). Learning Many Related Tasks at the Same Time with Backpropagation. NIPS\u201994: Proceedings of the 7th International Conference on Neural Information Processing Systems, Denver, CO, USA, 1 January 1994, MIT Press."},{"key":"ref_57","unstructured":"Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014). How transferable are features in deep neural networks?. arXiv."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Rahman, M.A., and Wang, Y. (2016). Optimizing intersection-over-union in deep neural networks for image segmentation. ISVC 2016: Advances in Visual Computing, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-50835-1_22"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Van Beers, F. (2018). Using Intersection over Union Loss to Improve Binary Image Segmentation. [Bachelor\u2019s Thesis, University of Groningen].","DOI":"10.5220\/0007347504380445"},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Wilcoxon, F. (1992). Individual comparisons by ranking methods. Breakthroughs in Statistics, Springer.","DOI":"10.1007\/978-1-4612-4380-9_16"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"1167","DOI":"10.5194\/isprs-archives-XLIII-B2-2020-1167-2020","article-title":"Vision-Based Approaches for Quantifying Cracks in Concrete Structures","volume":"43","author":"Shokri","year":"2020","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Luhmann, T., Robson, S., Kyle, S., and Boehm, J. (2019). Close-Range Photogrammetry and 3D Imaging, De Gruyter.","DOI":"10.1515\/9783110607253"},{"key":"ref_64","first-page":"126","article-title":"Robust structure-from-motion computation: Application to open-pit mine surveying from unmanned aerial images","volume":"5","author":"Shahbazi","year":"2017","journal-title":"J. Unmanned Veh. Syst."},{"key":"ref_65","unstructured":"(2022, February 19). OpenCV Camera Calibration. Available online: https:\/\/docs.opencv.org\/3.4\/d4\/d94\/tutorial_camera_calibration.html."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1109\/TMI.2014.2362993","article-title":"Rigorous geometric self-calibrating bundle adjustment for a dual fluoroscopic imaging system","volume":"34","author":"Lichti","year":"2014","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_67","unstructured":"Harris, C., and Stephens, M. (September, January 31). A Combined Corner and Edge Detector. Proceedings of the Alvey Vision Conference, Manchester, UK."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"Speeded-up robust features (SURF)","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1007\/s001380050120","article-title":"A compact algorithm for rectification of stereo pairs","volume":"12","author":"Fusiello","year":"2000","journal-title":"Mach. Vis. Appl."},{"key":"ref_70","unstructured":"Nielsen, C., and Okoniewski, M. (2019, January 15\u201320). GAN Data Augmentation Through Active Learning Inspired Sample Acquisition. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/22\/5793\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:19:42Z","timestamp":1760145582000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/22\/5793"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,16]]},"references-count":70,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["rs14225793"],"URL":"https:\/\/doi.org\/10.3390\/rs14225793","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,16]]}}}