{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T11:33:47Z","timestamp":1792496027921,"version":"4.5.0"},"reference-count":35,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2020,11,5]],"date-time":"2020-11-05T00:00:00Z","timestamp":1604534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001843","name":"Science and Engineering Research Board","doi-asserted-by":"publisher","award":["2016"],"award-info":[{"award-number":["2016"]}],"id":[{"id":"10.13039\/501100001843","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Immediate assessment of structural integrity of important civil infrastructures, like bridges, hospitals, or dams, is of utmost importance after natural disasters. Currently, inspection is performed manually by engineers who look for local damages and their extent on significant locations of the structure to understand its implication on its global stability. However, the whole process is time-consuming and prone to human errors. Due to their size and extent, some regions of civil structures are hard to gain access for manual inspection. In such situations, a vision-based system of Unmanned Aerial Vehicles (UAVs) programmed with Artificial Intelligence algorithms may be an effective alternative to carry out a health assessment of civil infrastructures in a timely manner. This paper proposes a framework of achieving the above-mentioned goal using computer vision and deep learning algorithms for detection of cracks on the concrete surface from its image by carrying out image segmentation of pixels, i.e., classification of pixels in an image of the concrete surface and whether it belongs to cracks or not. The image segmentation or dense pixel level classification is carried out using a deep neural network architecture named U-Net. Further, morphological operations on the segmented images result in dense measurements of crack geometry, like length, width, area, and crack orientation for individual cracks present in the image. The efficacy and robustness of the proposed method as a viable real-life application was validated by carrying out a laboratory experiment of a four-point bending test on an 8-foot-long concrete beam of which the video is recorded using a camera mounted on a UAV-based, as well as a still ground-based, video camera. Detection, quantification, and localization of damage on a civil infrastructure using the proposed framework can directly be used in the prognosis of the structure\u2019s ability to withstand service loads.<\/jats:p>","DOI":"10.3390\/s20216299","type":"journal-article","created":{"date-parts":[[2020,11,5]],"date-time":"2020-11-05T09:04:34Z","timestamp":1604567074000},"page":"6299","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":116,"title":["Vision and Deep Learning-Based Algorithms to Detect and Quantify Cracks on Concrete Surfaces from UAV Videos"],"prefix":"10.3390","volume":"20","author":[{"given":"Sutanu","family":"Bhowmick","sequence":"first","affiliation":[{"name":"Department of Civil and Environmental Engineering, Rice University, 6100 Main Street, Houston, TX 77005, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0088-1656","authenticated-orcid":false,"given":"Satish","family":"Nagarajaiah","sequence":"additional","affiliation":[{"name":"Department of Civil and Environmental Engineering, Rice University, 6100 Main Street, Houston, TX 77005, USA"},{"name":"Department of Mechanical Engineering, Rice University, 6100 Main Street, Houston, TX 77005, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5043-7460","authenticated-orcid":false,"given":"Ashok","family":"Veeraraghavan","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Rice University, 6100 Main Street, Houston, TX 77005, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"04020073","DOI":"10.1061\/(ASCE)ST.1943-541X.0002535","article-title":"Review of Bridge Structural Health Monitoring Aided by Big Data and Artificial Intelligence: From Condition Assessment to Damage Detection","volume":"146","author":"Sun","year":"2020","journal-title":"J. Struct. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.aei.2015.01.008","article-title":"A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure","volume":"29","author":"Koch","year":"2015","journal-title":"Adv. Eng. Inform."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1061\/(ASCE)0887-3801(2003)17:4(255)","article-title":"Analysis of Edge-Detection Techniques for Crack Identification in Bridges","volume":"17","author":"Abudayyeh","year":"2003","journal-title":"J. Comput. Civ. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Yamaguchi, T., and Hashimoto, S. (2006, January 6\u201310). Automated crack detection for concrete surface image using percolation model and edge information. Proceedings of the IECON Proceedings (Industrial Electronics Conference), Paris, France.","DOI":"10.1109\/IECON.2006.348070"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1061\/(ASCE)0887-3801(2006)20:3(210)","article-title":"Improved Image Analysis for Evaluating Concrete Damage","volume":"20","author":"Hutchinson","year":"2006","journal-title":"J. Comput. Civ. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/s00138-011-0394-0","article-title":"An innovative methodology for detection and quantification of cracks through incorporation of depth perception","volume":"24","author":"Jahanshahi","year":"2013","journal-title":"Mach. Vis. Appl."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1111\/mice.12256","article-title":"A texture-Based Video Processing Methodology Using Bayesian Data Fusion for Autonomous Crack Detection on Metallic Surfaces","volume":"32","author":"Chen","year":"2017","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1007\/s40534-020-00221-y","article-title":"Full-scale multi-functional test platform for investigating mechanical performance of track\u2013subgrade systems of high-speed railways","volume":"28","author":"Zhai","year":"2020","journal-title":"Railw. Eng. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"04015144","DOI":"10.1061\/(ASCE)ST.1943-541X.0001334","article-title":"Dynamic imaging: Real-time detection of local structural damage with blind separation of low-rank background and sparse innovation","volume":"142","author":"Yang","year":"2015","journal-title":"J. Struct. Eng."},{"key":"ref_10","first-page":"292","article-title":"Automatic detection and damage quantification of multiple cracks on concrete surface from video","volume":"4","author":"Bhowmick","year":"2020","journal-title":"Int. J. Sustain. Mater. Struct. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_12","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1111\/mice.12263","article-title":"Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks","volume":"32","author":"Cha","year":"2017","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4392","DOI":"10.1109\/TIE.2017.2764844","article-title":"NB-CNN: Deep Learning-based Crack Detection Using Convolutional Neural Network and Na\u00efve Bayes Data Fusion","volume":"65","author":"Chen","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ni, F., Zhang, J., and Chen, Z. (2019). Pixel-level crack delineation in images with convolutional feature fusion. Struct. Control Health Monit., 26.","DOI":"10.1002\/stc.2286"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kim, B., and Cho, S. (2019). Image-based concrete crack assessment using mask and region-based convolutional neural network. Struct. Control Health Monit., 26.","DOI":"10.1002\/stc.2436"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.neucom.2019.01.036","article-title":"DeepCrack: A deep hierarchical feature learning architecture for crack segmentation","volume":"338","author":"Liu","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.autcon.2019.04.005","article-title":"Computer vision-based concrete crack detection using U-net fully convolutional networks","volume":"104","author":"Liu","year":"2019","journal-title":"Autom. Constr."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"874","DOI":"10.1016\/j.autcon.2011.03.004","article-title":"Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation","volume":"20","author":"Zhu","year":"2011","journal-title":"Autom. Constr."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.autcon.2013.06.011","article-title":"Image-based retrieval of concrete crack properties for bridge inspection","volume":"39","author":"Adhikari","year":"2014","journal-title":"Autom. Constr."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201312). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_25","unstructured":"Yang, L., Li, B., Li, W., Liu, Z., Yang, G., and Xiao, J. (2017, January 24\u201328). Deep concrete inspection using unmanned aerial vehicle towards cssc database. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vancouver, BC, Canada."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, L., Yang, F., Zhang, Y.D., and Zhu, Y.J. (2016, January 25\u201328). Road crack detection using deep convolutional neural network. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7533052"},{"key":"ref_27","unstructured":"MATLAB (2018). Version 9.4.0813654 (R2018a), The MathWorks Inc."},{"key":"ref_28","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2020, October 30). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available online: tensorflow.org."},{"key":"ref_29","unstructured":"(2020, October 30). GitHub. Available online: https:\/\/github.com\/fchollet\/keras."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1109\/TSMC.1979.4310076","article-title":"A threshold selection method from gray-level histograms","volume":"9","author":"Otsu","year":"1979","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_32","unstructured":"DJI (2020, November 05). PHANTOM 4 ADVANCED\/ADVANCED+ DJI. Available online: https:\/\/www.dji.com\/cn\/phantom-4-adv."},{"key":"ref_33","unstructured":"Lucas, B.D., and Kanade, T. (1981, January 24\u201328). An Iterative Image Registration Technique with an Application to Stereo Vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence, Vancouver, BC, Canada."},{"key":"ref_34","unstructured":"Tomasi, C., and Kanade, T. (2020, November 05). Detection and Tracking of Point Features. Available online: https:\/\/cecas.clemson.edu\/~stb\/klt\/tomasi-kanade-techreport-1991.pdf."},{"key":"ref_35","unstructured":"Shi, J., and Carlo, T. (1994, January 21\u201323). Good features to track. Proceedings of the Computer Vision and Pattern Recognition, Seattle, WA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6299\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:29:40Z","timestamp":1760178580000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6299"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,5]]},"references-count":35,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["s20216299"],"URL":"https:\/\/doi.org\/10.3390\/s20216299","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,5]]}}}