{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T16:32:13Z","timestamp":1791304333607,"version":"4.3.1"},"reference-count":47,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,27]],"date-time":"2021-12-27T00:00:00Z","timestamp":1640563200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Automatic damage detection using deep learning warrants an extensive data source that captures complex pavement conditions. This paper proposes a thermal-RGB fusion image-based pavement damage detection model, wherein the fused RGB-thermal image is formed through multi-source sensor information to achieve fast and accurate defect detection including complex pavement conditions. The proposed method uses pre-trained EfficientNet B4 as the backbone architecture and generates an argument dataset (containing non-uniform illumination, camera noise, and scales of thermal images too) to achieve high pavement damage detection accuracy. This paper tests separately the performance of different input data (RGB, thermal, MSX, and fused image) to test the influence of input data and network on the detection results. The results proved that the fused image\u2019s damage detection accuracy can be as high as 98.34% and by using the dataset after augmentation, the detection model deems to be more stable to achieve 98.35% precision, 98.34% recall, and 98.34% F1-score.<\/jats:p>","DOI":"10.3390\/rs14010106","type":"journal-article","created":{"date-parts":[[2021,12,28]],"date-time":"2021-12-28T01:20:43Z","timestamp":1640654443000},"page":"106","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":84,"title":["Deep Learning-Based Thermal Image Analysis for Pavement Defect Detection and Classification Considering Complex Pavement Conditions"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1989-8602","authenticated-orcid":false,"given":"Cheng","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Civil Engineering, Xi\u2019an Jiaotong Liverpool University, Suzhou 215123, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sindhu","family":"Chandra","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering and Industrial Design, University of Liverpool, Liverpool L69 3BX, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yufan","family":"Han","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Xi\u2019an Jiaotong Liverpool University, Suzhou 215123, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7002-2908","authenticated-orcid":false,"given":"Hyungjoon","family":"Seo","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering and Industrial Design, University of Liverpool, Liverpool L69 3BX, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.ijfatigue.2017.11.013","article-title":"Investigation of fatigue and thermal cracking behavior of rejuvenated reclaimed asphalt pavement binders and mixtures","volume":"108","author":"Elkashef","year":"2018","journal-title":"Int. J. Fatigue"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1832","DOI":"10.1080\/14680629.2017.1350598","article-title":"Predicting thermal cracking of asphalt pavements from bitumen and mix properties","volume":"19","author":"Teltayev","year":"2018","journal-title":"Road Mater. Pavement Des."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1080\/10298436.2016.1155709","article-title":"An Optimum Selection Strategy of Reflective Cracking Mitigation Methods for an Asphalt Concrete Overlay over Flexible Pavements","volume":"19","author":"Golestani","year":"2018","journal-title":"Int. J. Pavement Eng. Tatari"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1080\/1029843042000198568","article-title":"State of the Art Report on Ageing Test Methods for Bituminous Pavement Materials","volume":"4","author":"Dan","year":"2003","journal-title":"Int. J. Pavement Eng. Tatari"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhang, C., Nateghinia, E., Miranda-Moreno, L.F., and Sun, L. (Int. J. Transp. Sci. Technol., 2021). Pavement distress detection using convolutional neural network (CNN): A case study in Montreal, Canada, Int. J. Transp. Sci. Technol., in press.","DOI":"10.1016\/j.ijtst.2021.04.008"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, C., Seo, H., and Zhao, Y. (2021). A novel pavement transverse cracks detection model using WT-CNN and STFT-CNN for smartphone data analysis. Int. J. Pavement Eng.","DOI":"10.1080\/10298436.2021.1945056"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"103055","DOI":"10.1016\/j.autcon.2019.103055","article-title":"Heating process monitoring and evaluation of hot in-place recycling of asphalt pavement using infrared thermal imaging","volume":"111","author":"Han","year":"2020","journal-title":"Autom. Constr."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"103788","DOI":"10.1016\/j.autcon.2021.103788","article-title":"Automated pixel-level pavement distress detection based on stereo vision and deep learning","volume":"129","author":"Guan","year":"2021","journal-title":"Autom. Constr."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhong, M., Sui, L., Wang, Z., and Hu, D. (2020). Pavement Crack Detection from Mobile Laser Scanning Point Clouds Using a Time Grid. Sensors, 20.","DOI":"10.3390\/s20154198"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Chen, C., Seo, H., and Zhao, Y. (2021). Pavement crack detection and classification based on fusion feature of LBP and PCA with SVM. Int. J. Pavement Eng., 1\u201310.","DOI":"10.1080\/10298436.2021.1888092"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1291","DOI":"10.1111\/mice.12622","article-title":"Automated pavement crack detection and segmentation based on two-step convolutional neural network","volume":"35","author":"Liu","year":"2020","journal-title":"Comput. Civ. Infrastruct. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.patrec.2011.11.004","article-title":"CrackTree: Automatic crack detection from pavement images","volume":"33","author":"Zou","year":"2012","journal-title":"Pattern Recognit. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"398","DOI":"10.4103\/0256-4602.62225","article-title":"Automatic Pavement Crack Detection Using Texture and Shape Descriptors","volume":"27","author":"Hu","year":"2010","journal-title":"IETE Tech. Rev."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"120080","DOI":"10.1016\/j.conbuildmat.2020.120080","article-title":"A triple-thresholds pavement crack detection method leveraging random structured forest","volume":"263","author":"Peng","year":"2020","journal-title":"Constr. Build. Mater."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"103833","DOI":"10.1016\/j.autcon.2021.103833","article-title":"Road Surface Damage Detection Based on Hierarchical Architecture Using Lightweight Auto-Encoder Network","volume":"130","author":"Seungbo","year":"2021","journal-title":"Autom. Constr."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103291","DOI":"10.1016\/j.autcon.2020.103291","article-title":"Hybrid Pixel-Level Concrete Crack Segmentation and Quantification across Complex Backgrounds Using Deep Learning","volume":"118","author":"Dongho","year":"2020","journal-title":"Autom. Constr."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1016\/j.conbuildmat.2017.09.110","article-title":"Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection","volume":"157","author":"Gopalakrishnan","year":"2017","journal-title":"Constr. Build. Mater."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1111\/mice.12263","article-title":"Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks","volume":"32","author":"Cha","year":"2017","journal-title":"Comput. Civ. Infrastruct. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"731","DOI":"10.1111\/mice.12334","article-title":"Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types","volume":"33","author":"Cha","year":"2017","journal-title":"Comput. Civ. Infrastruct. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"04018001","DOI":"10.1061\/(ASCE)CP.1943-5487.0000736","article-title":"Unified Approach to Pavement Crack and Sealed Crack Detection Using Preclassification Based on Transfer Learning","volume":"32","author":"Kaige","year":"2018","journal-title":"J. Comput. Civ. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"118513","DOI":"10.1016\/j.conbuildmat.2020.118513","article-title":"Deep machine learning approach to develop a new asphalt pavement condition index","volume":"247","author":"Majidifard","year":"2020","journal-title":"Constr. Build. Mater."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"103171","DOI":"10.1016\/j.autcon.2020.103171","article-title":"Deep learning-based roadway crack classification using laser-scanned range images: A comparative study on hyperparameter selection","volume":"114","author":"Zhou","year":"2020","journal-title":"Autom. Constr."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"04020143","DOI":"10.1061\/(ASCE)CF.1943-5509.0001557","article-title":"Real-Time Thermal Imaging-Based System for Asphalt Pavement Surface Distress Inspection and 3D Crack Profiling","volume":"35","author":"Golrokh","year":"2021","journal-title":"J. Perform. Constr. Facil."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1520\/GTJ20150245","article-title":"Crack Detection in Pillars Using Infrared Thermographic Imaging","volume":"40","author":"Seo","year":"2017","journal-title":"Geotech. Test. J."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"104118","DOI":"10.1016\/j.tust.2021.104118","article-title":"Infrared thermography for detecting cracks in pillar models with different reinforcing systems","volume":"116","author":"Seo","year":"2021","journal-title":"Tunn. Undergr. Space Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"04020038","DOI":"10.1061\/(ASCE)CP.1943-5487.0000918","article-title":"Machine Learning for Crack Detection: Review and Model Performance Comparison","volume":"34","author":"Yichang","year":"2020","journal-title":"J. Comput. Civ. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Islam, M.M., and Kim, J.-M. (2019). Vision-Based Autonomous Crack Detection of Concrete Structures Using a Fully Convolutional Encoder\u2013Decoder Network. Sensors, 19.","DOI":"10.3390\/s19194251"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"120474","DOI":"10.1016\/j.conbuildmat.2020.120474","article-title":"Comparison of crack segmentation using digital image correlation measurements and deep learning","volume":"261","author":"Rezaie","year":"2020","journal-title":"Constr. Build. Mater."},{"key":"ref_29","first-page":"1498","article-title":"Deepcrack: Learning Hierarchical Convolutional Features for Crack Detection","volume":"28","author":"Qin","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1111\/mice.12564","article-title":"Automatic detection method of cracks from concrete surface imagery using two-step light gradient boosting machine","volume":"36","author":"Chun","year":"2020","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"104391","DOI":"10.1016\/j.engappai.2021.104391","article-title":"Automatic Pixel-Level Crack Segmentation in Images Using Fully Convolutional Neural Network Based on Residual Blocks and Pixel Local Weights","volume":"104","author":"Raza","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Fan, Z., Li, C., Chen, Y., Wei, J., Loprencipe, G., Chen, X., and Di Mascio, P. (2020). Automatic Crack Detection on Road Pavements Using Encoder-Decoder Architecture. Materials, 13.","DOI":"10.3390\/ma13132960"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"103786","DOI":"10.1016\/j.autcon.2021.103786","article-title":"Semi-supervised semantic segmentation network for surface crack detection","volume":"128","author":"Wang","year":"2021","journal-title":"Autom. Constr."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Tang, W., Huang, S., Zhao, Q., Li, R., and Huangfu, L. (2021). An Iteratively Optimized Patch Label Inference Network for Automatic Pavement Distress Detection. IEEE Trans. Intell. Transp. Syst.","DOI":"10.1109\/TITS.2021.3084809"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1016\/j.optlaseng.2019.05.005","article-title":"A deep-learning-based approach for fast and robust steel surface defects classification","volume":"121","author":"Fu","year":"2019","journal-title":"Opt. Lasers Eng."},{"key":"ref_36","unstructured":"Andy, T. (2020, December 11). Thermal Analysis with Matlab and Flir Cameras. Available online: https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/49747-thermal-analysis-with-matlab-and-flir-cameras-demo-files."},{"key":"ref_37","first-page":"56","article-title":"A Retinex Image Enhancement Based on L Channel Illumination Estimation and Gamma Function","volume":"37","author":"Huang","year":"2018","journal-title":"Tech. Autom. Appl."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1016\/j.apsusc.2013.09.002","article-title":"A Noise Robust Method Based on Completed Local Binary Patterns for Hot-Rolled Steel Strip Surface Defects","volume":"285","author":"Kechen","year":"2013","journal-title":"Appl. Surf. Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.biosystemseng.2019.01.002","article-title":"The accuracy and utility of a low cost thermal camera and smartphone-based system to assess grapevine water status","volume":"179","author":"Petrie","year":"2019","journal-title":"Biosyst. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"106691","DOI":"10.1016\/j.asoc.2020.106691","article-title":"Automated medical diagnosis of COVID-19 through EfficientNet convolutional neural network","volume":"96","author":"Marques","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"111310","DOI":"10.1016\/j.chaos.2021.111310","article-title":"C+Effxnet: A Novel Hybrid Approach for Covid-19 Diagnosis on Ct Images Based on Cbam and Efficientnet","volume":"151","author":"Murat","year":"2021","journal-title":"Chaos Solitons Fractals"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"105652","DOI":"10.1016\/j.compag.2020.105652","article-title":"Efficientnet-B4-Ranger: A Novel Method for Greenhouse Cucumber Disease Recognition under Natural Complex Environment","volume":"176","author":"Pan","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_43","first-page":"6105","article-title":"Efficientnet: Rethinking Model Scaling for Convolutional Neural Networks","volume":"97","author":"Tan","year":"2019","journal-title":"Pap. Presented Int. Conf. Mach. Learn."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"101182","DOI":"10.1016\/j.ecoinf.2020.101182","article-title":"Plant Leaf Disease Classification Using Efficientnet Deep Learning Model","volume":"61","author":"Akyol","year":"2021","journal-title":"Ecol. Inform."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"109900","DOI":"10.1016\/j.measurement.2021.109900","article-title":"Application of image technology on pavement distress detection: A review","volume":"184","author":"Du","year":"2021","journal-title":"Measurement"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"04015118","DOI":"10.1061\/(ASCE)MT.1943-5533.0001413","article-title":"Use of Infrared Thermography to Detect Thermal Segregation in Asphalt Overlay and Reflective Cracking Potential","volume":"28","author":"Dhakal","year":"2016","journal-title":"J. Mater. Civ. Eng."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"3935","DOI":"10.1109\/TITS.2018.2791476","article-title":"A Kinect-Based Approach for 3D Pavement Surface Reconstruction and Cracking Recognition","volume":"19","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/1\/106\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:54:04Z","timestamp":1760169244000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/1\/106"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,27]]},"references-count":47,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["rs14010106"],"URL":"https:\/\/doi.org\/10.3390\/rs14010106","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,27]]}}}