{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T01:30:11Z","timestamp":1775871011239,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,4,20]],"date-time":"2024-04-20T00:00:00Z","timestamp":1713571200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>This study proposes a deep learning method for pavement defect detection, focusing on identifying potholes and cracks. A dataset comprising 10,828 images is collected, with 8662 allocated for training, 1083 for validation, and 1083 for testing. Vehicle attitude data are categorized based on three-axis acceleration and attitude change, with 6656 (64%) for training, 1664 (16%) for validation, and 2080 (20%) for testing. The Nvidia Jetson Nano serves as the vehicle-embedded system, transmitting IMU-acquired vehicle data and GoPro-captured images over a 5G network to the server. The server recognizes two damage categories, low-risk and high-risk, storing results in MongoDB. Severe damage triggers immediate alerts to maintenance personnel, while less severe issues are recorded for scheduled maintenance. The method selects YOLOv7 among various object detection models for pavement defect detection, achieving a mAP of 93.3%, a recall rate of 87.8%, a precision of 93.2%, and a processing speed of 30\u201340 FPS. Bi-LSTM is then chosen for vehicle vibration data processing, yielding 77% mAP, 94.9% recall rate, and 89.8% precision. Integration of the visual and vibration results, along with vehicle speed and travel distance, results in a final recall rate of 90.2% and precision of 83.7% after field testing.<\/jats:p>","DOI":"10.3390\/info15040239","type":"journal-article","created":{"date-parts":[[2024,4,22]],"date-time":"2024-04-22T06:10:18Z","timestamp":1713766218000},"page":"239","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Deep Learning-Based Road Pavement Inspection by Integrating Visual Information and IMU"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7716-7306","authenticated-orcid":false,"given":"Chen-Chiung","family":"Hsieh","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Tatung University, Taipei 104, Taiwan"}]},{"given":"Han-Wen","family":"Jia","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Tatung University, Taipei 104, Taiwan"}]},{"given":"Wei-Hsin","family":"Huang","sequence":"additional","affiliation":[{"name":"The Graduate Institute of Design Science, Tatung University, Taipei 104, Taiwan"}]},{"given":"Mei-Hua","family":"Hsih","sequence":"additional","affiliation":[{"name":"Department of Product Design, School of Arts and Design, Sanming University, Sanming 365004, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,20]]},"reference":[{"key":"ref_1","unstructured":"(2022, December 24). National Statistics, R.O.C. (Taiwan). Statistics on the Length of Roads in China in the Past Ten Years. 20 September 2022, Available online: https:\/\/statdb.dgbas.gov.tw\/pxweb\/Dialog\/View.asp."},{"key":"ref_2","unstructured":"(2022, December 24). Freeway Bureau, R.O.C. (Taiwan). Million Vehicle Kilometer Statistics. 13 December 2022, Available online: https:\/\/www.freeway.gov.tw\/Publish.aspx?cnid=1656&p=26767."},{"key":"ref_3","unstructured":"Tamkang University (2022, December 24). Pavement Engineering. 13 December 2022. Available online: https:\/\/mail.tku.edu.tw\/yinghaur\/lee\/te\/pdf."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1007\/s11831-016-9194-z","article-title":"Image-Based Techniques for Crack Detection, Classification and Quantification in Asphalt Pavement: A Review","volume":"24","author":"Zakeri","year":"2017","journal-title":"Arch. Computat. Methods Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"126162","DOI":"10.1016\/j.conbuildmat.2021.126162","article-title":"A Critical Review and Comparative Study on Image Segmentation-based Techniques for Pavement Crack Detection","volume":"321","author":"Kheradmandi","year":"2022","journal-title":"Constr. Build. Mater."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1016\/j.ijtst.2023.11.005","article-title":"A State-of-the-Art Survey of Deep Learning Models for Automated Pavement Crack Segmentation","volume":"13","author":"Gong","year":"2024","journal-title":"Int. J. Transp. Sci. Technol."},{"key":"ref_7","unstructured":"Qiu, J.Y. (2020). Research on Detection Based on Improved Mask RCNN Algorithms. [Master\u2019s Thesis, Department of Information and Communication Engineering, Chaoyang University]."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2023, January 17\u201324). YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00721"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"9628","DOI":"10.3390\/s111009628","article-title":"Adaptive Road Crack Detection System by Pavement Classification","volume":"11","author":"Balcones","year":"2011","journal-title":"Sensors"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"17146","DOI":"10.3390\/s140917146","article-title":"Non-Destructive Evaluation of Depth of Surface Cracks Using Ultrasonic Frequency Analysis","volume":"14","author":"Her","year":"2014","journal-title":"Sensors"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"11052","DOI":"10.1109\/TITS.2021.3099023","article-title":"GCN-Based Pavement Crack Detection Using Mobile LiDAR Point Clouds","volume":"23","author":"Feng","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_12","first-page":"578","article-title":"Convolutional Neural Networks based Potholes Detection Using Thermal Imaging","volume":"34","author":"Aparna","year":"2022","journal-title":"J. King Saud Univ.-Comput. Inf. Sci."},{"key":"ref_13","unstructured":"Shenu, P.M., and Soumya, J. (2012). Automated Detection of Dry and Water-Filled Potholes Using Multimodal Sensing System. [Doctoral Dissertation, Indian Institute of Technology Hyderabad]."},{"key":"ref_14","unstructured":"Nienaber, S., Booysen, M., and Kroon, R. (2015, January 6\u20139). Detecting Potholes Using Simple Image Processing Techniques and Real-World Footage. Proceedings of the 34th Annual Southern African Transport Conference, Pretoria, South Africa."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Varadharajan, S., Jose, S., Sharma, K., Wander, L., and Mertz, C. (2014, January 24\u201326). Vision for Road Inspection. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Steamboat Springs, CO, USA.","DOI":"10.1109\/WACV.2014.6836111"},{"key":"ref_16","unstructured":"Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., and Susstrunk, S. (2010). SLIC Superpixels, EPFL. EPFL Technical Report 149300."},{"key":"ref_17","first-page":"4907","article-title":"Support Vector Machines and Linear Regression Coincide with Very High-Dimensional Features","volume":"34","author":"Ardeshir","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_18","unstructured":"Japan International Cooperation Agency (2022, December 24). Pavement Inspection Guideline. Ministry of Transport, 13 December 2016. Available online: https:\/\/openjicareport.jica.go.jp\/pdf\/12286001_01.pdf."},{"key":"ref_19","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_20","unstructured":"Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2004). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Tan, M., Pang, R., and Le, Q.V. (2020, January 13\u201319). EfficientDet: Scalable and Efficient Object Detection. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01079"},{"key":"ref_22","unstructured":"Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., and Nie, W. (2022). YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, H., Sun, F., Gu, J., and Deng, L. (2022). SF-YOLOv5: A Lightweight Small Object Detection Algorithm Based on Improved Feature Fusion Mode. Sensors, 22.","DOI":"10.3390\/s22155817"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., and Sun, J. (2021, January 20\u201325). RepVGG: Making VGG-style ConvNets Great again. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01352"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020). End-to-End Object Detection with Transformers. arXiv.","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"ref_26","first-page":"691","article-title":"You Only Learn One Representation: Unified Network for Multiple Tasks","volume":"39","author":"Wang","year":"2021","journal-title":"J. Inf. Sci. Eng."},{"key":"ref_27","unstructured":"Xu, S., Wang, X., Lv, W., Chang, Q., Cui, C., Deng, K., Wang, G., Dang, Q., Wei, S., and Du, Y. (2022). PP-YOLOE: An Evolved Version of YOLO. arXiv."},{"key":"ref_28","unstructured":"Ge, Z., Liu, S., Wang, F., Li, Z., and Sun, J. (2021). YOLOX: Exceeding YOLO Series in 2021. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, C., Bochkovskiy, A., and Liao, H.M. (2021, January 20\u201325). Scaled-YOLOv4: Scaling Cross Stage Partial Network. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01283"},{"key":"ref_30","first-page":"103335","article-title":"YOLOv5s-M: A Deep Learning Network Model for Road Pavement Damage Detection from Urban Street-View Imagery","volume":"120","author":"Ren","year":"2023","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Wei, Y., and Chen, J. (202, January 21\u201323). Improved YOLOv4-Tiny Lightweight Country Road Pavement Damage Detection Algorithm. Proceedings of the 2022 2nd International Conference on Algorithms, High-Performance Computing and Artificial Intelligence (AHPCAI), Guangzhou, China.","DOI":"10.1109\/AHPCAI57455.2022.10087521"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Fassmeyer, P., Kortmann, F., Drews, P., and Funk, B. (2021, January 27\u201330). Towards a Camera-Based Road Damage Assessment and Detection for Autonomous Vehicles: Applying Scaled-YOLO and CVAE-WGAN. Proceedings of the 2021 IEEE 94th Vehicular Technology Conference (VTC2021-Fall), Norman, OK, USA.","DOI":"10.1109\/VTC2021-Fall52928.2021.9625213"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Jeong, D. (2020, January 10\u201313). Road Damage Detection Using YOLO with Smartphone Images. Proceedings of the 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA.","DOI":"10.1109\/BigData50022.2020.9377847"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Yang, N., Li, Y., and Ma, R. (2022). An Efficient Method for Detecting Asphalt Pavement Cracks and Sealed Cracks Based on a Deep Data-Driven Model. Appl. Sci., 12.","DOI":"10.3390\/app121910089"},{"key":"ref_35","unstructured":"Sherstinsky, A. (2018). Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network. arXiv."},{"key":"ref_36","unstructured":"Staudemeyer, R.C., and Morris, E.R. (2019). Understanding LSTM\u2014A Tutorial into Long Short-Term Memory Recurrent Neural Networks. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2673","DOI":"10.1109\/78.650093","article-title":"Bidirectional Recurrent Neural Networks","volume":"45","author":"Schuster","year":"1997","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_38","unstructured":"Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv."},{"key":"ref_39","unstructured":"(2022, December 24). goprotaiwancsl.com.tw. GoPro. Available online: https:\/\/www.goprotaiwancsl.com.tw\/."},{"key":"ref_40","unstructured":"Newstar (2024, February 29). Newstar AHRS Series Attitude and Heading Chip AH8. Available online: https:\/\/www.facebook.com\/watch\/?v=290400158530629."},{"key":"ref_41","unstructured":"Docs.novatel.com (2024, February 29). GPRMC. 23 July 2023. Available online: https:\/\/docs.novatel.com\/OEM7\/Content\/Logs\/GPRMC.htm."},{"key":"ref_42","unstructured":"Mathworks (2024, February 29). Attitude and Heading Reference System. Available online: https:\/\/www.mathworks.com\/help\/nav\/ref\/ahrs.html."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/4\/239\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:31:29Z","timestamp":1760106689000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/4\/239"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,20]]},"references-count":42,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["info15040239"],"URL":"https:\/\/doi.org\/10.3390\/info15040239","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,20]]}}}