{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T22:21:41Z","timestamp":1776723701384,"version":"3.51.2"},"reference-count":52,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,4,7]],"date-time":"2021-04-07T00:00:00Z","timestamp":1617753600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"This research is supported by the National Robotics Programme under its Robotics Enabling Capabilities and Technologies (Funding Agency Project No. 192 25 00051), National Robotics Programme under its Robot Domain Specific (Funding Agency Project No. W192","award":["192 25 00051"],"award-info":[{"award-number":["192 25 00051"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The pavement inspection task, which mainly includes crack and garbage detection, is essential and carried out frequently. The human-based or dedicated system approach for inspection can be easily carried out by integrating with the pavement sweeping machines. This work proposes a deep learning-based pavement inspection framework for self-reconfigurable robot named Panthera. Semantic segmentation framework SegNet was adopted to segment the pavement region from other objects. Deep Convolutional Neural Network (DCNN) based object detection is used to detect and localize pavement defects and garbage. Furthermore, Mobile Mapping System (MMS) was adopted for the geotagging of the defects. The proposed system was implemented and tested with the Panthera robot having NVIDIA GPU cards. The experimental results showed that the proposed technique identifies the pavement defects and litters or garbage detection with high accuracy. The experimental results on the crack and garbage detection are presented. It is found that the proposed technique is suitable for deployment in real-time for garbage detection and, eventually, sweeping or cleaning tasks.<\/jats:p>","DOI":"10.3390\/s21082595","type":"journal-article","created":{"date-parts":[[2021,4,7]],"date-time":"2021-04-07T11:31:59Z","timestamp":1617795119000},"page":"2595","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":42,"title":["Deep Learning Based Pavement Inspection Using Self-Reconfigurable Robot"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3243-9814","authenticated-orcid":false,"given":"Balakrishnan","family":"Ramalingam","sequence":"first","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6141-4600","authenticated-orcid":false,"given":"Abdullah Aamir","family":"Hayat","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6504-1530","authenticated-orcid":false,"given":"Mohan Rajesh","family":"Elara","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"given":"Braulio","family":"F\u00e9lix G\u00f3mez","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2997-6667","authenticated-orcid":false,"given":"Lim","family":"Yi","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4010-570X","authenticated-orcid":false,"given":"Thejus","family":"Pathmakumar","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"given":"Madan Mohan","family":"Rayguru","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"given":"Selvasundari","family":"Subramanian","sequence":"additional","affiliation":[{"name":"Layorz Private Limited, Tamil Nadu, Karur 639117, India"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Saidi, K.S., Bock, T., and Georgoulas, C. (2016). Robotics in construction. Springer Handbook of Robotics, Springer.","DOI":"10.1007\/978-3-319-32552-1_57"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.autcon.2016.06.001","article-title":"Toward a framework for robot-inclusive environments","volume":"69","author":"Tan","year":"2016","journal-title":"Autom. Constr."},{"key":"ref_3","unstructured":"Jeon, J., Jung, B., Koo, J.C., Choi, H.R., Moon, H., Pintado, A., and Oh, P. (2017, January 8\u201311). Autonomous robotic street sweeping: Initial attempt for curbside sweeping. Proceedings of the 2017 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Djuric, A., Saidi, R.A., and ElMaraghy, W. (2010, January 22\u201326). Global Kinematic Model generation for n-DOF reconfigurable machinery structure. Proceedings of the 2010 IEEE International Conference on Automation Science and Engineering, Vancouver, BC, Canada.","DOI":"10.1109\/COASE.2010.5584632"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"69816","DOI":"10.1109\/ACCESS.2020.2986838","article-title":"HTetro-infi: A reconfigurable floor cleaning robot with infinite morphologies","volume":"8","author":"Samarakoon","year":"2020","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hayat, A.A., Karthikeyan, P., Vega-Heredia, M., and Elara, M.R. (2019). Modeling and Assessing of Self-Reconfigurable Cleaning Robot hTetro Based on Energy Consumption. Energies, 12.","DOI":"10.3390\/en12214112"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hayat, A.A., Parween, R., Elara, M.R., Parsuraman, K., and Kandasamy, P.S. (2019, January 20\u201324). Panthera: Design of a reconfigurable pavement sweeping robot. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794268"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"159402","DOI":"10.1109\/ACCESS.2019.2950675","article-title":"Reconfigurable Pavement Sweeping Robot and Pedestrian Cohabitant Framework by Vision Techniques","volume":"7","author":"Le","year":"2019","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yi, L., Le, A.V., Ramalingam, B., Hayat, A.A., Elara, M.R., Minh, T.H.Q., G\u00f3mez, B.F., and Wen, L.K. (2021). Locomotion with Pedestrian Aware from Perception Sensor by Pavement Sweeping Reconfigurable Robot. Sensors, 21.","DOI":"10.3390\/s21051745"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"13969","DOI":"10.1109\/ACCESS.2020.2965327","article-title":"A Framework for Taxonomy and Evaluation of Self-Reconfigurable Robotic Systems","volume":"8","author":"Tan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hayat, A.A., Elangovan, K., Rajesh Elara, M., and Teja, M.S. (2019). Tarantula: Design, modeling, and kinematic identification of a quadruped wheeled robot. Appl. Sci., 9.","DOI":"10.3390\/app9010094"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chun, C., and Ryu, S.K. (2019). Road Surface Damage Detection Using Fully Convolutional Neural Networks and Semi-Supervised Learning. Sensors, 19.","DOI":"10.3390\/s19245501"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ramalingam, B., Yin, J., Rajesh Elara, M., Tamilselvam, Y.K., Mohan Rayguru, M., Muthugala, M.A.V.J., and F\u00e9lix G\u00f3mez, B. (2020). A Human Support Robot for the Cleaning and Maintenance of Door Handles Using a Deep-Learning Framework. Sensors, 20.","DOI":"10.3390\/s20123543"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Teng, T.W., Veerajagadheswar, P., Ramalingam, B., Yin, J., Elara Mohan, R., and G\u00f3mez, B.F. (2020). Vision Based Wall Following Framework: A Case Study With HSR Robot for Cleaning Application. Sensors, 20.","DOI":"10.3390\/s20113298"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhao, L., Li, F., Zhang, Y., Xu, X., Xiao, H., and Feng, Y. (2020). A Deep-Learning-based 3D Defect Quantitative Inspection System in CC Products Surface. Sensors, 20.","DOI":"10.3390\/s20040980"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wu, C., Wang, Z., Hu, S., Lepine, J., Na, X., Ainalis, D., and Stettler, M. (2020). An Automated Machine-Learning Approach for Road Pothole Detection Using Smartphone Sensor Data. Sensors, 20.","DOI":"10.3390\/s20195564"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lv, X., Duan, F., Jiang, J.J., Fu, X., and Gan, L. (2020). Deep Active Learning for Surface Defect Detection. Sensors, 20.","DOI":"10.3390\/s20061650"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Li, Y., Li, H., and Wang, H. (2018). Pixel-Wise Crack Detection Using Deep Local Pattern Predictor for Robot Application. Sensors, 18.","DOI":"10.3390\/s18093042"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wang, K., Yan, F., Zou, B., Tang, L., Yuan, Q., and Lv, C. (2019). Occlusion-Free Road Segmentation Leveraging Semantics for Autonomous Vehicles. Sensors, 19.","DOI":"10.3390\/s19214711"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chun, C., Lee, T., Kwon, S., and Ryu, S.K. (2020). Classification and Segmentation of Longitudinal Road Marking Using Convolutional Neural Networks for Dynamic Retroreflection Estimation. Sensors, 20.","DOI":"10.3390\/s20195560"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Balado, J., Mart\u00ednez-S\u00e1nchez, J., Arias, P., and Novo, A. (2019). Road Environment Semantic Segmentation with Deep Learning from MLS Point Cloud Data. Sensors, 19.","DOI":"10.3390\/s19163466"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1117\/1.JEI.24.6.061119","article-title":"Robust crack detection for unmanned aerial vehicles inspection in an a-contrario decision framework","volume":"24","author":"Aldea","year":"2015","journal-title":"J. Electron. Imaging"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2793","DOI":"10.1007\/s10489-018-01396-y","article-title":"Automatic crack detection for tunnel inspection using deep learning and heuristic image post-processing","volume":"49","author":"Protopapadakis","year":"2018","journal-title":"Appl. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Fan, R., Bocus, M.J., Zhu, Y., Jiao, J., Wang, L., Ma, F., Cheng, S., and Liu, M. (2019, January 9\u201312). Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8814000"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2470735","DOI":"10.1155\/2019\/2470735","article-title":"Real-Time Road Crack Mapping Using an Optimized Convolutional Neural Network","volume":"2019","author":"Hosseini","year":"2019","journal-title":"Complexity"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"103018","DOI":"10.1016\/j.autcon.2019.103018","article-title":"Densely connected deep neural network considering connectivity of pixels for automatic crack detection","volume":"110","author":"Mei","year":"2019","journal-title":"Autom. Constr."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"012020","DOI":"10.1088\/1742-6596\/1349\/1\/012020","article-title":"Deep convolution neural network for crack detection on asphalt pavement","volume":"1349","author":"Yusof","year":"2019","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_28","first-page":"100144","article-title":"Pavement crack detection and recognition using the architecture of segNet","volume":"18","author":"Chen","year":"2020","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Mandal, V., Uong, L., and Adu-Gyamfi, Y. (2018, January 10\u201313). Automated Road Crack Detection Using Deep Convolutional Neural Networks. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.","DOI":"10.1109\/BigData.2018.8622327"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1016\/j.autcon.2018.07.008","article-title":"Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network","volume":"94","author":"Nguyen","year":"2018","journal-title":"Autom. Constr."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhihong, C., Hebin, Z., Yanbo, W., Binyan, L., and Yu, L. (2017, January 26\u201328). A vision-based robotic grasping system using deep learning for garbage sorting. Proceedings of the 2017 36th Chinese Control Conference (CCC), Dalian, China.","DOI":"10.23919\/ChiCC.2017.8029147"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Mittal, G., Yagnik, K.B., Garg, M., and Krishnan, N.C. (2016, January 12\u201316). Spotgarbage: smartphone app to detect garbage using deep learning. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971731"},{"key":"ref_33","unstructured":"Thung, G., and Yang, M. (2016). Classification of Trash for Recyclability Status, Stanford University. CS229 Project Report."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1016\/j.neucom.2016.11.023","article-title":"G-MS2F: GoogLeNet based multi-stage feature fusion of deep CNN for scene recognition","volume":"225","author":"Tang","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ramalingam, B., Lakshmanan, A.K., Ilyas, M., Le, A.V., and Elara, M.R. (2018). Cascaded Machine-Learning Technique for Debris Classification in Floor-Cleaning Robot Application. Appl. Sci., 8.","DOI":"10.3390\/app8122649"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Fulton, M., Hong, J., Islam, M.J., and Sattar, J. (2018). Robotic Detection of Marine Litter Using Deep Visual Detection Models. arXiv.","DOI":"10.1109\/ICRA.2019.8793975"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Panboonyuen, T., Jitkajornwanich, K., Lawawirojwong, S., Srestasathiern, P., and Vateekul, P. (2017). Road Segmentation of Remotely-Sensed Images Using Deep Convolutional Neural Networks with Landscape Metrics and Conditional Random Fields. Remote Sens., 9.","DOI":"10.20944\/preprints201706.0012.v2"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Mancini, A., Malinverni, E.S., Frontoni, E., and Zingaretti, P. (2013, January 25\u201328). Road pavement crack automatic detection by MMS images. Proceedings of the 21st Mediterranean Conference on Control and Automation, Chania, Crete.","DOI":"10.1109\/MED.2013.6608934"},{"key":"ref_39","unstructured":"El-Sheimy, N. (2005, January 16\u201321). An Overview of Mobile Mapping Systems. Proceedings of the FIG Working Week 2005 and 8th International Conference on the Global Spatial Data Infrastructure (GSDI-8): From Pharaohs to Geoinformatics, Cairo, Egypt."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Cui, L., Qi, Z., Chen, Z., Meng, F., and Shi, Y. (2015, January 8\u20139). Pavement Distress Detection Using Random Decision Forests. Proceedings of the International Conference on Data Science, Sydney, Australia.","DOI":"10.1007\/978-3-319-24474-7_14"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1525","DOI":"10.1109\/TITS.2019.2910595","article-title":"Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection","volume":"21","author":"Yang","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1127","DOI":"10.1111\/mice.12387","article-title":"Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images","volume":"33","author":"Maeda","year":"2018","journal-title":"Comput. Aided Civ. Infrastruct. Eng."},{"key":"ref_43","unstructured":"Proen\u00e7a, P.F., and Sim\u00f5es, P. (2020). TACO: Trash Annotations in Context for Litter Detection. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"3434","DOI":"10.1109\/TITS.2016.2552248","article-title":"Automatic road crack detection using random structured forests","volume":"17","author":"Shi","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhang, L., Yang, F., Zhang, Y.D., and Zhu, Y.J. (2016, January 25\u201328). Road crack detection using deep convolutional neural network. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7533052"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"e2551","DOI":"10.1002\/stc.2551","article-title":"CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection","volume":"27","author":"Huyan","year":"2020","journal-title":"Struct. Control. Health Monit."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1111\/mice.12440","article-title":"Encoder\u2013decoder network for pixel-level road crack detection in black-box images","volume":"34","author":"Bang","year":"2019","journal-title":"Comput. Aided Civ. Infrastruct. Eng."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1111\/mice.12533","article-title":"Pavement defect detection with fully convolutional network and an uncertainty framework","volume":"35","author":"Tong","year":"2020","journal-title":"Comput. Aided Civ. Infrastruct. Eng."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1177\/0361198120907283","article-title":"Pavement Image Datasets: A New Benchmark Dataset to Classify and Densify Pavement Distresses","volume":"2674","author":"Majidifard","year":"2020","journal-title":"Transp. Res. Rec."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1111\/mice.12297","article-title":"Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep-Learning Network","volume":"32","author":"Zhang","year":"2017","journal-title":"Comput. Aided Civ. Infrastruct. Eng."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1109\/TCE.2018.2859629","article-title":"Deep Learning Based Robot for Automatically Picking Up Garbage on the Grass","volume":"64","author":"Bai","year":"2018","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Valdenegro-Toro, M. (2016, January 18\u201320). Submerged marine debris detection with autonomous underwater vehicles. Proceedings of the 2016 International Conference on Robotics and Automation for Humanitarian Applications (RAHA), Ettimadai, India.","DOI":"10.1109\/RAHA.2016.7931907"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/8\/2595\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:58:49Z","timestamp":1760363929000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/8\/2595"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,7]]},"references-count":52,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["s21082595"],"URL":"https:\/\/doi.org\/10.3390\/s21082595","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,7]]}}}