{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T05:57:09Z","timestamp":1776146229202,"version":"3.50.1"},"reference-count":49,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2020,8,14]],"date-time":"2020-08-14T00:00:00Z","timestamp":1597363200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008369","name":"Sihtasutus Archimedes","doi-asserted-by":"publisher","award":["LEP19022"],"award-info":[{"award-number":["LEP19022"]}],"id":[{"id":"10.13039\/100008369","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>In the manuscript, the issue of detecting and segmenting out pavement defects on highway roads is addressed. Specifically, computer vision (CV) methods are developed and applied to the problem based on deep learning of convolutional neural networks (ConvNets). A novel neural network structure is considered, based on a pipeline of three ConvNets and endowed with the capacity for context awareness, which improves grid-based search for defects on orthoframes by considering the surrounding image content\u2014an approach, which essentially draws inspiration from how humans tend to solve the task of image segmentation. Also, methods for assessing the quality of segmentation are discussed. The contribution also describes the complete procedure of working with pavement defects in an industrial setting, involving the workcycle of defect annotation, ConvNet training and validation. The results of ConvNet evaluation provided in the paper hint at a successful implementation of the proposed technique.<\/jats:p>","DOI":"10.3390\/a13080198","type":"journal-article","created":{"date-parts":[[2020,8,14]],"date-time":"2020-08-14T08:28:35Z","timestamp":1597393715000},"page":"198","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Pavement Defect Segmentation in Orthoframes with a Pipeline of Three Convolutional Neural Networks"],"prefix":"10.3390","volume":"13","author":[{"given":"Roland","family":"L\u00f5uk","sequence":"first","affiliation":[{"name":"Department of Software Science, Tallinn University of Technology, 19086 Tallinn, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0316-7036","authenticated-orcid":false,"given":"Andri","family":"Riid","sequence":"additional","affiliation":[{"name":"Department of Software Science, Tallinn University of Technology, 19086 Tallinn, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6370-0096","authenticated-orcid":false,"given":"Ren\u00e9","family":"Pihlak","sequence":"additional","affiliation":[{"name":"Department of Software Science, Tallinn University of Technology, 19086 Tallinn, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7158-8484","authenticated-orcid":false,"given":"Aleksei","family":"Tepljakov","sequence":"additional","affiliation":[{"name":"Department of Computer Systems, Tallinn University of Technology, 19086 Tallinn, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,8,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ragnoli, A., De Blasiis, M.R., and Di Benedetto, A. (2018). Pavement distress detection methods: A review. Infrastructures, 3.","DOI":"10.20944\/preprints201809.0567.v1"},{"key":"ref_2","unstructured":"Vavrik, W., Evans, L., Sargand, S., and Stefanski, J. (2013, July 01). PCR Evaluation: Considering Transition from Manual to Semi-Automated Pavement Distress Collection and Analysis, Available online: https:\/\/rosap.ntl.bts.gov\/view\/dot\/26795."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"119397","DOI":"10.1016\/j.conbuildmat.2020.119397","article-title":"A cost effective solution for pavement crack inspection using cameras and deep neural networks","volume":"256","author":"Mei","year":"2020","journal-title":"Constr. Build Mater."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3701","DOI":"10.1109\/JSTARS.2018.2865528","article-title":"Detection of Asphalt Pavement Potholes and Cracks Based on the Unmanned Aerial Vehicle Multispectral Imagery","volume":"11","author":"Pan","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1005","DOI":"10.1109\/JSTARS.2018.2791970","article-title":"High-Resolution Velocity Analysis Method Using the l-1 Norm Regularized Least-Squares Method for Pavement Inspection","volume":"11","author":"Yi","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_6","unstructured":"Choi, J., Zhu, L., and Kurosu, H. (2016, July 19). Detection of Cracks in Paved Road Surface Using Laser Scan Image Data. Available online: https:\/\/pdfs.semanticscholar.org\/e591\/fcd67903e8c6210a0ec2151ea336536766f9.pdf."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Bhat, S., Naik, S., Gaonkar, M., Sawant, P., Aswale, S., and Shetgaonkar, P. (2020, January 24\u201325). A Survey On Road Crack Detection Techniques. Proceedings of the International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE), Vellore, India.","DOI":"10.1109\/ic-ETITE47903.2020.67"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"14531","DOI":"10.1109\/ACCESS.2020.2966881","article-title":"Review of pavement defect detection methods","volume":"8","author":"Cao","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Azhar, K., Murtaza, F., Yousaf, M.H., and Habib, H.A. (2016, January 15\u201318). Computer vision based detection and localization of potholes in asphalt pavement images. Proceedings of the Canadian Conference on Electrical and Computer Engineering, Vancouver, BC, Canada.","DOI":"10.1109\/CCECE.2016.7726722"},{"key":"ref_10","unstructured":"Jenkins, M.D., Carr, T.A., Iglesias, M.I., Buggy, T., and Morison, G. (2018, January 3\u20137). A deep convolutional neural network for semantic pixel-wise segmentation of road and pavement surface cracks. Proceedings of the 26th European Signal Processing Conference (EUSIPCO), Rome, Italy."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Nie, M., and Wang, K. (2018, January 10\u201312). Pavement Distress Detection Based on Transfer Learning. Proceedings of the 2018 5th International Conference on Systems and Informatics, Nanjing, China.","DOI":"10.1109\/ICSAI.2018.8599473"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Riid, A., L\u00f5uk, R., Pihlak, R., Tepljakov, A., and Vassiljeva, K. (2019). Pavement Distress Detection with Deep Learning Using the Orthoframes Acquired by a Mobile Mapping System. Appl. Sci., 9.","DOI":"10.3390\/app9224829"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yusof, N., Osman, M., Hussain, Z., Noor, M., Ibrahim, A., Tahir, N., and Abidin, N. (December, January 29). Automated Asphalt Pavement Crack Detection and Classification using Deep Convolution Neural Network. Proceedings of the 2019 9th IEEE International Conference on Control System, Computing and Engineering, Penang, Malaysia.","DOI":"10.1109\/ICCSCE47578.2019.9068551"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e2551","DOI":"10.1002\/stc.2551","article-title":"CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection","volume":"27","author":"Huyan","year":"2020","journal-title":"Struct. Contr. Health Monit."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sun, M., Guo, R., Zhu, J., and Fan, W. (2020, January 6\u20138). Roadway Crack Segmentation Based on an Encoder-decoder Deep Network with Multi-scale Convolutional Blocks. Proceedings of the 10th Annual Computing and Communication Workshop and Conference, Las Vegas, NV, USA.","DOI":"10.1109\/CCWC47524.2020.9031213"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"51446","DOI":"10.1109\/ACCESS.2020.2980086","article-title":"Semi-Supervised Semantic Segmentation Using Adversarial Learning for Pavement Crack Detection","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Akagic, A., Buza, E., and Omanovic, S. (2017, January 22\u201326). Pothole detection: An efficient vision based method using RGB color space image segmentation. Proceedings of the 40th International Convention on Information and Communication Technology, Opatija, Croatia.","DOI":"10.23919\/MIPRO.2017.7973589"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Seichter, D., Eisenbach, M., Stricker, R., and Gross, H.M. (2018, January 20\u201324). How to Improve Deep Learning based Pavement Distress Detection while Minimizing Human Effort. Proceedings of the IEEE International Conference on Automation Science and Engineering, Munich, Germany.","DOI":"10.1109\/COASE.2018.8560372"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Chen, J., Liu, G., and Chen, X. (2019, January 6\u20138). Road crack image segmentation using global context unet. Proceedings of the ACM International Conference Proceeding Series, Beijing, China.","DOI":"10.1145\/3374587.3374602"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1016\/j.conbuildmat.2017.09.110","article-title":"Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection","volume":"157","author":"Gopalakrishnan","year":"2017","journal-title":"Constr. Build Mater."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Tepljakov, A., Riid, A., Pihlak, R., Vassiljeva, K., and Petlenkov, E. (2019, January 1\u20133). Deep Learning for Detection of Pavement Distress using Nonideal Photographic Images. Proceedings of the 42nd International Conference on Telecommunications and Signal Processing, Budapest, Hungary.","DOI":"10.1109\/TSP.2019.8769086"},{"key":"ref_22","unstructured":"(2020, June 30). EyeVi\u2014Mobile Mapping Based Visual Intelligence. Available online: https:\/\/www.eyevi.tech."},{"key":"ref_23","unstructured":"Soppe, T. (2017). Pavement Defect Inventory. [Master\u2019s Thesis, Tallinn University of Applied Sciences]."},{"key":"ref_24","unstructured":"Masters, D., and Luschi, C. (2018). Revisiting Small Batch Training for Deep Neural Networks. arXiv, Available online: https:\/\/arxiv.org\/abs\/1804.07612."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv, Available online: https:\/\/arxiv.org\/abs\/1505.04597.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 12). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"L\u00f5uk, R., Tepljakov, A., and Riid, A. (2020, January 6\u20138). A Two-Stream Context-Aware ConvNet for Pavement Distress Detection. Proceedings of the 43rd International Conference on Telecommunications and Signal Processing (TSP), Milan, Italy. in press.","DOI":"10.1109\/TSP49548.2020.9163538"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kornblith, S., Shlens, J., and Le, Q.V. (2018). Do Better ImageNet Models Transfer Better?. arXiv, Available online: https:\/\/arxiv.org\/abs\/1805.08974.","DOI":"10.1109\/CVPR.2019.00277"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"961","DOI":"10.1007\/s11263-018-1070-x","article-title":"Augmented Reality Meets Computer Vision: Efficient Data Generation for Urban Driving Scenes","volume":"126","author":"Alhaija","year":"2018","journal-title":"Int. J. Comput. Vis"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3434","DOI":"10.1109\/TITS.2016.2552248","article-title":"Automatic road crack detection using random structured forests","volume":"17","author":"Shi","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Augustauskas, R., and Lipnickas, A. (2020). Improved Pixel-Level Pavement-Defect Segmentation Using a Deep Autoencoder. Sensors, 20.","DOI":"10.3390\/s20092557"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The pascal visual object classes (voc) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_34","unstructured":"PyQt (2020, August 13). PyQt Reference Guide. Available online: https:\/\/www.riverbankcomputing.com\/static\/Docs\/PyQt5\/."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Torbert, S. (2016). Applied Computer Science, Springer International Publishing.","DOI":"10.1007\/978-3-319-30866-1"},{"key":"ref_36","unstructured":"Centre for Intelligent Systems (2020, August 13). DATM Annotation Tool GitHub Page. Available online: https:\/\/github.com\/is-centre\/datm-annotation-tool."},{"key":"ref_37","unstructured":"Sener, O., and Savarese, S. (2017). Active learning for convolutional neural networks: A core-set approach. arXiv, Available online: https:\/\/arxiv.org\/abs\/1708.00489."},{"key":"ref_38","unstructured":"Ducoffe, M., and Precioso, F. (2018). Adversarial Active Learning for Deep Networks: A Margin Based Approach. arXiv, Available online: https:\/\/arxiv.org\/abs\/1802.09841."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Rahman, M.A., and Wang, Y. (2016, January 12\u201314). Optimizing intersection-over-union in deep neural networks for image segmentation. Proceedings of the International Symposium on Visual Computing, Las Vegas, NV, USA.","DOI":"10.1007\/978-3-319-50835-1_22"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Van Beers, F., Lindstr\u00f6m, A., Okafor, E., and Wiering, M.A. (2019, January 19\u201321). Deep Neural Networks with Intersection over Union Loss for Binary Image Segmentation. Proceedings of the ICPRAM, Prague, Czech Republic.","DOI":"10.5220\/0007347504380445"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Smith, L., and Topin, N. (2019). Super-convergence: Very fast training of neural networks using large learning rates. arXiv, Available online: https:\/\/doi.org\/10.1117\/12.2520589.","DOI":"10.1117\/12.2520589"},{"key":"ref_42","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019, January 8\u201314). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Proceedings of the In Advances in Neural Information Processing Systems 32, Vancouver, BC, Canada."},{"key":"ref_43","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv, Available online: https:\/\/arxiv.org\/abs\/1412.6980."},{"key":"ref_44","unstructured":"Buslaev, A., Parinov, A., Khvedchenya, E., Iglovikov, V.I., and Kalinin, A.A. (2018). Albumentations: Fast and flexible image augmentations. arXiv, Available online: https:\/\/arxiv.org\/abs\/1809.06839."},{"key":"ref_45","unstructured":"Yakubovskiy, P. (2020, June 30). Segmentation Models Pytorch. Available online: https:\/\/github.com\/qubvel\/segmentation_models.pytorch."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-Excitation Networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4\u20139). Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. Proceedings of the Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Fransisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Nowling, R.J., Bukowy, J., McGarry, S.D., Nencka, A.S., Blasko, O., Urbain, J., Lowman, A., Barrington, A., Banerjee, A., and Iczkowski, K.A. (2019, January 19\u201322). Classification before Segmentation: Improved U-Net Prostate Segmentation. Proceedings of the IEEE EMBS International Conference on Biomedical Health Informatics (BHI), Chicago, IL, USA.","DOI":"10.1109\/BHI.2019.8834494"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/13\/8\/198\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:01:05Z","timestamp":1760176865000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/13\/8\/198"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,14]]},"references-count":49,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,8]]}},"alternative-id":["a13080198"],"URL":"https:\/\/doi.org\/10.3390\/a13080198","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,14]]}}}