{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T09:35:54Z","timestamp":1783330554453,"version":"3.54.6"},"reference-count":40,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2022,12,17]],"date-time":"2022-12-17T00:00:00Z","timestamp":1671235200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Harbor and Marine Technology Center (HMTC), Institute of Transportation (IOT), Ministry of Transportation and Communication (MOTC)","award":["MOTC-IOT-110-H1CB001g"],"award-info":[{"award-number":["MOTC-IOT-110-H1CB001g"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In order to save manpower on rail track inspection, computer vision-based methodologies are developed. We propose utilizing the YOLOv4-Tiny neural network to identify track defects in real time. There are ten defects covering fasteners, rail surfaces, and sleepers from the upward and six defects about the rail waist from the sideward. The proposed real-time inspection system includes a high-performance notebook, two sports cameras, and three parallel processes. The hardware is mounted on a flat cart running at 30 km\/h. The inspection results about the abnormal track components could be queried by defective type, time, and the rail hectometer stake. In the experiments, data augmentation by a Cycle Generative Adversarial Network (GAN) is used to increase the dataset. The number of images is 3800 on the upward and 967 on the sideward. Five object detection neural network models\u2014YOLOv4, YOLOv4-Tiny, YOLOX-Tiny, SSD512, and SSD300\u2014were tested. The YOLOv4-Tiny model with 150 FPS is selected as the recognition kernel, as it achieved 91.7%, 92%, and 91% for the mAP, precision, and recall of the defective track components from the upward, respectively. The mAP, precision, and recall of the defective track components from the sideward are 99.16%, 96%, and 94%, respectively.<\/jats:p>","DOI":"10.3390\/s22249970","type":"journal-article","created":{"date-parts":[[2022,12,19]],"date-time":"2022-12-19T09:31:01Z","timestamp":1671442261000},"page":"9970","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":40,"title":["An Online Rail Track Fastener Classification System Based on YOLO Models"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7716-7306","authenticated-orcid":false,"given":"Chen-Chiung","family":"Hsieh","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Tatung University, Taipei 104, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ti-Yun","family":"Hsu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Tatung University, Taipei 104, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei-Hsin","family":"Huang","sequence":"additional","affiliation":[{"name":"The Graduate Institute of Design Science, Tatung University, Taipei 104, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,17]]},"reference":[{"key":"ref_1","unstructured":"Wong, J.H. (2022, May 23). Digitization Upgrade the Safety\u2014Taiwan Rail Track Inspection APP. Electronic Commercial Times, Available online: https:\/\/reurl.cc\/yrqZQM."},{"key":"ref_2","unstructured":"Plasser & Theurer (2022, November 03). Non-Contacting Measurement under Load in Real Time at up to 100 km\/h. Available online: https:\/\/www.plassertheurer.com\/en\/machine\/track-recording-cars\/em80h."},{"key":"ref_3","unstructured":"Ensco (2022, November 11). Track Inspection Services. Available online: https:\/\/www.ensco.com\/rail\/track-inspection-services."},{"key":"ref_4","unstructured":"Fugro (2022, May 23). RILA3.0, Fugro. Available online: https:\/\/www.fugro.com\/about-fugro\/our-expertise\/innovations\/rila#tabbed1."},{"key":"ref_5","unstructured":"Tech, J.C. (2022, November 11). AI Automatic Track Inspection Robot. Available online: https:\/\/www.linkedin.com\/company\/jc-technology."},{"key":"ref_6","unstructured":"Mermec (2022, November 15). Track Inspection: V-Cube. Available online: https:\/\/www.mermecgroup.com\/measuring-trains-br-and-systems\/track-inspection\/524\/track-inspection-v-cube.php."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1109\/TITS.2013.2287155","article-title":"Rail component detection, optimization, and assessment for automatic rail track inspection","volume":"15","author":"Li","year":"2014","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1109\/TITS.2016.2568758","article-title":"Deep multitask learning for railway track inspection","volume":"18","author":"Gibert","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","first-page":"151","article-title":"Rail track condition monitoring: A review on deep learning approaches","volume":"1","author":"Ji","year":"2021","journal-title":"Intell. Robot."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/BF00116251","article-title":"Induction of decision trees","volume":"1","author":"Quinlan","year":"1986","journal-title":"Mach. Learn."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Mucherino, A., Papajorgji, P.J., and Pardalos, P.M. (2009). k-Nearest neighbor classification. Data Mining in Agriculture, Springer. Springer Optimization and Its Applications.","DOI":"10.1007\/978-0-387-88615-2"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kao, M.Y. (2008). Support vector machines. Encyclopedia of Algorithms, Springer.","DOI":"10.1007\/978-0-387-30162-4"},{"key":"ref_13","unstructured":"Haykin, S. (1998). Neural Networks: A Comprehensive Foundation, Prentice Hall. [2nd ed.]."},{"key":"ref_14","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Nets. Advances in Neural Information Processing Systems 27 (NIPS 2014), NeurIPS."},{"key":"ref_15","unstructured":"(2022, January 05). Rail Defects Handbook, Available online: https:\/\/extranet.artc.com.au\/docs\/eng\/track-civil\/guidelines\/rail\/RC2400.pdf."},{"key":"ref_16","first-page":"1190","article-title":"A real time detection method of track fasteners missing of railway based on machine vision","volume":"14","author":"Ma","year":"2018","journal-title":"Int. J. Perform. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Gibert, X., Patel, V.M., and Chellappa, R. (2015, January 5\u20139). Robust fastener detection for autonomous visual railway track inspection. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV.2015.98"},{"key":"ref_18","unstructured":"Stella, E., Mazzeo, P., Nitti, M., Cicirelli, G., Distante, A., and D\u2019Orazio, T. (2003, January 6). Visual recognition of missing fastening elements for railroad maintenance. Proceedings of the IEEE 5th International Conference on Intelligent Transportation Systems, Singapore."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xia, Y., Xie, F., and Jiang, Z. (2011, January 13). Broken railway fastener detection based on adaboost algorithm. Proceedings of the 2010 International Conference on Optoelectronics and Image Processing, Haikou, China.","DOI":"10.1109\/ICOIP.2010.303"},{"key":"ref_20","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). ImageNet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS\u201912), Lake Tahoe, NV, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2015). You only look once: Unified, real-time object detection. arXiv.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_22","unstructured":"Eunus, S.I., Adnan, A., and Hossain, S. (2022, November 15). Railway Track Fault Detection. Kaggle. Available online: https:\/\/www.kaggle.com\/datasets\/salmaneunus\/railway-track-fault-detection."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"103484","DOI":"10.1016\/j.autcon.2020.103484","article-title":"Automated crack severity level detection and classification for ballastless track slab using deep convolutional neural network","volume":"124","author":"Wang","year":"2021","journal-title":"Autom. Constr."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chandran, P., Asber, J., Thiery, F., Odelius, J., and Rantatalo, M. (2021). An investigation of railway fastener detection using image processing and augmented deep learning. Sustainability, 13.","DOI":"10.3390\/su132112051"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"6509","DOI":"10.1109\/TII.2020.2966033","article-title":"Achieving predictive and proactive maintenance for high-speed railway power equipment with LSTM-RNN","volume":"16","author":"Wang","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1375","DOI":"10.1109\/TITS.2020.2969993","article-title":"Railway Traffic Object Detection using Differential Feature Fusion Convolution Neural Network","volume":"22","author":"Ye","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Jang, J., Shin, M., Lim, S., Park, J., Kim, J., and Paik, J. (2019). Intelligent image-based railway inspection system using deep learning-based object detection and weber contrast-based image comparison. Sensors, 19.","DOI":"10.3390\/s19214738"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1109\/TIM.2018.2868490","article-title":"Deep architecture for high-speed railway insulator surface defect detection: Denoising auto-encoder with multitask learning","volume":"68","author":"Kang","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, T., Yang, F., and Tsui, K.-L. (2020). Real-time detection of railway track component via one-stage deep learning networks. Sensors, 20.","DOI":"10.3390\/s20154325"},{"key":"ref_30","first-page":"3429","article-title":"Offline deep-learning-based defective track fastener detection and inspection system","volume":"32","author":"Hsieh","year":"2020","journal-title":"Sens. Mater."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Su, Z., Han, K., Song, W., and Ning, K. (2022, January 3\u20135). Railway fastener defect detection based on improved YOLOv5 algorithm. Proceedings of the IEEE 6th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Beijing, China.","DOI":"10.1109\/IAEAC54830.2022.9929911"},{"key":"ref_32","unstructured":"Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv, Available online: https:\/\/arxiv.org\/abs\/2004.10934."},{"key":"ref_33","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv, Available online: https:\/\/arxiv.org\/abs\/1804.02767."},{"key":"ref_34","unstructured":"Zheng, G., Liu, S.T., Wang, F., Li, Z.M., and Sun, J. (2021). YOLOX: Exceeding YOLO Series in 2021. arXiv, Available online: https:\/\/arxiv.org\/abs\/2107.08430."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2015). SSD: Single Shot Multibox Detector. arXiv, Available online: https:\/\/arxiv.org\/abs\/1512.02325.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_37","unstructured":"(2022, May 24). OpenCV Dev. Team. approxPolyDP, OpenCV, 21 Jan. Available online: https:\/\/docs.opencv.org\/2.4\/modules\/imgproc\/doc\/structural_analysis_and_shape_descriptors.html."},{"key":"ref_38","unstructured":"Hsieh, C.C., Hsieh, Y.P., Lai, J.Y., Huang, W.H., Hsieh, S.L., Hsu, T.Y., Du, Y.H., and Jia, H.W. (2022). A comprehensive detection system of defective track component by deep learning. Transp. Plan. J., in press."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017). Unpaired Image-To-Image Translation Using Cycle-Consistent Adversarial Networks. arXiv, Available online: https:\/\/arxiv.org\/abs\/1703.10593.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_40","unstructured":"Lin, T.T. (2022, May 24). LabelImg. GitHub. Available online: https:\/\/github.com\/tzutalin\/labelImg."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/24\/9970\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:43:24Z","timestamp":1760147004000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/24\/9970"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,17]]},"references-count":40,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["s22249970"],"URL":"https:\/\/doi.org\/10.3390\/s22249970","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,17]]}}}