{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T14:49:20Z","timestamp":1776782960054,"version":"3.51.2"},"reference-count":45,"publisher":"Public Library of Science (PLoS)","issue":"1","license":[{"start":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T00:00:00Z","timestamp":1769126400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100024165","name":"Helmut-Schmidt-Universit\u00e4t\/Universit\u00e4t der Bundeswehr Hamburg","doi-asserted-by":"publisher","award":["Open-Access-Publication-Fund"],"award-info":[{"award-number":["Open-Access-Publication-Fund"]}],"id":[{"id":"10.13039\/100024165","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>To ensure the operational safety of trains, it is essential to monitor the condition of the rails. In order to detect problems ranging from wear and tear to possible sabotage, no comprehensive and continuous monitoring is carried out today. The objective of this study is to develop and robustly validate a deep-learning model capable of reliably classifying a broad range of safety-critical rail surface defects. The use of deep learning to classify defect types based on optical images is a promising approach, but the existing literature does not yet achieve a robust, high-performing classification for a broad range of failure types. Many approaches rely mainly on local feature extraction, but this carries the risk of overlooking global relationships, which are crucial for distinguishing certain defect types. This study addresses this gap by utilizing the specific features of this problem domain (small, local, and global defect types). To this end, DualSightNet is introduced as a hybrid architecture enhanced by an attention module for classifying a broad range of railway track surface defects. The model achieves a five-fold cross-validated average balanced accuracy of 97.55\u2009% on a peer-reviewed, real-world dataset of 5,153 images covering seven defect types, recorded directly from an inspection vehicle under operational conditions, indicating strong generalization across the diverse real-world variations represented in the dataset. Compared to existing CNN- or Transformer-based approaches, DualSightNet is the first approach that combines local and global feature extraction through a gating-based fusion mechanism and then enhances the fused representation using an attention module, which enables substantially more robust multiclass defect recognition. This sets a new benchmark for our problem domain, surpassing previous approaches, which either lack broad defect coverage or do not employ rigorous cross-validated evaluation. Our results have far-reaching practical implications, proving that by leveraging problem-specific features, neural networks are able to robustly classify a broad range of defect types. The inference time of the proposed system (3.00 ms per image) makes DualSightNet suitable for deployment in automated inspection vehicles and real-time monitoring scenarios.<\/jats:p>","DOI":"10.1371\/journal.pone.0340789","type":"journal-article","created":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T18:44:52Z","timestamp":1769193892000},"page":"e0340789","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":1,"title":["DualSightNet: A novel dual architecture for visual quality control of railway infrastructure"],"prefix":"10.1371","volume":"21","author":[{"given":"Christopher","family":"Mai","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luca","family":"Eisentraut","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonas","family":"Ketterer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2263-6408","authenticated-orcid":true,"given":"Ricardo","family":"Buettner","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2026,1,23]]},"reference":[{"key":"pone.0340789.ref001","doi-asserted-by":"crossref","first-page":"123344","DOI":"10.1016\/j.apenergy.2024.123344","article-title":"On the right track? Energy use, carbon emissions and intensities of world rail transportation 1840 -2020","volume":"367","author":"B Tostes","year":"2024","journal-title":"Applied Energy."},{"issue":"1","key":"pone.0340789.ref002","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.retrec.2012.12.003","article-title":"The economics of railway safety","volume":"43","author":"AW Evans","year":"2013","journal-title":"Research in Transportation Economics."},{"issue":"5","key":"pone.0340789.ref003","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1007\/s11554-024-01548-z","article-title":"A real-time foreign object detection method based on deep learning in complex open railway environments","volume":"21","author":"B Zhang","year":"2024","journal-title":"J Real-Time Image Proc."},{"key":"pone.0340789.ref004","doi-asserted-by":"crossref","first-page":"105714","DOI":"10.1016\/j.autcon.2024.105714","article-title":"UAV-based studies in railway infrastructure monitoring","volume":"167","author":"P Aela","year":"2024","journal-title":"Automation in Construction."},{"issue":"7553","key":"pone.0340789.ref005","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"Y LeCun","year":"2015","journal-title":"Nature."},{"issue":"4","key":"pone.0340789.ref006","doi-asserted-by":"crossref","first-page":"040801","DOI":"10.1115\/1.4049535","article-title":"Image-based surface defect detection using deep learning: a review","volume":"21","author":"PM Bhatt","year":"2021","journal-title":"Journal of Computing and Information Science in Engineering."},{"key":"pone.0340789.ref007","doi-asserted-by":"crossref","first-page":"196702","DOI":"10.1109\/ACCESS.2024.3521497","article-title":"A novel deep learning-based approach for defect detection of synthetic leather using Gaussian filtering","volume":"12","author":"C Mai","year":"2024","journal-title":"IEEE Access."},{"issue":"1","key":"pone.0340789.ref008","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/s40537-016-0043-6","article-title":"A survey of transfer learning","volume":"3","author":"K Weiss","year":"2016","journal-title":"J Big Data."},{"issue":"1","key":"pone.0340789.ref009","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"N Srivastava","year":"2014","journal-title":"Journal of Machine Learning Research."},{"key":"pone.0340789.ref010","doi-asserted-by":"crossref","first-page":"43201","DOI":"10.1109\/ACCESS.2025.3549271","article-title":"Improvement of deep learning models using retinal filter: a systematic evaluation of the effect of gaussian filtering with a focus on industrial inspection data","volume":"13","author":"R Buettner","year":"2025","journal-title":"IEEE Access."},{"key":"pone.0340789.ref011","doi-asserted-by":"crossref","first-page":"103326","DOI":"10.1016\/j.rineng.2024.103326","article-title":"An end-to-end approach to detect railway track defects based on supervised and self-supervised learning","volume":"24","author":"M Haroon","year":"2024","journal-title":"Results in Engineering."},{"key":"pone.0340789.ref012","doi-asserted-by":"crossref","first-page":"181997","DOI":"10.1109\/ACCESS.2024.3509986","article-title":"CBAM-SwinT-BL: small rail surface defect detection method based on swin transformer with block level CBAM enhancement","volume":"12","author":"J Zhao","year":"2024","journal-title":"IEEE Access."},{"issue":"7","key":"pone.0340789.ref013","doi-asserted-by":"crossref","first-page":"3588","DOI":"10.3390\/app15073588","article-title":"PerMSCA-YOLO: a perceptual multi-scale convolutional attention enhanced YOLOv8 model for rail defect detection","volume":"15","author":"J Zhang","year":"2025","journal-title":"Applied Sciences."},{"issue":"1","key":"pone.0340789.ref014","doi-asserted-by":"crossref","first-page":"384","DOI":"10.3390\/app13010384","article-title":"Image classification-based defect detection of railway tracks using fiber bragg grating ultrasonic sensors","volume":"13","author":"D-Z Dang","year":"2022","journal-title":"Applied Sciences."},{"issue":"3","key":"pone.0340789.ref015","doi-asserted-by":"crossref","first-page":"891","DOI":"10.55730\/1300-0632.3817","article-title":"Defect classification of railway fasteners using image preprocessing and alightweight convolutional neural network","volume":"30","author":"\u0130 Aydin","year":"2022","journal-title":"Turkish Journal of Electrical Engineering and Computer Sciences."},{"issue":"2","key":"pone.0340789.ref016","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1007\/s13296-024-00927-3","article-title":"Rail Flaw B-scan image analysis using a hierarchical classification model","volume":"25","author":"G Hu","year":"2024","journal-title":"Int J Steel Struct."},{"key":"pone.0340789.ref017","doi-asserted-by":"crossref","unstructured":"Jatoi S, Abro B, Mushtaq N, Shah Syed AA, Narejo S, Rathi M, et al. From detection to diagnosis: elevating track fault identification with transfer learning. In: 2024 International Conference on Robotics and Automation in Industry (ICRAI), 2024. p. 1\u20136. https:\/\/doi.org\/10.1109\/icrai62391.2024.10894696","DOI":"10.1109\/ICRAI62391.2024.10894696"},{"key":"pone.0340789.ref018","doi-asserted-by":"crossref","first-page":"110050","DOI":"10.1016\/j.dib.2024.110050","article-title":"Railway track surface faults dataset","volume":"52","author":"A Arain","year":"2024","journal-title":"Data Brief."},{"key":"pone.0340789.ref019","doi-asserted-by":"crossref","first-page":"105965","DOI":"10.1016\/j.engappai.2023.105965","article-title":"MSRConvNet: classification of railway track defects using multi-scale residual convolutional neural network","volume":"121","author":"H Acikgoz","year":"2023","journal-title":"Engineering Applications of Artificial Intelligence."},{"issue":"23","key":"pone.0340789.ref020","doi-asserted-by":"crossref","first-page":"7937","DOI":"10.3390\/s21237937","article-title":"Detection and classification system for rail surface defects based on eddy current","volume":"21","author":"TA Alvarenga","year":"2021","journal-title":"Sensors (Basel)."},{"key":"pone.0340789.ref021","doi-asserted-by":"crossref","unstructured":"Li W, Wang J, Qin X, Jing G, Liu X. Artificial intelligence-aided detection of rail defects based on ultrasonic imaging data. Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and Rapid Transit. 2023;238(1):118\u201327. https:\/\/doi.org\/10.1177\/09544097231214578","DOI":"10.1177\/09544097231214578"},{"issue":"6","key":"pone.0340789.ref022","doi-asserted-by":"crossref","first-page":"225","DOI":"10.12700\/APH.19.6.2022.6.16","article-title":"Rail defect classification with deep learning method","volume":"19","author":"S Lu","year":"2022","journal-title":"Acta Polytech Hung."},{"key":"pone.0340789.ref023","doi-asserted-by":"crossref","first-page":"107246","DOI":"10.1016\/j.measurement.2019.107246","article-title":"Train rail defect classification detection and its parameters learning method","volume":"151","author":"F Wu","year":"2020","journal-title":"Measurement."},{"issue":"12","key":"pone.0340789.ref024","doi-asserted-by":"crossref","first-page":"6999","DOI":"10.1109\/TNNLS.2021.3084827","article-title":"A survey of convolutional neural networks: analysis, applications, and prospects","volume":"33","author":"Z Li","year":"2022","journal-title":"IEEE Trans Neural Netw Learn Syst."},{"key":"pone.0340789.ref025","doi-asserted-by":"crossref","first-page":"127547","DOI":"10.1016\/j.eswa.2025.127547","article-title":"ResNet-Swin transformer based intrusion detection system for in-vehicle network","volume":"279","author":"Z Wu","year":"2025","journal-title":"Expert Systems with Applications."},{"key":"pone.0340789.ref026","doi-asserted-by":"crossref","unstructured":"Deng J, Dong W, Socher R, Li L-J, Kai Li, Li F-F. ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009. p. 248\u201355. https:\/\/doi.org\/10.1109\/cvpr.2009.5206848","DOI":"10.1109\/CVPR.2009.5206848"},{"issue":"8","key":"pone.0340789.ref027","doi-asserted-by":"crossref","first-page":"5200","DOI":"10.1109\/TNNLS.2021.3119238","article-title":"Spiking deep residual networks","volume":"34","author":"Y Hu","year":"2023","journal-title":"IEEE Trans Neural Netw Learn Syst."},{"key":"pone.0340789.ref028","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016. p. 770\u20138. https:\/\/doi.org\/10.1109\/cvpr.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"pone.0340789.ref029","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.neunet.2022.01.012","article-title":"Low-degree term first in ResNet, its variants and the whole neural network family","volume":"148","author":"T Sun","year":"2022","journal-title":"Neural Netw."},{"key":"pone.0340789.ref030","unstructured":"Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Bach F, Blei D, editors. Proceedings of the 32nd International Conference on Machine Learning. vol. 37 of Proceedings of Machine Learning Research. Lille, France: PMLR; 2015. p. 448\u201356."},{"key":"pone.0340789.ref031","article-title":"Gated multimodal units for information fusion","author":"J Arevalo","year":"2017","journal-title":"arXiv preprint"},{"issue":"1","key":"pone.0340789.ref032","doi-asserted-by":"crossref","first-page":"100104","DOI":"10.1016\/j.nexres.2024.100104","article-title":"ResNet50-based deep learning model for accurate brain tumor detection in MRI scans","volume":"2","author":"A Rath","year":"2025","journal-title":"Next Research."},{"key":"pone.0340789.ref033","doi-asserted-by":"crossref","first-page":"112009","DOI":"10.1016\/j.asoc.2024.112009","article-title":"Explainable ResNet50 learning model based on copula entropy for cotton plant disease prediction","volume":"164","author":"H Askr","year":"2024","journal-title":"Applied Soft Computing."},{"key":"pone.0340789.ref034","doi-asserted-by":"crossref","unstructured":"Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, et al. Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE\/CVF International Conference on Computer Vision (ICCV). 2021. p. 9992\u201310002. https:\/\/doi.org\/10.1109\/iccv48922.2021.00986","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"pone.0340789.ref035","doi-asserted-by":"crossref","unstructured":"Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q. ECA-Net: efficient channel attention for deep convolutional neural networks. In: 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2020. p. 11531\u20139. https:\/\/doi.org\/10.1109\/cvpr42600.2020.01155","DOI":"10.1109\/CVPR42600.2020.01155"},{"key":"pone.0340789.ref036","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"F Pedregosa","year":"2011","journal-title":"Journal of Machine Learning Research."},{"key":"pone.0340789.ref037","article-title":"Evaluation: from precision, recall and f-factor to ROC, informedness, markedness & correlation","volume":"2","author":"D Powers","year":"2008","journal-title":"Mach Learn Technol."},{"key":"pone.0340789.ref038","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"F Pedregosa","year":"2011","journal-title":"The Journal of Machine Learning Research."},{"issue":"1","key":"pone.0340789.ref039","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","author":"J Cohen","year":"1960","journal-title":"Educational and Psychological Measurement."},{"key":"pone.0340789.ref040","doi-asserted-by":"crossref","first-page":"36065","DOI":"10.1109\/ACCESS.2025.3544923","article-title":"Deep learning-based detection of tuberculosis using a gaussian chest X-ray image filter as a software lens","volume":"13","author":"L Eisentraut","year":"2025","journal-title":"IEEE Access."},{"issue":"3","key":"pone.0340789.ref041","doi-asserted-by":"crossref","first-page":"533","DOI":"10.3390\/rs16030533","article-title":"Selecting and interpreting multiclass loss and accuracy assessment metrics for classifications with class imbalance: guidance and best practices","volume":"16","author":"S Farhadpour","year":"2024","journal-title":"Remote Sensing."},{"issue":"4","key":"pone.0340789.ref042","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"M Sokolova","year":"2009","journal-title":"Information Processing & Management."},{"issue":"1","key":"pone.0340789.ref043","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1002\/(SICI)1097-4571(199401)45:1<12::AID-ASI2>3.0.CO;2-L","article-title":"The relationship between recall and precision","volume":"45","author":"M Buckland","year":"1994","journal-title":"J Am Soc Inf Sci."},{"issue":"5","key":"pone.0340789.ref044","doi-asserted-by":"crossref","first-page":"503","DOI":"10.3390\/medicina57050503","article-title":"Foundational statistical principles in medical research: sensitivity, specificity, positive predictive value, and negative predictive value","volume":"57","author":"TF Monaghan","year":"2021","journal-title":"Medicina (Kaunas)."},{"issue":"21","key":"pone.0340789.ref045","doi-asserted-by":"crossref","first-page":"9863","DOI":"10.3390\/app14219863","article-title":"Performance metrics for multilabel emotion classification: comparing micro, macro, and weighted F1-scores","volume":"14","author":"MC Hinojosa Lee","year":"2024","journal-title":"Applied Sciences."}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0340789","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T18:44:59Z","timestamp":1769193899000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0340789"}},"subtitle":[],"editor":[{"given":"Asadullah","family":"Shaikh","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2026,1,23]]},"references-count":45,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1,23]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0340789","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,23]]}}}