{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,10]],"date-time":"2025-11-10T21:22:20Z","timestamp":1762809740998,"version":"build-2065373602"},"reference-count":22,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,5,12]],"date-time":"2025-05-12T00:00:00Z","timestamp":1747008000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2021YFF0501101","62106074","52272347","2024JJ7132"],"award-info":[{"award-number":["2021YFF0501101","62106074","52272347","2024JJ7132"]}]},{"name":"National Natural Science Foundation of China","award":["2021YFF0501101","62106074","52272347","2024JJ7132"],"award-info":[{"award-number":["2021YFF0501101","62106074","52272347","2024JJ7132"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2021YFF0501101","62106074","52272347","2024JJ7132"],"award-info":[{"award-number":["2021YFF0501101","62106074","52272347","2024JJ7132"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Science Fund of Hunan","award":["2021YFF0501101","62106074","52272347","2024JJ7132"],"award-info":[{"award-number":["2021YFF0501101","62106074","52272347","2024JJ7132"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>To address the challenges in existing multi-sensor data fusion methods for rail surface defect diagnosis, particularly their limitations in fully exploiting potential synergistic information among multimodal data and effectively bridging the semantic gap between heterogeneous multi-source data, this paper proposes a diagnostic approach based on a Progressive Joint Representation Graph Attention Fusion Network (PJR-GAFN). The methodology comprises five principal phases: Firstly, shared and specific autoencoders are used to extract joint representations of multimodal features through shared and modality-specific representations. Secondly, a squeeze-and-excitation module is implemented to amplify defect-related features while suppressing non-essential characteristics. Thirdly, a progressive fusion module is introduced to comprehensively utilize cross-modal synergistic information during feature extraction. Fourthly, a source domain classifier and domain discriminator are employed to capture modality-invariant features across different modalities. Finally, the spatial attention aggregation properties of graph attention networks are leveraged to fuse multimodal features, thereby fully exploiting contextual semantic information. Experimental results on real-world rail surface defect datasets from domestic railway lines demonstrate that the proposed method achieves 95% diagnostic accuracy, confirming its effectiveness in rail surface defect detection.<\/jats:p>","DOI":"10.3390\/bdcc9050127","type":"journal-article","created":{"date-parts":[[2025,5,12]],"date-time":"2025-05-12T04:21:23Z","timestamp":1747023683000},"page":"127","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Rail Surface Defect Diagnosis Based on Image\u2013Vibration Multimodal Data Fusion"],"prefix":"10.3390","volume":"9","author":[{"given":"Zhongmei","family":"Wang","sequence":"first","affiliation":[{"name":"College of Railway Transportation, Hunan University of Technology, Zhuzhou 412007, China"}]},{"given":"Shenao","family":"Peng","sequence":"additional","affiliation":[{"name":"College of Railway Transportation, Hunan University of Technology, Zhuzhou 412007, China"}]},{"given":"Wenxiu","family":"Ao","sequence":"additional","affiliation":[{"name":"College of Railway Transportation, Hunan University of Technology, Zhuzhou 412007, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1694-0975","authenticated-orcid":false,"given":"Jianhua","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Railway Transportation, Hunan University of Technology, Zhuzhou 412007, China"}]},{"given":"Changfan","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Railway Transportation, Hunan University of Technology, Zhuzhou 412007, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,12]]},"reference":[{"key":"ref_1","first-page":"356","article-title":"Strategy analysis on railway insured transportation and freight insurance cooperative development based on Hotelling model","volume":"20","author":"Fenling","year":"2023","journal-title":"J. Railw. Sci. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/s10033-025-01186-x","article-title":"An Investigation of Vibrations of a Flexible Rotor System with the Unbalanced Force and Time-Varying Bearing Force","volume":"38","author":"Xu","year":"2025","journal-title":"Chin. J. Mech. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1109\/EMR.2023.3262282","article-title":"Predictive Maintenance for Railway Domain: A Systematic Literature Review","volume":"51","author":"Binder","year":"2023","journal-title":"IEEE Eng. Manag. Rev."},{"key":"ref_4","first-page":"248","article-title":"Real-Time Detection System of Rail Surface Defects Based on Semantic Segmentation","volume":"57","author":"Li","year":"2021","journal-title":"Comput. Eng. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"132480","DOI":"10.1109\/ACCESS.2024.3427347","article-title":"A Multi-Modal Approach to Rail Surface Condition Analysis: The MFDF-Net","volume":"12","author":"Shen","year":"2024","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"21971","DOI":"10.1109\/TITS.2022.3189677","article-title":"CUFuse: Camera and Ultrasound Data Fusion for Rail Defect Detection","volume":"23","author":"Chen","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: An overview","volume":"61","author":"Schmidhuber","year":"2015","journal-title":"Neural Netw."},{"key":"ref_9","first-page":"1","article-title":"An Anomaly Detection Method for Railway Track Using Semisupervised Learning and Vision-Lidar Decision Fusion","volume":"73","author":"Ge","year":"2024","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"972","DOI":"10.1109\/LSP.2023.3299218","article-title":"CSANet: Contour and Semantic Feature Alignment Fusion Network for Rail Surface Defect Detection","volume":"30","author":"Yang","year":"2023","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"105342","DOI":"10.1016\/j.autcon.2024.105342","article-title":"3D tensor-based point cloud and image fusion for robust detection and measurement of rail surface defects","volume":"161","author":"Wang","year":"2024","journal-title":"Autom. Constr."},{"key":"ref_12","unstructured":"Piao, Y., Ji, W., Li, J., Zhang, M., and Lu, H. (November, January 27). Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_13","first-page":"285","article-title":"Rail Surface Defect Method Based on Bimodal-Modal Deep Learning","volume":"59","author":"Zhao","year":"2023","journal-title":"Comput. Eng. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, N., Zhang, N., and Han, J. (2020, January 13\u201319). Learning Selective Self-Mutual Attention for RGB-D Saliency Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01377"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yang, T., Xu, T., Cheng, Y., Tang, Z., Su, S., and Cao, Y. (2023, January 24\u201326). A fusion method based on 1D vibration signals and 2D images for detection of railway surface defects. Proceedings of the 2023 3rd International Conference on Neural Networks, Information and Communication Engineering (NNICE), Guangzhou, China.","DOI":"10.1109\/NNICE58320.2023.10105728"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-Excitation Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_17","unstructured":"Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2017). Graph attention networks. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Choi, J.Y., and Han, J.M. (2024). Deep Learning (Fast R-CNN)-Based Evaluation of Rail Surface Defects. Appl. Sci., 14.","DOI":"10.3390\/app14051874"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"04024041","DOI":"10.1061\/JPCFEV.CFENG-4816","article-title":"Autonomous Rail Surface Defect Identification Based on an Improved One-Stage Object Detection Algorithm","volume":"38","author":"Wang","year":"2024","journal-title":"J. Perform. Constr. Facil."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"100178","DOI":"10.1016\/j.geits.2024.100178","article-title":"Remote condition monitoring of rail tracks using distributed acoustic sensing (DAS): A deep CNN-LSTM-SW based model","volume":"3","author":"Rahman","year":"2024","journal-title":"Green Energy Intell. Transp."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"482","DOI":"10.3390\/ai5020024","article-title":"ECARRNet: An Efficient LSTM-Based Ensembled Deep Neural Network Architecture for Railway Fault Detection","volume":"5","author":"Eunus","year":"2024","journal-title":"AI"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lessmeier, C., Kimotho, J.K., Zimmer, D., and Sextro, W. (2016, January 19\u201321). Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: A benchmark data set for data-driven classification. Proceedings of the PHM Society European Conference, Chengdu, China.","DOI":"10.36001\/phme.2016.v3i1.1577"}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/5\/127\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:31:06Z","timestamp":1760031066000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/5\/127"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,12]]},"references-count":22,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["bdcc9050127"],"URL":"https:\/\/doi.org\/10.3390\/bdcc9050127","relation":{},"ISSN":["2504-2289"],"issn-type":[{"type":"electronic","value":"2504-2289"}],"subject":[],"published":{"date-parts":[[2025,5,12]]}}}