{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T09:46:40Z","timestamp":1782726400229,"version":"3.54.5"},"reference-count":45,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T00:00:00Z","timestamp":1782691200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100019082","name":"Shanghai Aerospace Science and Technology Innovation Foundation","doi-asserted-by":"publisher","award":["SAST2021-054"],"award-info":[{"award-number":["SAST2021-054"]}],"id":[{"id":"10.13039\/501100019082","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100019082","name":"Shanghai Aerospace Science and Technology Innovation Foundation","doi-asserted-by":"publisher","award":["SAST2023-005"],"award-info":[{"award-number":["SAST2023-005"]}],"id":[{"id":"10.13039\/501100019082","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100016073","name":"Aviation Science Fund","doi-asserted-by":"publisher","award":["20240001053008"],"award-info":[{"award-number":["20240001053008"]}],"id":[{"id":"10.13039\/501100016073","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100016073","name":"Aviation Science Fund","doi-asserted-by":"publisher","award":["201907053004"],"award-info":[{"award-number":["201907053004"]}],"id":[{"id":"10.13039\/501100016073","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004826","name":"Beijing Natural Science Foundation","doi-asserted-by":"publisher","award":["L252146"],"award-info":[{"award-number":["L252146"]}],"id":[{"id":"10.13039\/501100004826","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Railway scene understanding is critical for ensuring train operational safety and advancing intelligent railway systems. Existing railway vision methods mainly focus on perception and classification, while lacking regulation-guided semantic reasoning capabilities in complex environments. To address these limitations, this paper proposes a retrieval-augmented generation (RAG)-enhanced vision\u2013language framework for railway signal cognition and safety reasoning. The proposed method integrates railway signal perception, regulatory knowledge retrieval, and multi-modal reasoning to improve factual consistency, reasoning reliability, and operational interpretability. In addition, a dedicated railway signal dataset comprising 500 standardized railway scene images with structured QA annotations is constructed to support regulation-oriented multi-modal recognition evaluation. Experimental results show that the proposed framework improves reasoning accuracy from 28.40% to 67.20% with an average end-to-end inference latency of 11.31 s per sample, and the inference speed can be further improved by adjusting experimental configurations to trade off between efficiency and accuracy, demonstrating the potential of RAG-enhanced architectures as a foundational step toward reliable multi-modal cognition in intelligent railway systems.<\/jats:p>","DOI":"10.3390\/computers15070416","type":"journal-article","created":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T08:45:20Z","timestamp":1782722720000},"page":"416","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["RAG-Enhanced Vision\u2013Language Framework and Dataset for Railway Signal Cognition and Safety Reasoning"],"prefix":"10.3390","volume":"15","author":[{"given":"Qunbo","family":"Wang","sequence":"first","affiliation":[{"name":"School of Automation and Intelligence, Beijing Jiaotong University, Beijing 100044, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shiyi","family":"Xiong","sequence":"additional","affiliation":[{"name":"School of Automation and Intelligence, Beijing Jiaotong University, Beijing 100044, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiawei","family":"Li","sequence":"additional","affiliation":[{"name":"School of Automation and Intelligence, Beijing Jiaotong University, Beijing 100044, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weiliang","family":"Li","sequence":"additional","affiliation":[{"name":"School of Automation and Intelligence, Beijing Jiaotong University, Beijing 100044, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chu","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Automation and Intelligence, Beijing Jiaotong University, Beijing 100044, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sen","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Automation and Intelligence, Beijing Jiaotong University, Beijing 100044, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xize","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Software, Northwestern Polytechnical University, Xi\u2019an 710072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chao","family":"Fan","sequence":"additional","affiliation":[{"name":"School of Software, Northwestern Polytechnical University, Xi\u2019an 710072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenjun","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Beihang University, Beijing 100191, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,29]]},"reference":[{"key":"ref_1","unstructured":"Li, Y., Li, L., Dao, A., Zhou, X., Qiao, Y., Mai, Z., Lee, D., Chen, Z., Tan, Z., and Bansal, M. (2025). IndustryNav: Exploring Spatial Reasoning of Embodied Agents in Dynamic Industrial Navigation. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1016\/j.ejor.2006.10.034","article-title":"A branch and bound algorithm for scheduling trains in a railway network","volume":"183","author":"Pacciarelli","year":"2007","journal-title":"Eur. J. Oper. Res."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"6427","DOI":"10.1109\/TITS.2024.3412170","article-title":"Railway intrusion detection based on machine vision: A survey, challenges, and perspectives","volume":"25","author":"Cao","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MCAS.2010.936782","article-title":"Automatic train control system development and simulation for high-speed railways","volume":"10","author":"Dong","year":"2010","journal-title":"IEEE Circuits Syst. Mag."},{"key":"ref_5","first-page":"100584","article-title":"Rail surface defect detection using a transformer-based network","volume":"38","author":"Guo","year":"2024","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1109\/TITS.2016.2568758","article-title":"Deep multitask learning for railway track inspection","volume":"18","author":"Gibert","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1177\/03611981251380276","article-title":"Quantification of Post-Rainfall Moisture Content in Pavement Unbound Layers Using Long-Term Pavement Performance Data","volume":"2680","author":"Li","year":"2025","journal-title":"Transp. Res. Rec. J. Transp. Res. Board"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"105362","DOI":"10.1016\/j.jappgeo.2024.105362","article-title":"Simultaneous prediction of petrophysical properties and formation layered thickness from acoustic logging data using a modular cascading residual neural network (MCARNN) with physical constraints","volume":"224","author":"Zhan","year":"2024","journal-title":"J. Appl. Geophys."},{"key":"ref_9","first-page":"469","article-title":"A deep full-sequence convolutional neural network for 3D sedimentary microfacies prediction using time-frequency seismic data","volume":"65","author":"Wang","year":"2026","journal-title":"Geophys. Prospect. Pet."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"4130","DOI":"10.1109\/TASE.2026.3653476","article-title":"Trajectory Planning Method for Fracture Reduction of Parallel Robots Based on DMP and APF","volume":"23","author":"Su","year":"2026","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4814","DOI":"10.1109\/TMECH.2024.3394491","article-title":"Adaptive human-robot interaction torque estimation with high accuracy and strong tracking ability for a lower limb rehabilitation robot","volume":"29","author":"Liang","year":"2024","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hu, Y., Yang, J., Chen, L., Li, K., Sima, C., Zhu, X., Chai, S., Du, S., Lin, T., and Wang, W. (2023). Planning-oriented autonomous driving. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE Computer Society.","DOI":"10.1109\/CVPR52729.2023.01712"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lv, C., Qi, M., Liu, L., and Ma, H. (2025). T2sg: Traffic topology scene graph for topology reasoning in autonomous driving. Proceedings of the Computer Vision and Pattern Recognition Conference, IEEE.","DOI":"10.1109\/CVPR52734.2025.01603"},{"key":"ref_14","unstructured":"Chen, H., Xiong, M., Lu, Y., Han, W., Deng, A., He, Y., Wu, J., Li, Y., Liu, Y., and Hooi, B. (2026). Mlr-bench: Evaluating ai agents on open-ended machine learning research. Advances in Neural Information Processing Systems 38, Curran Associates, Inc."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sima, C., Renz, K., Chitta, K., Chen, L., Zhang, H., Xie, C., Bei\u00dfwenger, J., Luo, P., Geiger, A., and Li, H. (2024). Drivelm: Driving with graph visual question answering. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-031-72943-0_15"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yu, C., Zhang, Y., Liu, Z., Ding, Z., Sun, Y., and Jin, Z. (2025). Frame: Feedback-refined agent methodology for enhancing medical research insights. Proceedings of the Findings of the Association for Computational Linguistics: ACL 2025, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2025.findings-acl.400"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Abisheva, G., Goranin, N., Razakhova, B., Aidynov, T., and Satybaldina, D. (2024). Specifics of data collection and data processing during formation of RailVista dataset for machine learning-and deep learning-based applications. Sensors, 24.","DOI":"10.3390\/s24165239"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wang, S., Yu, Z., Jiang, X., Lan, S., Shi, M., Chang, N., Kautz, J., Li, Y., and Alvarez, J.M. (2025). Omnidrive: A holistic vision-language dataset for autonomous driving with counterfactual reasoning. Proceedings of the Computer Vision and Pattern Recognition Conference, IEEE.","DOI":"10.1109\/CVPR52734.2025.02090"},{"key":"ref_19","unstructured":"An, R., Zeng, K., Lu, M., Yang, S., Zhang, R., Ji, H., Liang, H., and Zhang, W. (2025). Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Xing, Z., Zhang, X., Hu, Y., Jiang, B., He, T., Zhang, Q., Long, X., and Yin, W. (2025). Goalflow: Goal-driven flow matching for multimodal trajectories generation in end-to-end autonomous driving. Proceedings of the Computer Vision and Pattern Recognition Conference, IEEE.","DOI":"10.1109\/CVPR52734.2025.00157"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ye, H., Qi, M., Liu, Z., Liu, L., and Ma, H. (2025). Safedriverag: Towards safe autonomous driving with knowledge graph-based retrieval-augmented generation. Proceedings of the 33rd ACM International Conference on Multimedia, Association for Computing Machinery.","DOI":"10.1145\/3746027.3755868"},{"key":"ref_22","first-page":"539","article-title":"Research on revision of relevant provisions of high-speed railway emergency disposal in the regulations on railway technical management (RRTM)","volume":"2","author":"Deng","year":"2023","journal-title":"Railw. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hess, G., Lindstr\u00f6m, C., Fatemi, M., Petersson, C., and Svensson, L. (2025). Splatad: Real-time lidar and camera rendering with 3d gaussian splatting for autonomous driving. Proceedings of the Computer Vision and Pattern Recognition Conference, IEEE.","DOI":"10.1109\/CVPR52734.2025.01119"},{"key":"ref_24","first-page":"1","article-title":"Vision technologies with applications in traffic surveillance systems: A holistic survey","volume":"58","author":"Zhou","year":"2025","journal-title":"ACM Comput. Surv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Huang, X., Wang, J., Xia, Q., Chen, S., Yang, B., Li, X., Wang, C., and Wen, C. (2025). V2x-r: Cooperative lidar-4d radar fusion with denoising diffusion for 3d object detection. Proceedings of the Computer Vision and Pattern Recognition Conference, IEEE.","DOI":"10.1109\/CVPR52734.2025.02551"},{"key":"ref_26","unstructured":"Xie, T., Zhang, J., Bai, H., and Nowak, R. (2024). Deep active learning in the open world. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1177\/03611981251342236","article-title":"Bidirectional Long Short-Term Memory Approach for Infrastructure Health Monitoring Using On-Board Vibration Response","volume":"2679","year":"2025","journal-title":"Transp. Res. Rec."},{"key":"ref_28","first-page":"1","article-title":"Vision language models in autonomous driving: A survey and outlook","volume":"9","author":"Zhou","year":"2024","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"5450","DOI":"10.1109\/TITS.2023.3330570","article-title":"From simple to complex scenes: Learning robust feature representations for accurate human parsing","volume":"46","author":"Liu","year":"2024","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","first-page":"1","article-title":"Mask-guided mamba fusion for drone-based visible-infrared vehicle detection","volume":"62","author":"Wang","year":"2024","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Yuan, H., Li, X., Zhou, C., Li, Y., Chen, K., and Loy, C.C. (2024). Open-vocabulary sam: Segment and recognize twenty-thousand classes interactively. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-031-72775-7_24"},{"key":"ref_32","unstructured":"Sinhamahapatra, P., Bose, S., Roscher, K., and G\u00fcnnemann, S. (2025). Zero-shot open-vocabulary ood object detection and grounding using vision language models. Proceedings of the Northern Lights Deep Learning Conference 2025, PMLR."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1007\/s40534-025-00380-w","article-title":"Monitoring of railway tracks maintenance needs using dynamic responses collected by an in-service train","volume":"33","author":"Ghiasi","year":"2025","journal-title":"Railw. Eng. Sci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"12376","DOI":"10.1609\/aaai.v40i15.38230","article-title":"Perception in plan: Coupled perception and planning for end-to-end autonomous driving","volume":"Volume 40","author":"Zhang","year":"2026","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"ref_35","unstructured":"Feng, B., Mei, Z., Ost, J., Ghilotti, F., Li, B., Girgis, R., Majumdar, A., and Heide, F. (2025). Verdi: Vlm-embedded reasoning for autonomous driving. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2425","DOI":"10.1007\/s11263-022-01657-x","article-title":"Explainability of deep vision-based autonomous driving systems: Review and challenges","volume":"130","author":"Zablocki","year":"2022","journal-title":"Int. J. Comput. Vis."},{"key":"ref_37","unstructured":"Zhang, S., Li, R., Deng, S., Zheng, Z., Zhang, Y., Li, J., Zhang, K., Zhang, T., Wu, W., and Wang, Q. (2026). RailVQA: A Benchmark and Framework for Efficient Interpretable Visual Cognition in Automatic Train Operation. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Xie, S., Kong, L., Dong, Y., Sima, C., Zhang, W., Chen, Q.A., Liu, Z., and Pan, L. (2025). Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data and Metric Perspectives. Proceedings of the IEEE\/CVF International Conference on Computer Vision, IEEE.","DOI":"10.1109\/ICCV51701.2025.00621"},{"key":"ref_39","unstructured":"Qi, J., Xu, Z., Shao, R., Chen, Y., Di, J., Cheng, Y., Wang, Q., and Huang, L. (2024). Rora-vlm: Robust retrieval-augmented vision language models. arXiv."},{"key":"ref_40","unstructured":"Zhang, S., Huang, W., Gao, Z., Chen, H., and Lv, C. (2024). Wisead: Knowledge augmented end-to-end autonomous driving with vision-language model. arXiv."},{"key":"ref_41","unstructured":"Ding, Z., Zhang, H., Wu, P., Pang, G., Yang, Z., Wang, P., and Zhang, Y. (2025). Slowfastvad: Video anomaly detection via integrating simple detector and rag-enhanced vision-language model. arXiv."},{"key":"ref_42","unstructured":"Zhang, J., Yang, X., Wang, T., Yao, Y., Petiushko, A., and Li, B. (2025). Safeauto: Knowledge-enhanced safe autonomous driving with multimodal foundation models. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"110126","DOI":"10.1016\/j.ress.2024.110126","article-title":"A knowledge graph-based hazard prediction approach for preventing railway operational accidents","volume":"247","author":"Liu","year":"2024","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"3580","DOI":"10.1109\/TKDE.2024.3352100","article-title":"Unifying large language models and knowledge graphs: A roadmap","volume":"36","author":"Pan","year":"2024","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Xiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., and Nie, J.Y. (2024). C-pack: Packed resources for general chinese embeddings. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Association for Computing Machinery.","DOI":"10.1145\/3626772.3657878"}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/15\/7\/416\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T09:16:23Z","timestamp":1782724583000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/15\/7\/416"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,29]]},"references-count":45,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["computers15070416"],"URL":"https:\/\/doi.org\/10.3390\/computers15070416","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,29]]}}}