{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,10]],"date-time":"2026-05-10T08:21:16Z","timestamp":1778401276063,"version":"3.51.4"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,12,29]],"date-time":"2022-12-29T00:00:00Z","timestamp":1672272000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,12,29]],"date-time":"2022-12-29T00:00:00Z","timestamp":1672272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Discov Artif Intell"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The ever increasing demand in passenger and freight transportation is leading to the saturation of railway network capacity. Virtual Coupling (VC) has been proposed within the European Horizon 2020 Shift2Rail Joint Undertaking as a potential solution to address this problem. It allows to dynamically connect two or more trains in a single convoy, thus reducing headway between them. In this work, we investigate the main challenges related to the potential deployment of VC in railways. Its feasibility through Reinforcement Learning techniques is explored, discussing about technical implementation and performance issues. A qualitative analysis based on a Deep Deterministic Policy Gradient control algorithm is proposed. The aim is to give a first insight towards the definition of a qualitative and technology roadmap which could lead to the deployment of artificial intelligence applications aiming at enhancing rail safety and automation.<\/jats:p>","DOI":"10.1007\/s44163-022-00042-4","type":"journal-article","created":{"date-parts":[[2022,12,29]],"date-time":"2022-12-29T13:07:47Z","timestamp":1672319267000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Roadmap and challenges for reinforcement learning control in railway virtual coupling"],"prefix":"10.1007","volume":"2","author":[{"given":"Giacomo","family":"Basile","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elena","family":"Napoletano","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alberto","family":"Petrillo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefania","family":"Santini","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,12,29]]},"reference":[{"key":"42_CR1","unstructured":"Movingrail\u2014moving block and virtual coupling new generations of rail signalling.\u00a0https:\/\/cordis.europa.eu\/project\/id\/826347."},{"key":"42_CR2","unstructured":"X2rail3\u2014advanced signalling, automation and communication system (ip2 and ip5)\u2013 prototyping the future by means of capacity increase, autonomy and flexible communication.\u00a0https:\/\/projects.shift2rail.org\/s2r_ip2_n.aspx?p=X2RAIL-3."},{"issue":"6","key":"42_CR3","doi-asserted-by":"publisher","first-page":"2545","DOI":"10.1109\/TITS.2019.2920290","volume":"21","author":"C Di Meo","year":"2019","unstructured":"Di Meo C, Di Vaio M, Flammini F, Nardone R, Santini S, Vittorini V. Ertms\/etcs virtual coupling: proof of concept and numerical analysis. IEEE Trans Intell Transp Syst. 2019;21(6):2545\u201356.\u00a0https:\/\/elib.dlr.de\/137137\/1\/X2R3-TD2.8%20Virtually%20Coupled%20Train%20Sets_pdf.pdf.","journal-title":"IEEE Trans Intell Transp Syst"},{"key":"42_CR4","doi-asserted-by":"publisher","first-page":"91427","DOI":"10.1109\/ACCESS.2021.3091550","volume":"9","author":"P Singh","year":"2021","unstructured":"Singh P, Dulebenets MA, Pasha J, Gonzalez EDS, Lau Y-Y, Kampmann R. Deployment of autonomous trains in rail transportation: current trends and existing challenges. IEEE Access. 2021;9:91427\u201361.","journal-title":"IEEE Access"},{"key":"42_CR5","unstructured":"Park J, Lee B-H, Eun Y. Virtual coupling of railway vehicles: gap reference for merge and separation, robust control, and position measurement. IEEE Trans Intell Transp Syst 2020"},{"key":"42_CR6","unstructured":"RAILS: Roadmaps for A.I. Integration in the Rail Sector (2021). https:\/\/rails-project.eu\/"},{"key":"42_CR7","unstructured":"Schenker M. S2r innovation days presentation x2r3-td2.8 virtually coupled train sets (2020)"},{"key":"42_CR8","doi-asserted-by":"crossref","unstructured":"Flammini F, Marrone S, Nardone R, Petrillo A, Santini S, Vittorini V. Towards railway virtual coupling. In: 2018 IEEE international conference on electrical systems for aircraft, railway, ship propulsion and road vehicles & international transportation electrification conference (ESARS-ITEC). IEEE; 2018. p. 1\u20136","DOI":"10.1109\/ESARS-ITEC.2018.8607523"},{"key":"42_CR9","volume":"18","author":"M Nold","year":"2021","unstructured":"Nold M, Corman F. Dynamic train unit coupling and decoupling at cruising speed: systematic classification, operational potentials, and research agenda. J Rail Transp Plan Manag. 2021;18: 100241.","journal-title":"J Rail Transp Plan Manag"},{"key":"42_CR10","unstructured":"Quaglietta E. Analysis of platooning train operations under v2v communication-based signaling: Fundamental modelling and capacity impacts of virtual coupling. In: Proceedings of the 98th transportation research board annual meeting. Transportation Research Board (TRB); 2019"},{"key":"42_CR11","volume":"15","author":"E Quaglietta","year":"2020","unstructured":"Quaglietta E, Wang M, Goverde RM. A multi-state train-following model for the analysis of virtual coupling railway operations. J Rail Transp Plan Manag. 2020;15: 100195.","journal-title":"J Rail Transp Plan Manag"},{"issue":"8","key":"42_CR12","doi-asserted-by":"publisher","first-page":"799","DOI":"10.1177\/0361198120925074","volume":"2674","author":"J Aoun","year":"2020","unstructured":"Aoun J, Quaglietta E, Goverde RM. Investigating market potentials and operational scenarios of virtual coupling railway signaling. Transp Res Rec. 2020;2674(8):799\u2013812.","journal-title":"Transp Res Rec"},{"key":"42_CR13","volume":"22","author":"E Quaglietta","year":"2022","unstructured":"Quaglietta E, Spartalis P, Wang M, Goverde RM, van Koningsbruggen P. Modelling and analysis of virtual coupling with dynamic safety margin considering risk factors in railway operations. J Rail Transp Plan Manag. 2022;22: 100313.","journal-title":"J Rail Transp Plan Manag"},{"key":"42_CR14","unstructured":"Matthias\u00a0Grimm MP. Kupplungseinrichtung f\u00fcr Schienenfahrzeuge. https:\/\/patents.google.com\/patent\/DE102007050937A1\/de"},{"key":"42_CR15","doi-asserted-by":"publisher","unstructured":"Wu Q, Ge X, Han Q-L, Wang B, Wu H, Cole C, Spiryagin M. Dynamics and control simulation of railway virtual coupling. Veh Syst Dyn 2022; 1\u201325.\u00a0https:\/\/doi.org\/10.1080\/00423114.2022.2105241.","DOI":"10.1080\/00423114.2022.2105241"},{"issue":"2","key":"42_CR16","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1049\/iet-its.2019.0330","volume":"14","author":"Y Zhang","year":"2020","unstructured":"Zhang Y, Wang H. Topological manifold-based monitoring method for train-centric virtual coupling control systems. IET Intel Transp Syst. 2020;14(2):91\u2013102.","journal-title":"IET Intel Transp Syst"},{"issue":"7","key":"42_CR17","doi-asserted-by":"publisher","first-page":"2728","DOI":"10.1109\/TITS.2019.2914910","volume":"20","author":"J Felez","year":"2019","unstructured":"Felez J, Kim Y, Borrelli F. A model predictive control approach for virtual coupling in railways. IEEE Trans Intell Transp Syst. 2019;20(7):2728\u201339.","journal-title":"IEEE Trans Intell Transp Syst"},{"key":"42_CR18","doi-asserted-by":"publisher","first-page":"56354","DOI":"10.1109\/ACCESS.2021.3071820","volume":"9","author":"Z Wu","year":"2021","unstructured":"Wu Z, Gao C, Tang T. A virtually coupled metro train platoon control approach based on model predictive control. IEEE Access. 2021;9:56354\u201363.","journal-title":"IEEE Access"},{"key":"42_CR19","doi-asserted-by":"crossref","unstructured":"Liu Y, Liu R, Wei C, Xun J, Tang T. Distributed model predictive control strategy for constrained high-speed virtually coupled train set. IEEE Trans Veh Technol 2021;71(1):171\u201383.","DOI":"10.1109\/TVT.2021.3130715"},{"issue":"12","key":"42_CR20","doi-asserted-by":"publisher","first-page":"13340","DOI":"10.1109\/TVT.2021.3122257","volume":"70","author":"SB Prathiba","year":"2021","unstructured":"Prathiba SB, Raja G, Dev K, Kumar N, Guizani M. A hybrid deep reinforcement learning for autonomous vehicles smart-platooning. IEEE Trans Veh Technol. 2021;70(12):13340\u201350.","journal-title":"IEEE Trans Veh Technol"},{"key":"42_CR21","doi-asserted-by":"crossref","unstructured":"Coppola A, Petrillo A, Rizzo R, Santini S. Adaptive cruise control for autonomous electric vehicles based on q-learning algorithm. In: 2021 AEIT international annual conference (AEIT). IEEE; 2021. p. 1\u20136","DOI":"10.23919\/AEIT53387.2021.9627059"},{"key":"42_CR22","unstructured":"RAILS: Deliverable D2.1: WP2 Report on case studies and analysis of transferability from other sectors (safety and automation). 2021. https:\/\/rails-project.eu\/downloads\/deliverables"},{"key":"42_CR23","doi-asserted-by":"crossref","unstructured":"Ning L, Li Y, Zhou M, Song H, Dong H. A deep reinforcement learning approach to high-speed train timetable rescheduling under disturbances. In: 2019 IEEE intelligent transportation systems conference (ITSC). IEEE; 2019. p. 3469\u20133474","DOI":"10.1109\/ITSC.2019.8917180"},{"key":"42_CR24","doi-asserted-by":"publisher","first-page":"708","DOI":"10.1016\/j.ins.2021.04.088","volume":"570","author":"M Shang","year":"2021","unstructured":"Shang M, Zhou Y, Fujita H. Deep reinforcement learning with reference system to handle constraints for energy-efficient train control. Inf Sci. 2021;570:708\u201321.","journal-title":"Inf Sci"},{"key":"42_CR25","unstructured":"Schenker M, Parise R, Goikoetxea J. Concept and performance analysis of virtual coupling for railway vehicles. In: Proceedings of the 3rd SmartRaCon scientific seminar, vol. 38. Deutsches Zentrum f\u00fcr Luft-und Raumfahrt eV Institut f\u00fcr Verkehrssystemtechnik. 2021. p. 81\u201391"},{"key":"42_CR26","volume-title":"Principles of railway location and design","author":"S Yi","year":"2017","unstructured":"Yi S. Principles of railway location and design. Academic Press; 2017."},{"key":"42_CR27","doi-asserted-by":"crossref","unstructured":"van Nunen E, Esposto F, Saberi AK, Paardekooper J-P. Evaluation of safety indicators for truck platooning. In: 2017 IEEE intelligent vehicles symposium (IV). IEEE; 2017. p. 1013\u20131018","DOI":"10.1109\/IVS.2017.7995847"},{"key":"42_CR28","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1016\/j.apenergy.2016.01.097","volume":"168","author":"C Fiori","year":"2016","unstructured":"Fiori C, Ahn K, Rakha HA. Power-based electric vehicle energy consumption model: model development and validation. Appl Energy. 2016;168:257\u201368.","journal-title":"Appl Energy"},{"issue":"4","key":"42_CR29","doi-asserted-by":"publisher","first-page":"1272","DOI":"10.1109\/TCST.2019.2908146","volume":"28","author":"Y Wu","year":"2019","unstructured":"Wu Y, Li SE, Cort\u00e9s J, Poolla K. Distributed sliding mode control for nmiscar heterogeneous platoon systems with positive definite topologies. IEEE Trans Control Syst Technol. 2019;28(4):1272\u201383.","journal-title":"IEEE Trans Control Syst Technol"},{"key":"42_CR30","doi-asserted-by":"crossref","unstructured":"Rasmussen CE. Gaussian processes in machine learning. In: Summer school on machine learning. Springer; 2003. p. 63\u201371","DOI":"10.1007\/978-3-540-28650-9_4"},{"issue":"4","key":"42_CR31","doi-asserted-by":"publisher","first-page":"3211","DOI":"10.1007\/s11831-020-09496-0","volume":"28","author":"A Thakkar","year":"2021","unstructured":"Thakkar A, Lohiya R. A review on machine learning and deep learning perspectives of IDS for IoT: recent updates, security issues, and challenges. Arch Comput Methods Eng. 2021;28(4):3211\u201343.","journal-title":"Arch Comput Methods Eng"},{"key":"42_CR32","unstructured":"Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D. Continuous control with deep reinforcement learning. 2015. arXiv preprint arXiv:1509.02971"},{"key":"42_CR33","doi-asserted-by":"publisher","unstructured":"Kiran BR, et al. Deep reinforcement learning for autonomous driving: a survey. IEEE Trans Intell Transp Syst. 2022;23(6):4909\u201326. https:\/\/doi.org\/10.1109\/TITS.2021.3054625.","DOI":"10.1109\/TITS.2021.3054625"},{"key":"42_CR34","unstructured":"RAILS: Deliverable D2.1: WP2 Report on case studies and analysis of transferability from other sectors (safety and automation). 2022. https:\/\/rails-project.eu\/downloads\/deliverables"},{"key":"42_CR35","unstructured":"Farag A, AbdelAziz OM, Hussein A, Shehata OM. Reinforcement learning based approach for multi-vehicle platooning problem with nmiscar dynamic behavior. https:\/\/www.researchgate.net\/profile\/Amr-Ramadan-6\/publication\/349313418_Reinforcement_Learning_Based_Approach_for_Multi-Vehicle_Platooning_Problem_with_Nonlinear_Dynamic_Behavior\/links\/602a65ec92851c4ed57317a3\/Reinforcement-Learning-Based-Approach-for-Multi-Vehicle-Platooning-Problem-with-Nonlinear-Dynamic-Behavior.pdf."},{"issue":"2","key":"42_CR36","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1109\/TIV.2020.3012947","volume":"6","author":"Y Lin","year":"2020","unstructured":"Lin Y, McPhee J, Azad NL. Comparison of deep reinforcement learning and model predictive control for adaptive cruise control. IEEE Trans Intell Veh. 2020;6(2):221\u201331.","journal-title":"IEEE Trans Intell Veh"},{"key":"42_CR37","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2020.102886","volume":"125","author":"Y Liu","year":"2021","unstructured":"Liu Y, Zhou Y, Su S, Xun J, Tang T. An analytical optimal control approach for virtually coupled high-speed trains with local and string stability. Transp Res Part C Emerg Technol. 2021;125: 102886.","journal-title":"Transp Res Part C Emerg Technol"},{"key":"42_CR38","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2020.102662","volume":"117","author":"M Zhu","year":"2020","unstructured":"Zhu M, Wang Y, Pu Z, Hu J, Wang X, Ke R. Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving. Transp Res Part C Emerg Technol. 2020;117: 102662.","journal-title":"Transp Res Part C Emerg Technol"},{"key":"42_CR39","doi-asserted-by":"publisher","unstructured":"Aradi S. Survey of deep reinforcement learning for motion planning of autonomous vehicles. IEEE Trans Intell Transp Syst. 2022;23(2):740\u201359. https:\/\/doi.org\/10.1109\/TITS.2020.3024655.","DOI":"10.1109\/TITS.2020.3024655"},{"key":"42_CR40","unstructured":"Zhu Z, Lin K, Zhou J. Transfer learning in deep reinforcement learning: a survey. arXiv preprint arXiv:2009.07888 (2020)"},{"issue":"3","key":"42_CR41","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1002\/rob.21918","volume":"37","author":"S Grigorescu","year":"2020","unstructured":"Grigorescu S, Trasnea B, Cocias T, Macesanu G. A survey of deep learning techniques for autonomous driving. J Field Robot. 2020;37(3):362\u201386.","journal-title":"J Field Robot"},{"key":"42_CR42","doi-asserted-by":"crossref","unstructured":"Chen J, Yuan B, Tomizuka M. Model-free deep reinforcement learning for urban autonomous driving. In: 2019 IEEE intelligent transportation systems conference (ITSC). IEEE; 2019. p. 2765\u20132771","DOI":"10.1109\/ITSC.2019.8917306"}],"container-title":["Discover Artificial Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44163-022-00042-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s44163-022-00042-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44163-022-00042-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,29]],"date-time":"2022-12-29T13:08:17Z","timestamp":1672319297000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s44163-022-00042-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,29]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["42"],"URL":"https:\/\/doi.org\/10.1007\/s44163-022-00042-4","relation":{},"ISSN":["2731-0809"],"issn-type":[{"value":"2731-0809","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,29]]},"assertion":[{"value":"27 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 November 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 December 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"27"}}