{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,22]],"date-time":"2025-05-22T04:43:54Z","timestamp":1747889034065,"version":"3.41.0"},"reference-count":30,"publisher":"Public Library of Science (PLoS)","issue":"5","license":[{"start":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T00:00:00Z","timestamp":1747785600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Fujian Provinces Education Research Project for Young and Middle-aged Teachers","award":["JAT231096"],"award-info":[{"award-number":["JAT231096"]}]},{"name":"Minjiang University Talent Introduction Research Project","award":["MJU24003, MJY23035"],"award-info":[{"award-number":["MJU24003, MJY23035"]}]},{"name":"Innovation Star Talent Program of the Third Batch in Fujian Province","award":["003002"],"award-info":[{"award-number":["003002"]}]},{"name":"Special Fund for Education and Scientific Research of Fujian Provincial Department of Finance","award":["GY-Z21001"],"award-info":[{"award-number":["GY-Z21001"]}]},{"name":"Scientific Research Foundation of Fujian University of Technology","award":["GY-Z22071"],"award-info":[{"award-number":["GY-Z22071"]}]},{"name":"The Natural Science Foundation of Fujian Province","award":["2024J011180, 2023J05251, 2024J08275, 2022J01117"],"award-info":[{"award-number":["2024J011180, 2023J05251, 2024J08275, 2022J01117"]}]},{"name":"Science and Education Joint Special Project of Minjiang University","award":["MJKJ24005"],"award-info":[{"award-number":["MJKJ24005"]}]},{"name":"2023 National College Students' Innovation and Entrepreneurship Training Program","award":["202310395011"],"award-info":[{"award-number":["202310395011"]}]},{"name":"The Fujian Provincial Social Science Foundation","award":["FJ2024C196"],"award-info":[{"award-number":["FJ2024C196"]}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>In recent decades, automatic train operation (ATO) systems have been gradually adopted by many metro systems, primarily due to their cost-effectiveness and practicality. However, a critical examination reveals computational constraints, adaptability to unforeseen conditions and multi-objective balancing that our research aims to address. In this paper, expert knowledge is combined with deep reinforcement learning algorithm (Proximal Policy Optimization, PPO) and two enhanced intelligent train operation algorithms (EITO) are proposed. The first algorithm, EITO<jats:sub>E<\/jats:sub>, is based on an expert system containing expert rules and a heuristic expert inference method. On the basis of EITO<jats:sub>E<\/jats:sub>, we propose EITO<jats:sub>P<\/jats:sub> algorithm using the PPO algorithm to optimize multiple objectives by designing reinforcement learning strategies, rewards, and value functions. We also develop the double minimal-time distribution (DMTD) calculation method in the EITO implementation to achieve longer coasting distances and further optimize the energy consumption. Compared with previous works, EITO enables the control of continuous train operation without reference to offline speed profiles and optimizes several key performance indicators online. Finally, we conducted comparative tests of the manual driving, intelligent driving algorithm (ITOR, STON), and the algorithms proposed in this paper, EITO, using real line data from the Yizhuang Line of Beijing Metro (YLBS). The test results show that the EITO outperform the current intelligent driving algorithms and manual driving in terms of energy consumption and passengers\u2019 comfort. In addition, we further validated the robustness of EITO by selecting some complex lines with speed limits, gradients and different running times for testing on the YLBS. Overall, the EITO<jats:sub>P<\/jats:sub> algorithm has the best performance.<\/jats:p>","DOI":"10.1371\/journal.pone.0323478","type":"journal-article","created":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T20:23:39Z","timestamp":1747859019000},"page":"e0323478","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Enhanced intelligent train operation algorithms for metro train based on expert system and deep reinforcement learning"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6265-0274","authenticated-orcid":true,"given":"Yunhu","family":"Huang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenzhu","family":"Lai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dewang","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Geng","family":"Lin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiateng","family":"Yin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2025,5,21]]},"reference":[{"key":"pone.0323478.ref001","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1016\/j.trc.2017.09.009","article-title":"Research and development of automatic train operation for railway transportation systems: A survey","volume":"85","author":"J Yin","year":"2017","journal-title":"Transport Res Part C: Emerg Technol"},{"key":"pone.0323478.ref002","doi-asserted-by":"crossref","first-page":"8519","DOI":"10.1007\/s10489-020-02152-x","article-title":"Deep traffic congestion prediction model based on road segment grouping","volume":"51","author":"Y Tu","year":"2021","journal-title":"Appl Intell"},{"issue":"9","key":"pone.0323478.ref003","article-title":"Deviation of peak hours for metro stations based on least square support vector machine","volume":"18","author":"L Yu","year":"2023","journal-title":"PLoS One"},{"issue":"7","key":"pone.0323478.ref004","doi-asserted-by":"crossref","first-page":"1257","DOI":"10.1109\/9.867018","article-title":"On an optimal control problem of train operation","volume":"45","author":"E Khmelnitsky","year":"2000","journal-title":"IEEE Trans Automat Contr"},{"issue":"5","key":"pone.0323478.ref005","doi-asserted-by":"crossref","first-page":"1913","DOI":"10.1109\/TITS.2014.2303146","article-title":"A two-objective timetable optimization model in subway systems","volume":"15","author":"X Yang","year":"2014","journal-title":"IEEE Trans Intell Transport Syst"},{"issue":"6","key":"pone.0323478.ref006","doi-asserted-by":"crossref","first-page":"3337","DOI":"10.1109\/TITS.2015.2445920","article-title":"Efficient real-time train scheduling for urban rail transit systems using iterative convex programming","volume":"16","author":"Y Wang","year":"2015","journal-title":"IEEE Trans Intell Transport Syst"},{"issue":"4","key":"pone.0323478.ref007","doi-asserted-by":"crossref","first-page":"2215","DOI":"10.1109\/TITS.2015.2402160","article-title":"Multiobjective optimization for train speed trajectory in CTCS high-speed railway with hybrid evolutionary algorithm","volume":"16","author":"W ShangGuan","year":"2015","journal-title":"IEEE Trans Intell Transport Syst"},{"issue":"3","key":"pone.0323478.ref008","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1049\/iet-epa:20070381","article-title":"Coasting point optimisation for mass rail transit lines using artificial neural networks and genetic algorithms","volume":"2","author":"S Akba","year":"2008","journal-title":"IET Electr Power Appl"},{"issue":"5","key":"pone.0323478.ref009","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1016\/j.omega.2011.12.001","article-title":"Optimizing trains movement on a railway network","volume":"40","author":"L Yang","year":"2012","journal-title":"Omega"},{"issue":"6","key":"pone.0323478.ref010","doi-asserted-by":"crossref","first-page":"2561","DOI":"10.1109\/TITS.2014.2320757","article-title":"Intelligent train operation algorithms for subway by expert system and reinforcement learning","volume":"15","author":"J Yin","year":"2014","journal-title":"IEEE Trans Intell Transp Syst"},{"issue":"3","key":"pone.0323478.ref011","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1016\/j.aei.2016.07.004","article-title":"Data-driven train operation models based on data mining and driving experience for the diesel-electric locomotive","volume":"30","author":"C Zhang","year":"2016","journal-title":"Adv Eng Inform"},{"key":"pone.0323478.ref012","article-title":"Smart train operation algorithms based on expert knowledge and reinforcement learning","author":"K Zhou","year":"2020","journal-title":"IEEE Trans Syst Man Cybern Syst"},{"issue":"7","key":"pone.0323478.ref013","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1016\/j.conengprac.2011.03.003","article-title":"Optimization of train-speed trajectory and control for mass rapid transit systems","volume":"19","author":"B-R Ke","year":"2011","journal-title":"Control Eng Pract"},{"key":"pone.0323478.ref014","doi-asserted-by":"crossref","unstructured":"Yong-Duan Song, Qi Song, Wen-Chuan Cai. Fault-tolerant adaptive control of high-speed trains under traction\/braking failures: A virtual parameter-based approach. IEEE Trans Intell Transport Syst. 2014;15(2):737\u201348. doi: 10.1109\/tits.2013.2290310","DOI":"10.1109\/TITS.2013.2290310"},{"issue":"15","key":"pone.0323478.ref015","doi-asserted-by":"crossref","first-page":"6115","DOI":"10.1016\/j.eswa.2013.04.034","article-title":"A high speed railway control system based on the fuzzy control method","volume":"40","author":"WY Liu","year":"2013","journal-title":"Expert Syst Appl"},{"issue":"3","key":"pone.0323478.ref016","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1109\/TITS.2013.2296655","article-title":"Energy-efficient train operation in urban rail transit using real-time traffic information","volume":"15","author":"Q Gu","year":"2014","journal-title":"IEEE Trans Intell Transp Syst"},{"issue":"4","key":"pone.0323478.ref017","doi-asserted-by":"crossref","first-page":"1733","DOI":"10.1109\/TITS.2013.2266255","article-title":"Approximation-based robust adaptive automatic train control: An approach for actuator saturation","volume":"14","author":"S Gao","year":"2013","journal-title":"IEEE Trans Intell Transp Syst"},{"issue":"10","key":"pone.0323478.ref018","doi-asserted-by":"crossref","first-page":"10656","DOI":"10.1109\/TVT.2020.3019699","article-title":"Speed profile tracking by an adaptive controller for subway train based on neural network and PID algorithm","volume":"69","author":"Q Pu","year":"2020","journal-title":"IEEE Trans Veh Technol"},{"issue":"6","key":"pone.0323478.ref019","doi-asserted-by":"crossref","first-page":"2346","DOI":"10.1109\/TCYB.2019.2890974","article-title":"Online reinforcement learning control for the personalization of a robotic knee prosthesis","volume":"50","author":"Y Wen","year":"2020","journal-title":"IEEE Trans Cybern"},{"issue":"12","key":"pone.0323478.ref020","doi-asserted-by":"crossref","first-page":"7589","DOI":"10.1109\/TNNLS.2021.3085767","article-title":"Robust adaptive control for a small unmanned helicopter using reinforcement learning","volume":"33","author":"B Xian","year":"2022","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"issue":"6","key":"pone.0323478.ref021","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: A brief survey","volume":"34","author":"K Arulkumaran","year":"2017","journal-title":"IEEE Signal Process Mag"},{"key":"pone.0323478.ref022","article-title":"Continuous control with deep reinforcement learning","author":"TP Lillicrap","year":"2015","journal-title":"arXiv preprint arXiv:1509.02971"},{"issue":"4","key":"pone.0323478.ref023","doi-asserted-by":"crossref","first-page":"1116","DOI":"10.1109\/TITS.2011.2143409","article-title":"Computationally inexpensive tracking control of high-speed trains with traction\/braking saturation","volume":"12","author":"Q Song","year":"2011","journal-title":"IEEE Trans Intell Transp Syst"},{"issue":"8","key":"pone.0323478.ref024","doi-asserted-by":"crossref","first-page":"8300","DOI":"10.3390\/en8088300","article-title":"Learning agent for a heat-pump thermostat with a set-back strategy using model-free reinforcement learning","volume":"8","author":"F Ruelens","year":"2015","journal-title":"Energies"},{"issue":"9","key":"pone.0323478.ref025","first-page":"1441","article-title":"Adaptive backstepping control of train systems with traction\/braking dynamics and uncertain resistive forces","volume":"49","author":"Q Song","year":"2011","journal-title":"Energies"},{"issue":"4","key":"pone.0323478.ref026","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/S0167-9236(97)00047-X","article-title":"A survey of tools for the validation and verification of knowledge - based systems: 1985\u20131995","volume":"21","author":"S Murrell","year":"1997","journal-title":"Decis Support Syst"},{"issue":"4","key":"pone.0323478.ref027","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1109\/THMS.2019.2912447","article-title":"Human-centered reinforcement learning: A survey","volume":"49","author":"G Li","year":"2019","journal-title":"IEEE Trans Human-Mach Syst"},{"issue":"7","key":"pone.0323478.ref028","doi-asserted-by":"crossref","first-page":"4600","DOI":"10.1109\/TSMC.2021.3098451","article-title":"Proximal policy optimization with policy feedback","volume":"52","author":"Y Gu","year":"2022","journal-title":"IEEE Trans Syst Man Cybern, Syst"},{"issue":"12","key":"pone.0323478.ref029","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0261436","article-title":"Rules of incidental operation risk propagation in metro networks under fully automatic operations mode","volume":"16","author":"W Chen","year":"2021","journal-title":"PLoS One"},{"issue":"3","key":"pone.0323478.ref030","doi-asserted-by":"crossref","first-page":"1526","DOI":"10.1109\/TITS.2013.2265171","article-title":"Online learning algorithms for train automatic stop control using precise location data of balises","volume":"14","author":"D Chen","year":"2013","journal-title":"IEEE Trans Intell Transp Syst"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0323478","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T20:24:13Z","timestamp":1747859053000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0323478"}},"subtitle":[],"editor":[{"given":"Qing-Chang","family":"Lu","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2025,5,21]]},"references-count":30,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5,21]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0323478","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,21]]}}}