{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:12:49Z","timestamp":1760058769350,"version":"build-2065373602"},"reference-count":34,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T00:00:00Z","timestamp":1745884800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"publisher","award":["62003011","2021B1203"],"award-info":[{"award-number":["62003011","2021B1203"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Open Research Project of Big Data Application Technologies Laboratory, China Academy of Transportation Sciences","award":["62003011","2021B1203"],"award-info":[{"award-number":["62003011","2021B1203"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Efficient ship scheduling in inland waterways is critical for maritime transportation safety and economic viability. However, traditional scheduling methods, primarily based on First Come First Served (FCFS) principles, often produce suboptimal results due to their inability to account for complex spatial\u2013temporal dependencies, directional asymmetries, and varying ship characteristics. This paper introduces SSRL (Ship Scheduling through Reinforcement Learning), a novel framework that addresses these limitations by integrating three complementary components: (1) a Q-learning framework that discovers optimal scheduling policies through environmental interaction rather than predefined rules; (2) a clustering mechanism that reduces the high-dimensional state space by grouping similar ship states; and (3) a sliding window approach that decomposes the scheduling problem into manageable subproblems, enabling real-time decision-making. We evaluated SSRL through extensive experiments using both simulated scenarios and real-world data from the Xiaziliang Restricted Waterway in China. Results demonstrate that SSRL reduces total ship waiting time by 90.6% compared with TSRS, 48.4% compared with FAHP-ES, and 32.6% compared with OSS-SW, with an average reduction of 57.2% across these baseline methods. SSRL maintains superior performance across varying traffic densities and uncertainty conditions, with the optimal information window length of 13\u201314 ships providing the best balance between solution quality and computational efficiency. Beyond performance improvements, SSRL offers significant practical advantages: it requires minimal computation for online implementation, adapts to dynamic maritime environments without manual reconfiguration, and can potentially be extended to other complex transportation scheduling domains.<\/jats:p>","DOI":"10.3390\/sym17050679","type":"journal-article","created":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T11:35:13Z","timestamp":1746185713000},"page":"679","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["SSRL: A Clustering-Based Reinforcement Learning Approach for Efficient Ship Scheduling in Inland Waterways"],"prefix":"10.3390","volume":"17","author":[{"given":"Shaojun","family":"Gan","sequence":"first","affiliation":[{"name":"School of Metropolitan Transportation, Beijing University of Technology, Beijing 100124, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Metropolitan Transportation, Beijing University of Technology, Beijing 100124, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongdun","family":"Li","sequence":"additional","affiliation":[{"name":"China Academy of Transportation Sciences, Beijing 100029, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1016\/j.ejor.2021.04.026","article-title":"Vessel velocity decisions in inland waterway transportation under uncertainty","volume":"296","author":"Buchem","year":"2022","journal-title":"Eur. J. Oper. Res."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhang, J., Wan, C., He, A., Zhang, D., and Soares, C.G. (2021). A two-stage black-spot identification model for inland waterway transportation. Reliab. Eng. Syst. Saf., 213.","DOI":"10.1016\/j.ress.2021.107677"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Yang, W., Liao, P., Jiang, S., and Wang, H. (2024). Analysis of vessel traffic flow characteristics in inland restricted waterways using multi-source data. arXiv.","DOI":"10.1016\/j.oceaneng.2024.120065"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, D., Shi, G., and Kang, Z. (2021). Fuzzy scheduling problem of vessels in one-way waterway. J. Mar. Sci. Eng., 9.","DOI":"10.3390\/jmse9101064"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.oceaneng.2018.11.025","article-title":"Vessel traffic scheduling method for the controlled waterways in the upper Yangtze River","volume":"172","author":"Liang","year":"2019","journal-title":"Ocean Eng."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Gan, S., Liang, S., Li, K., Deng, J., and Cheng, T. (September, January 31). Ship trajectory prediction for intelligent traffic management using clustering and ANN. Proceedings of the 2016 UKACC 11th International Conference on Control (CONTROL), Belfast, UK.","DOI":"10.1109\/CONTROL.2016.7737569"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Xia, Z., Guo, Z., Wang, W., and Jiang, Y. (2021). Joint optimization of ship scheduling and speed reduction: A new strategy considering high transport efficiency and low carbon of ships in port. Ocean Eng., 233.","DOI":"10.1016\/j.oceaneng.2021.109224"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"132893","DOI":"10.1109\/ACCESS.2020.3009775","article-title":"A Co-Scheduling Problem of Ship Lift and Ship Lock at the Three Gorges Dam","volume":"8","author":"Zhao","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3221","DOI":"10.1002\/ldr.2984","article-title":"Tracking geomorphic recovery in process-based river management","volume":"29","author":"Fryirs","year":"2018","journal-title":"Land Degrad. Dev."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Lagos, M.S., Mu\u00f1oz, J.F., Su\u00e1rez, F.I., Fuenzalida, M.J., Y\u00e1\u00f1ez-Morroni, G., and Sanzana, P. (2024). Investigating the effects of channelization in the Silala River: A review of the implementation of a coupled MIKE-11 and MIKE-SHE modeling system. Wiley Interdiscip. Rev. Water, 11.","DOI":"10.1002\/wat2.1673"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Liu, S., Zheng, Q., Tian, H., and Guo, W. (2024). Ship scheduling problem in an anchorage-to-quay channel with water discharge restrictions. Ocean Eng., 309.","DOI":"10.1016\/j.oceaneng.2024.118432"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhai, D., Fu, X., Xu, H.Y., Yin, X.F., Vasundhara, J., and Zhang, W. (2020, January 9\u201313). Multi-Layer Scheduling Optimization for Intelligent Mobility of Maritime Operation. Proceedings of the 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA), Kristiansand, Norway.","DOI":"10.1109\/ICIEA48937.2020.9248206"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chen, C., and Chen, X. (2017, January 8\u201310). Scheduling optimization in restricted channels based on the agent technology and bayesian network. Proceedings of the 2017 4th International Conference on Transportation Information and Safety (ICTIS), Banff, AB, Canada.","DOI":"10.1109\/ICTIS.2017.8047779"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1082","DOI":"10.1016\/j.ejor.2019.08.002","article-title":"Optimising cargo loading and ship scheduling in tidal areas","volume":"280","author":"Ferson","year":"2020","journal-title":"Eur. J. Oper. Res."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, X., Li, R., Wang, C., Xue, B., and Guo, W. (2024). Robust optimization for a class of ship traffic scheduling problem with uncertain arrival and departure times. Eng. Appl. Artif. Intell., 133.","DOI":"10.1016\/j.engappai.2024.108257"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Eisen, H.E., Van der Lei, J.E., Zuidema, J., Koch, T., and Dugundji, E.R. (2021). An Evaluation of First-Come, First-Served Scheduling in a Geometrically-Constrained Wet Bulk Terminal. Front. Future Transp., 2.","DOI":"10.3389\/ffutr.2021.709822"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Gan, S., Wang, Y., Li, K., and Liang, S. (2021). Efficient online one-way traffic scheduling for restricted waterways. Ocean Eng., 237.","DOI":"10.1016\/j.oceaneng.2021.109515"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/TITS.2016.2560131","article-title":"Long-term ship speed prediction for intelligent traffic signaling","volume":"18","author":"Gan","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, S., Zhang, Y., Guo, W., Tian, H., and Tang, K. (2024). Ship scheduling problem based on channel-lock coordination in flood season. Expert Syst. Appl., 254.","DOI":"10.1016\/j.eswa.2024.124393"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.trd.2016.09.013","article-title":"The waterway ship scheduling problem","volume":"60","author":"Shi","year":"2018","journal-title":"Transp. Res. Part D Transp. Environ."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yang, X., Gu, W., and Wang, S. (2023). Optimal scheduling of vessels passing a waterway bottleneck. Ocean Coast. Manag., 244.","DOI":"10.1016\/j.ocecoaman.2023.106809"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Aritua, B., Cheng, L., van Liere, R., and de Leijer, H. (2021). Blue Routes for a New Era: Developing Inland Waterways Transportation in China, World Bank Publications.","DOI":"10.1596\/978-1-4648-1584-3"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Jian, L., Xing, Y., Ke-zhong, L., and Zhi-tao, Y. (2015, January 25\u201328). Study on the fluency of one-way waterway transportation based on First Come First Served (FCFS) model. Proceedings of the 2015 International Conference on Transportation Information and Safety (ICTIS), Wuhan, China.","DOI":"10.1109\/ICTIS.2015.7232137"},{"key":"ref_24","first-page":"37","article-title":"Simulation on the traffic capacity of the Three Gorges ship lock based on SIVAK","volume":"41","author":"Liu","year":"2015","journal-title":"J. Dalian Marit. Univ."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"83","DOI":"10.4031\/MTSJ.53.1.9","article-title":"A Self-Organizing Grouping Approach for Ship Traffic Scheduling in Restricted One-Way Waterway","volume":"53","author":"Xin","year":"2019","journal-title":"Mar. Technol. Soc. J."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, R., Zhang, X., Jiang, L., Yang, Z., and Guo, W. (2022). An adaptive heuristic algorithm based on reinforcement learning for ship scheduling optimization problem. Ocean Coast. Manag., 230.","DOI":"10.1016\/j.ocecoaman.2022.106375"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"17547","DOI":"10.1109\/TITS.2024.3431199","article-title":"Deep Reinforcement Learning for Channel Traffic Scheduling in Dry Bulk Export Terminals","volume":"25","author":"Wang","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"4909","DOI":"10.1109\/TITS.2021.3054625","article-title":"Deep reinforcement learning for autonomous driving: A survey","volume":"23","author":"Kiran","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/TITS.2020.3008612","article-title":"Deep reinforcement learning for intelligent transportation systems: A survey","volume":"23","author":"Haydari","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_30","unstructured":"Fan, J., Wang, Z., Xie, Y., and Yang, Z. (2020, January 13\u201318). A theoretical analysis of deep Q-learning. Proceedings of the Learning for Dynamics and Control, PMLR, Virtual."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1170","DOI":"10.1016\/j.ins.2019.10.035","article-title":"A scheduling scheme in the cloud computing environment using deep Q-learning","volume":"512","author":"Tong","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"739","DOI":"10.1109\/TSC.2018.2867482","article-title":"A double deep Q-learning model for energy-efficient edge scheduling","volume":"12","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Serv. Comput."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"3027","DOI":"10.1109\/TFUZZ.2018.2796074","article-title":"Significantly fast and robust fuzzy c-means clustering algorithm based on morphological reconstruction and membership filtering","volume":"26","author":"Lei","year":"2018","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1109\/MCS.2011.940571","article-title":"Receding horizon control","volume":"31","author":"Mattingley","year":"2011","journal-title":"IEEE Control Syst."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/5\/679\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:24:14Z","timestamp":1760030654000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/5\/679"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,29]]},"references-count":34,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["sym17050679"],"URL":"https:\/\/doi.org\/10.3390\/sym17050679","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2025,4,29]]}}}