{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T15:00:42Z","timestamp":1783090842345,"version":"3.54.6"},"reference-count":31,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,11,5]],"date-time":"2024-11-05T00:00:00Z","timestamp":1730764800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>As the decarbonization strategies of automated container terminals (ACTs) continue to advance, electrically powered Battery-Automated Guided Vehicles (B-AGVs) are being widely adopted in ACTs. The U-shaped ACT, as a novel layout, faces higher AGV energy consumption due to its deep yard characteristics. A key issue is how to adopt charging strategies suited to varying conditions to reduce the operational capacity loss caused by charging. This paper proposes a simulation-based optimization method for AGV charging strategies in U-shaped ACTs based on an improved Proximal Policy Optimization (PPO) algorithm. Firstly, Gated Recurrent Unit (GRU) structures are incorporated into the PPO to capture temporal correlations in state information. To effectively limit policy update magnitudes in the PPO, we improve the clipping function. Secondly, a simulation model is established by mimicking the operational process of the U-shaped ACTs. Lastly, iterative training of the proposed method is conducted based on the simulation model. The experimental results indicate that the proposed method converges faster than standard PPO and Deep Q-network (DQN). When comparing the proposed method-based charging threshold with a fixed charging threshold strategy across six different scenarios with varying charging rates, the proposed charging strategy demonstrates better adaptability to terminal condition variations in two-thirds of the scenarios.<\/jats:p>","DOI":"10.3390\/systems12110472","type":"journal-article","created":{"date-parts":[[2024,11,5]],"date-time":"2024-11-05T11:36:39Z","timestamp":1730806599000},"page":"472","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Simulation and Optimization of Automated Guided Vehicle Charging Strategy for U-Shaped Automated Container Terminal Based on Improved Proximal Policy Optimization"],"prefix":"10.3390","volume":"12","author":[{"given":"Yongsheng","family":"Yang","sequence":"first","affiliation":[{"name":"Institute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianyi","family":"Liang","sequence":"additional","affiliation":[{"name":"Institute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junkai","family":"Feng","sequence":"additional","affiliation":[{"name":"Institute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 201306, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1057\/s41278-022-00240-y","article-title":"Container Terminal Automation: Revealing Distinctive Terminal Characteristics and Operating Parameters","volume":"24","author":"Knatz","year":"2022","journal-title":"Marit. Econ. Logist."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"108804","DOI":"10.1016\/j.cie.2022.108804","article-title":"Multi-Equipment Coordinated Scheduling Strategy of U-Shaped Automated Container Terminal Considering Energy Consumption","volume":"174","author":"Niu","year":"2022","journal-title":"Comput. Ind. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2437","DOI":"10.1109\/TASE.2021.3085329","article-title":"Performance Estimation and Design Optimization of a Congested Automated Container Terminal","volume":"19","author":"Xiang","year":"2022","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chen, J., Huang, T., Xie, X., Lee, P.T.-W., and Hua, C. (2019). Constructing Governance Framework of a Green and Smart Port. JMSE, 7.","DOI":"10.3390\/jmse7040083"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"106873","DOI":"10.1016\/j.ocecoaman.2023.106873","article-title":"Battery-Powered Automated Guided Vehicles Scheduling Problem in Automated Container Terminals for Minimizing Energy Consumption","volume":"246","author":"Yang","year":"2023","journal-title":"Ocean Coast. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"107258","DOI":"10.1016\/j.cie.2021.107258","article-title":"Modeling and Analysis for an Automated Container Terminal Considering Battery Management","volume":"156","author":"Xiang","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Xiao, S., Huang, J., Hu, H., and Gu, Y. (2024). Automatic Guided Vehicle Scheduling in Automated Container Terminals Based on a Hybrid Mode of Battery Swapping and Charging. JMSE, 12.","DOI":"10.3390\/jmse12020305"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yang, Y., Sun, S., Zhong, M., Feng, J., Wen, F., and Song, H. (2023). A Refined Collaborative Scheduling Method for Multi-Equipment at U-Shaped Automated Container Terminals Based on Rail Crane Process Optimization. J. Mar. Sci. Eng., 11.","DOI":"10.3390\/jmse11030605"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"107695","DOI":"10.1016\/j.cie.2021.107695","article-title":"Integrated Scheduling Optimization of U-Shaped Automated Container Terminal under Loading and Unloading Mode","volume":"162","author":"Xu","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"105517","DOI":"10.1016\/j.cor.2021.105517","article-title":"Scheduling Heterogeneous Multi-Load AGVs with Battery Constraints","volume":"136","author":"Dang","year":"2021","journal-title":"Comput. Oper. Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"855","DOI":"10.1016\/j.ejor.2021.08.008","article-title":"A Matheuristic for AGV Scheduling with Battery Constraints","volume":"298","author":"Singh","year":"2022","journal-title":"Eur. J. Oper. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1016\/j.ejor.2020.07.049","article-title":"Pickup and Delivery Problem with Recharging for Material Handling Systems Utilising Autonomous Mobile Robots","volume":"289","author":"Jun","year":"2021","journal-title":"Eur. J. Oper. Res."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yang, X., Hu, H., Cheng, C., and Wang, Y. (2023). Automated Guided Vehicle (AGV) Scheduling in Automated Container Terminals (ACTs) Focusing on Battery Swapping and Speed Control. JMSE, 11.","DOI":"10.3390\/jmse11101852"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"107014","DOI":"10.1016\/j.ocecoaman.2023.107014","article-title":"Novel AGV Resilient Scheduling for Automated Container Terminals Considering Charging Strategy","volume":"250","author":"Song","year":"2024","journal-title":"Ocean Coast. Manag."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"58","DOI":"10.2507\/IJSIMM16(1)5.368","article-title":"A Fuzzy Hybrid GA-PSO Algorithm for Multi-Objective AGV Scheduling in FMS","volume":"16","author":"Mousavi","year":"2017","journal-title":"Int. J. Simul. Model."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103110","DOI":"10.1016\/j.tre.2023.103110","article-title":"A Two-Stage Stochastic Programming for AGV Scheduling with Random Tasks and Battery Swapping in Automated Container Terminals","volume":"174","author":"Li","year":"2023","journal-title":"Transp. Res. Part E Logist. Transp. Rev."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Abderrahim, M., Bekrar, A., Trentesaux, D., Aissani, N., and Bouamrane, K. (2020). Manufacturing 4.0 Operations Scheduling with AGV Battery Management Constraints. Energies, 13.","DOI":"10.3390\/en13184948"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"102208","DOI":"10.1016\/j.simpat.2020.102208","article-title":"Meta-Model Based Simulation Optimization for Automated Guided Vehicle System under Different Charging Mechanisms","volume":"106","author":"Chen","year":"2021","journal-title":"Simul. Model. Pract. Theory"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1016\/j.cie.2018.01.026","article-title":"Increasing Manufacturing Flexibility through Battery Management of Automated Guided Vehicles","volume":"117","author":"Kabir","year":"2018","journal-title":"Comput. Ind. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"102146","DOI":"10.1016\/j.simpat.2020.102146","article-title":"Simulation Model and Performance Evaluation of Battery-Powered AGV Systems in Automated Container Terminals","volume":"106","author":"Ma","year":"2021","journal-title":"Simul. Model. Pract. Theory"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1080\/00207543.2018.1475761","article-title":"Comparative analysis of different routing heuristics for the battery management of automated guided vehicles","volume":"57","author":"Kabir","year":"2018","journal-title":"Int. J. Prod. Res."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Han, W., Xu, J., Sun, Z., Liu, B., Zhang, K., Zhang, Z., and Mei, X. (2022). Digital Twin-Based Automated Guided Vehicle Scheduling: A Solution for Its Charging Problems. Appl. Sci., 12.","DOI":"10.3390\/app12073354"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Park, J.-S., and Kim, J.-W. (2024). Multi-AGV Scheduling under Limited Buffer Capacity and Battery Charging Using Simulation Techniques. Appl. Sci., 14.","DOI":"10.3390\/app14031197"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5246","DOI":"10.1109\/TSG.2018.2879572","article-title":"Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning","volume":"10","author":"Wan","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1016\/j.aej.2022.12.057","article-title":"Deep Reinforcement Learning Based Optimization of Automated Guided Vehicle Time and Energy Consumption in a Container Terminal","volume":"67","author":"Drungilas","year":"2023","journal-title":"Alex. Eng. J."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"107687","DOI":"10.1016\/j.engappai.2023.107687","article-title":"A Digital Twin-Based Decision Support Approach for AGV Scheduling","volume":"130","author":"Gao","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhang, L., Yan, Y., and Hu, Y. (2023). Deep Reinforcement Learning for Dynamic Scheduling of Energy-Efficient Automated Guided Vehicles. J. Intell. Manuf.","DOI":"10.1007\/s10845-023-02208-y"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"7722","DOI":"10.1080\/00207543.2024.2325583","article-title":"Real-time AGV scheduling optimisation method with deep reinforcement learning for energy-efficiency in the container terminal yard","volume":"62","author":"Gong","year":"2024","journal-title":"Int. J. Prod. Res."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"492","DOI":"10.1016\/j.jmsy.2023.12.008","article-title":"Automated Guided Vehicle Dispatching and Routing Integration via Digital Twin with Deep Reinforcement Learning","volume":"72","author":"Zhang","year":"2024","journal-title":"J. Manuf. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zheng, X., Liang, C., Wang, Y., Shi, J., and Lim, G. (2022). Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning Approach. Mathematics, 10.","DOI":"10.3390\/math10234575"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hu, H., Wang, Y., Tong, W., Zhao, J., and Gu, Y. (2023). Path Planning for Autonomous Vehicles in Unknown Dynamic Environment Based on Deep Reinforcement Learning. Appl. Sci., 13.","DOI":"10.3390\/app131810056"}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/12\/11\/472\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:26:50Z","timestamp":1760113610000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/12\/11\/472"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,5]]},"references-count":31,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["systems12110472"],"URL":"https:\/\/doi.org\/10.3390\/systems12110472","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,5]]}}}