{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T15:32:24Z","timestamp":1783611144949,"version":"3.55.0"},"reference-count":36,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T00:00:00Z","timestamp":1691712000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The National Key Research and Development Program of China","award":["2022YFE0101000"],"award-info":[{"award-number":["2022YFE0101000"]}]},{"name":"The National Key Research and Development Program of China","award":["2022CQBSHTB2010"],"award-info":[{"award-number":["2022CQBSHTB2010"]}]},{"name":"The National Key Research and Development Program of China","award":["22XJZXZD05"],"award-info":[{"award-number":["22XJZXZD05"]}]},{"name":"Chongqing Postdoctoral Research Special Funding Project","award":["2022YFE0101000"],"award-info":[{"award-number":["2022YFE0101000"]}]},{"name":"Chongqing Postdoctoral Research Special Funding Project","award":["2022CQBSHTB2010"],"award-info":[{"award-number":["2022CQBSHTB2010"]}]},{"name":"Chongqing Postdoctoral Research Special Funding Project","award":["22XJZXZD05"],"award-info":[{"award-number":["22XJZXZD05"]}]},{"DOI":"10.13039\/501100004502","name":"school-level research projects","doi-asserted-by":"publisher","award":["2022YFE0101000"],"award-info":[{"award-number":["2022YFE0101000"]}],"id":[{"id":"10.13039\/501100004502","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004502","name":"school-level research projects","doi-asserted-by":"publisher","award":["2022CQBSHTB2010"],"award-info":[{"award-number":["2022CQBSHTB2010"]}],"id":[{"id":"10.13039\/501100004502","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004502","name":"school-level research projects","doi-asserted-by":"publisher","award":["22XJZXZD05"],"award-info":[{"award-number":["22XJZXZD05"]}],"id":[{"id":"10.13039\/501100004502","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper proposes a vehicle-parking trajectory planning method that addresses the issues of a long trajectory planning time and difficult training convergence during automatic parking. The process involves two stages: finding a parking space and parking planning. The first stage uses model predictive control (MPC) for trajectory tracking from the initial position of the vehicle to the starting point of the parking operation. The second stage employs the proximal policy optimization (PPO) algorithm to transform the parking behavior into a reinforcement learning process. A four-dimensional reward function is set to evaluate the strategy based on a formal reward, guiding the adjustment of neural network parameters and reducing the exploration of invalid actions. Finally, a simulation environment is built for the parking scene, and a network framework is designed. The proposed method is compared with the deep deterministic policy gradient and double-delay deep deterministic policy gradient algorithms in the same scene. Results confirm that the MPC controller accurately performs trajectory-tracking control with minimal steering wheel angle changes and smooth, continuous movement. The PPO-based reinforcement learning method achieves shorter learning times, totaling only 30% and 37.5% of the deep deterministic policy gradient (DDPG) and twin-delayed deep deterministic policy gradient (TD3), and the number of iterations to reach convergence for the PPO algorithm with the introduction of the four-dimensional evaluation metrics is 75% and 68% shorter compared to the DDPG and TD3 algorithms, respectively. This study demonstrates the effectiveness of the proposed method in addressing a slow convergence and long training times in parking trajectory planning, improving parking timeliness.<\/jats:p>","DOI":"10.3390\/s23167124","type":"journal-article","created":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T12:10:23Z","timestamp":1691755823000},"page":"7124","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Model-Based Predictive Control and Reinforcement Learning for Planning Vehicle-Parking Trajectories for Vertical Parking Spaces"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2803-9439","authenticated-orcid":false,"given":"Junren","family":"Shi","sequence":"first","affiliation":[{"name":"School of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-7685-8102","authenticated-orcid":false,"given":"Kexin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Changhao","family":"Piao","sequence":"additional","affiliation":[{"name":"School of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lizhi","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"108563","DOI":"10.1016\/j.automatica.2019.108563","article-title":"Decentralized optimal control of connected automated vehicles at signal-free intersections including comfort-constrained turns and safety guarantees","volume":"109","author":"Yue","year":"2019","journal-title":"Automatica"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"58443","DOI":"10.1109\/ACCESS.2020.2983149","article-title":"A survey of autonomous driving: Common practices and emerging technologies","volume":"8","author":"Yurtsever","year":"2020","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pendleton, S.D., Andersen, H., Dux, X., Shen, X., Meghjani, M., Eng, Y.H., Rus, D., and Ang, M.H. (2017). Perception, planning, control, and coordination for autonomous vehicles. Machines, 5.","DOI":"10.3390\/machines5010006"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1826","DOI":"10.1109\/TITS.2019.2913998","article-title":"A review of motion planning for highway autonomous driving","volume":"21","author":"Claussmann","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1146\/annurev-control-060117-105157","article-title":"Planning and decision-making for autonomous vehicles","volume":"1","author":"Schwarting","year":"2018","journal-title":"Annu. Rev. Control Robot. Auton. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"102142","DOI":"10.1016\/j.ijinfomgt.2020.102142","article-title":"On the training of a neural network for online path planning with offline path planning algorithms","volume":"57","author":"Sung","year":"2021","journal-title":"Int. J. Inf. Manag."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"102820","DOI":"10.1016\/j.scs.2021.102820","article-title":"Intelligent charge scheduling and eco-routing mechanism for electric vehicles: A multi-objective heuristic approach","volume":"69","author":"Chakraborty","year":"2021","journal-title":"Sustain. Cities Soc."},{"key":"ref_8","unstructured":"Ngo, T.G., Dao, T.K., Thandapani, J., Nguyen, T.T., Pham, D.T., and Vu, V.D. (2021). Communication and Intelligent Systems, Springer."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Arulkumaran, K., Deisenroth, M.P., Brundage, M., and Bharath, A.A. (2017). A brief survey of deep reinforcement learning. arXiv.","DOI":"10.1109\/MSP.2017.2743240"},{"key":"ref_10","unstructured":"Li, Y. (2017). Deep reinforcement learning: An overview. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhang, P., Xiong, L., Yu, Z., Fang, P., Yan, S., Yao, J., and Zhou, Y. (2019). Reinforcement learning-based end-to-end parking for automatic parking system. Sensors, 19.","DOI":"10.3390\/s19183996"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Thunyapoo, B., Ratchadakorntham, C., Siricharoen, P., and Susutti, W. (2020, January 24\u201327). Self-Parking car simulation using reinforcement learning approach for moderate complexity parking scenario. Proceedings of the 2020 17th International Conference on Electrical Engineering\/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), Phuket, Thailand.","DOI":"10.1109\/ECTI-CON49241.2020.9158298"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Bejar, E., and Morn, A. (2019, January 7\u20139). Reverse parking a car-like mobile robot with deep reinforcement learning and preview control. Proceedings of the 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.","DOI":"10.1109\/CCWC.2019.8666613"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1007\/s12239-020-0085-9","article-title":"Trajectory planning for automated parking systems using deep reinforcement learning","volume":"21","author":"Du","year":"2020","journal-title":"Int. J. Automot. Technol."},{"key":"ref_15","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 19\u201324). Asynchronous Methods for Deep Reinforcement Learning. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1109\/TAC.2019.2921659","article-title":"Polling-systems-based Autonomous Vehicle Coordination in Traffic Intersections with No Traffic Signals","volume":"65","author":"Miculescu","year":"2016","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chen, X., Ma, H., Wan, J., Li, B., and Xia, T. (2017, January 21\u201326). Multi-view 3D object detection network for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.691"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"100057","DOI":"10.1016\/j.array.2021.100057","article-title":"Deep Learning for Object Detection and Scene Perception in Self-Driving Cars: Survey, Challenges, and Open Issues","volume":"10","author":"Gupta","year":"2021","journal-title":"Array"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009, January 14\u201318). Curriculum learning. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.","DOI":"10.1145\/1553374.1553380"},{"key":"ref_21","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_22","unstructured":"Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 21\u201326). Deterministic Policy Gradient Algorithms. Proceedings of the 31st International Conference on Machine Learning, Beijing, China."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"18","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"17298814211042730","DOI":"10.1177\/17298814211042730","article-title":"Autonomous land vehicle path planning algorithm based on improved heuristic function of A-Star","volume":"18","author":"Zhang","year":"2021","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Boroujeni, Z., Goehring, D., Ulbrich, F., Neumann, D., and Rojas, R. (2017, January 27\u201328). Flexible unit A-star trajectory planning for autonomous vehicles on structured road maps. Proceedings of the 2017 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Vienna, Austria.","DOI":"10.1109\/ICVES.2017.7991893"},{"key":"ref_26","unstructured":"Gurenko, B.V., and Vasileva, M.A. (2021). International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, Springer."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5355","DOI":"10.1109\/LRA.2020.3005126","article-title":"Efficient sampling-based maximum entropy inverse reinforcement learning with application to autonomous driving","volume":"5","author":"Wu","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2655","DOI":"10.1109\/ACCESS.2020.3047385","article-title":"An adaptive motion planning technique for on-road autonomous driving","volume":"9","author":"Jin","year":"2020","journal-title":"IEEE Access"},{"key":"ref_29","first-page":"5910503","article-title":"Research on intelligent vehicle path planning based on rapidly-exploring random tree","volume":"2020","author":"Shi","year":"2020","journal-title":"Math. Probl. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1030","DOI":"10.1109\/TASE.2021.3050762","article-title":"R2-RRT*: Reliability-based robust mission planning of offroad autonomous ground vehicle under uncertain terrain environment","volume":"19","author":"Jiang","year":"2021","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"8269698","DOI":"10.1155\/2018\/8269698","article-title":"An overview of nature-inspired, conventional, and hybrid methods of autonomous vehicle path planning","volume":"2018","author":"Ayawli","year":"2018","journal-title":"J. Adv. Transp."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"104211","DOI":"10.1016\/j.engappai.2021.104211","article-title":"Recent advances in motion and behavior planning techniques for software architecture of autonomous vehicles: A state-of-the-art survey","volume":"101","author":"Sharma","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_33","first-page":"2943","article-title":"Eco-driving at signalized intersections: A multiple signal optimization approach","volume":"22","author":"Hao","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/j.trc.2019.01.026","article-title":"Urban traffic signal control with connected and automated vehicles: A survey","volume":"101","author":"Qiangqiang","year":"2019","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1016\/j.trc.2018.10.008","article-title":"Unravelling effects of cooperative adaptive cruise control deactivation on traffic flow characteristics at merging bottlenecks","volume":"96","author":"Xiao","year":"2018","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4490","DOI":"10.1109\/TITS.2020.3045123","article-title":"Cooperative ramp merging design and field implementation: A digital twin approach based on vehicle-to-cloud communication","volume":"23","author":"Liao","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/16\/7124\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:31:44Z","timestamp":1760128304000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/16\/7124"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,11]]},"references-count":36,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["s23167124"],"URL":"https:\/\/doi.org\/10.3390\/s23167124","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,11]]}}}