{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T13:56:43Z","timestamp":1787061403578,"version":"build-2736575974"},"reference-count":47,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2026,5,28]],"date-time":"2026-05-28T00:00:00Z","timestamp":1779926400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Automated valet parking requires reliable long-range slot searching and precise low-speed docking in confined structured lots. This paper proposes a hierarchical cruise-to-park framework that combines nonlinear model predictive control (NMPC) for predefined-route cruising with a Twin Delayed Deep Deterministic Policy Gradient (TD3) agent for terminal parking. The system is implemented in a structured Simulink environment with Unreal Engine-based geometry-aware sensing modules. During cruising, a camera-based module detects available slots and triggers the transition to parking. The NMPC uses a custom cost function to improve tracking on curved approaches, while the TD3 policy uses LiDAR feedback and reward shaping with an explicit time penalty to encourage efficient, stable docking. Simulation results demonstrate smooth phase transition, accurate cruising, and effective terminal parking in the training slot. Validation on six previously unseen target slots within the same parking-lot environment shows encouraging intra-lot target-slot transferability without retraining. Additional PPO and SAC comparisons and a time-penalty ablation further evaluate the relative learning performance and the effect of reward design, supporting the proposed architecture as a practical baseline for integrated cruise-to-park automated valet parking studies.<\/jats:p>","DOI":"10.3390\/s26113409","type":"journal-article","created":{"date-parts":[[2026,5,28]],"date-time":"2026-05-28T09:57:43Z","timestamp":1779962263000},"page":"3409","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A Hierarchical NMPC and TD3-Based Framework for Seamless Cruise-to-Park Automated Valet Parking"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-1473-3656","authenticated-orcid":false,"given":"Dajie","family":"Tian","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH 43210, USA"},{"name":"Automated Driving Lab, Department of Mechanical and Aerospace Engineering, The Ohio State University, Columbus, OH 43212, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8823-1820","authenticated-orcid":false,"given":"Levent","family":"Guvenc","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH 43210, USA"},{"name":"Automated Driving Lab, Department of Mechanical and Aerospace Engineering, The Ohio State University, Columbus, OH 43212, USA"},{"name":"Department of Mechanical and Aerospace Engineering, The Ohio State University, Columbus, OH 43210, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,5,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"102935","DOI":"10.1016\/j.jnca.2020.102935","article-title":"From Smart Parking towards Autonomous Valet Parking: A Survey, Challenges and Future Works","volume":"175","author":"Khalid","year":"2021","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/TIV.2016.2578706","article-title":"A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles","volume":"1","author":"Paden","year":"2016","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3388","DOI":"10.1109\/TITS.2017.2687047","article-title":"Parking Like a Human: A Direct Trajectory Planning Solution","volume":"18","author":"Liu","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4909","DOI":"10.1109\/TITS.2021.3054625","article-title":"Deep Reinforcement Learning for Autonomous Driving: A Survey","volume":"23","author":"Kiran","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1109\/TITS.2019.2891665","article-title":"Re-Plannable Automated Parking System with a Standalone Around View Monitor for Narrow Parking Lots","volume":"21","author":"Jang","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"103606","DOI":"10.1016\/j.robot.2020.103606","article-title":"A Novel Path Planning Methodology for Automated Valet Parking Based on Directional Graph Search and Geometry Curve","volume":"132","author":"Qin","year":"2020","journal-title":"Robot. Auton. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shi, J., Li, K., Piao, C., Gao, J., and Chen, L. (2023). Model-Based Predictive Control and Reinforcement Learning for Planning Vehicle-Parking Trajectories for Vertical Parking Spaces. Sensors, 23.","DOI":"10.3390\/s23167124"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Song, S., Chen, H., Sun, H., and Liu, M. (2020). Data Efficient Reinforcement Learning for Integrated Lateral Planning and Control in Automated Parking System. Sensors, 20.","DOI":"10.3390\/s20247297"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"11970","DOI":"10.1109\/TITS.2021.3109011","article-title":"Optimization-Based Trajectory Planning for Autonomous Parking with Irregularly Placed Obstacles: A Lightweight Iterative Framework","volume":"23","author":"Li","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ren, H., Niu, Y., Li, Y., Yang, L., and Gao, H. (2025). Automatic Parking Trajectory Planning Based on Warm Start Nonlinear Dynamic Optimization. Sensors, 25.","DOI":"10.3390\/s25010112"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1109\/TIV.2022.3156429","article-title":"Online Trajectory Replanning for Sudden Environmental Changes During Automated Parking: A Parallel Stitching Method","volume":"7","author":"Li","year":"2022","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1109\/JAS.2023.123975","article-title":"Path Planning and Tracking Control for Parking via Soft Actor-Critic Under Non-Ideal Scenarios","volume":"11","author":"Tang","year":"2024","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2604","DOI":"10.1109\/TIV.2022.3167616","article-title":"Deep Reinforcement Learning with NMPC Assistance Nash Switching for Urban Autonomous Driving","volume":"8","author":"Alighanbari","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1109\/TIV.2023.3329785","article-title":"Event-Triggered Model Predictive Control with Deep Reinforcement Learning for Autonomous Driving","volume":"9","author":"Dang","year":"2024","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"18547","DOI":"10.1109\/TITS.2024.3436045","article-title":"Efficient Optimization-Based Trajectory Planning for Unmanned Systems in Confined Environments","volume":"25","author":"Fan","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.1109\/TCST.2024.3367468","article-title":"Hierarchical Planning for Autonomous Parking in Dynamic Environments","volume":"32","author":"Wang","year":"2024","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3723","DOI":"10.1109\/TIV.2023.3268088","article-title":"Trajectory Planning for Autonomous Valet Parking in Narrow Environments with Enhanced Hybrid A* Search and Nonlinear Optimization","volume":"8","author":"Lian","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Jia, M., Tao, M., Xu, M., Zhang, P., Qiu, J., Bergsieker, G., and Chen, J. (2024). RL-MPC: Reinforcement Learning Aided Model Predictive Controller for Autonomous Vehicle Lateral Control. WCX SAE World Congress Experience, SAE International. SAE Technical Paper.","DOI":"10.4271\/2024-01-2565"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1016\/j.ifacol.2024.09.056","article-title":"Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic Programming","volume":"58","author":"Bertsekas","year":"2024","journal-title":"IFAC-PapersOnLine"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4570","DOI":"10.1109\/TITS.2020.3046039","article-title":"End-to-End Trainable One-Stage Parking Slot Detection Integrating Global and Local Information","volume":"23","author":"Suhr","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_21","unstructured":"Fujimoto, S., van Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing Function Approximation Error in Actor-Critic Methods. Proceedings of the 35th International Conference on Machine Learning (ICML 2018), Stockholm, Sweden. Available online: https:\/\/proceedings.mlr.press\/v80\/fujimoto18a.html."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jeng, S.-L., and Chiang, C. (2023). End-to-End Autonomous Navigation Based on Deep Reinforcement Learning with a Survival Penalty Function. Sensors, 23.","DOI":"10.3390\/s23208651"},{"key":"ref_23","unstructured":"MathWorks Train PPO Agent for Automatic Parking Valet; MATLAB & Simulink Documentation, The MathWorks, Inc.. Available online: https:\/\/www.mathworks.com\/help\/reinforcement-learning\/ug\/train-ppo-agent-for-automatic-parking-valet.html."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kong, J., Pfeiffer, M., Schildbach, G., and Borrelli, F. (July, January 28). Kinematic and Dynamic Vehicle Models for Autonomous Driving Control Design. Proceedings of the 2015 IEEE Intelligent Vehicles Symposium (IV), Seoul, Republic of Korea.","DOI":"10.1109\/IVS.2015.7225830"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"789","DOI":"10.1016\/S0005-1098(99)00214-9","article-title":"Constrained Model Predictive Control: Stability and Optimality","volume":"36","author":"Mayne","year":"2000","journal-title":"Automatica"},{"key":"ref_26","unstructured":"MathWorks Configure Optimization Solver for Nonlinear MPC; MATLAB & Simulink Documentation, The MathWorks, Inc.. Available online: https:\/\/www.mathworks.com\/help\/mpc\/ug\/configure-optimization-solver-for-nonlinear-mpc.html."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"e12614","DOI":"10.1049\/itr2.12614","article-title":"A Reinforcement Learning-Based Reverse-Parking System for Autonomous Vehicles","volume":"19","author":"Arrabi","year":"2025","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.1007\/s11431-022-2273-5","article-title":"Deep Reinforcement Learning-Based Drift Parking Control of Automated Vehicles","volume":"66","author":"Leng","year":"2023","journal-title":"Sci. China Technol. Sci."},{"key":"ref_29","unstructured":"Ng, A.Y., Harada, D., and Russell, S.J. (1999). Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping. Proceedings of the Sixteenth International Conference on Machine Learning (ICML 1999), Morgan Kaufmann."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"103829","DOI":"10.1016\/j.artint.2022.103829","article-title":"Reward (Mis)design for Autonomous Driving","volume":"316","author":"Knox","year":"2023","journal-title":"Artif. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1109\/TITS.2017.2756099","article-title":"Hierarchical Trajectory Planning of an Autonomous Car Based on the Integration of a Sampling and an Optimization Method","volume":"19","author":"Lim","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1633","DOI":"10.1109\/TASE.2022.3183610","article-title":"Deep Learning-Based Trajectory Planning and Control for Autonomous Ground Vehicle Parking Maneuver","volume":"20","author":"Chai","year":"2023","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"154485","DOI":"10.1109\/ACCESS.2020.3017770","article-title":"Reinforcement Learning-Based Motion Planning for Automatic Parking System","volume":"8","author":"Zhang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Polack, P., Altch\u00e9, F., d\u2019Andr\u00e9a-Novel, B., and de La Fortelle, A. (2017, January 11\u201314). The kinematic bicycle model: A consistent model for planning feasible trajectories for autonomous vehicles?. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995816"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sch\u00f6rner, P., Conzelmann, M., Fleck, T., Zofka, M.R., and Z\u00f6llner, J.M. (2021, January 19\u201322). Park my Car! Automated Valet Parking with Different Vehicle Automation Levels by V2X Connected Smart Infrastructure. Proceedings of the 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), Indianapolis, IN, USA.","DOI":"10.1109\/ITSC48978.2021.9565095"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"109751","DOI":"10.1016\/j.isci.2024.109751","article-title":"Toward autonomous vehicles: A survey on cooperative vehicle-infrastructure system","volume":"27","author":"Ji","year":"2024","journal-title":"iScience"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1777","DOI":"10.1109\/TMC.2025.3608179","article-title":"Online Location Planning for AI-Defined Vehicles: Optimizing Joint Tasks of Order Serving and Spatio-Temporal Heterogeneous Model Fine-Tuning","volume":"25","author":"Zheng","year":"2026","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Wang, Z., Han, K., and Tiwari, P. (2021). Motion Estimation of Connected and Automated Vehicles Under Communication Delay and Packet Loss of V2X Communications, SAE International. SAE Technical Paper.","DOI":"10.4271\/2021-01-0107"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1146\/annurev-control-071723-102940","article-title":"The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems","volume":"7","author":"Hsu","year":"2024","journal-title":"Annu. Rev. Control Robot. Auton. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Guvenc, L., Aksun-Guvenc, B., Zhu, S., and Gelbal, S.Y. (2022). Autonomous Road Vehicle Path Planning and Tracking Control, IEEE Press.","DOI":"10.1002\/9781119747970"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Cao, X., Chen, H., Gelbal, S.Y., Aksun-Guvenc, B., and Guvenc, L. (2023). Vehicle-in-Virtual-Environment (VVE) Method for Autonomous Driving System Development, Evaluation and Demonstration. Sensors, 23.","DOI":"10.3390\/s23115088"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"53","DOI":"10.4271\/12-01-02-0003","article-title":"Localization and Perception for Control and Decision-Making of a Low-Speed Autonomous Shuttle in a Campus Pilot Deployment","volume":"1","author":"Wen","year":"2018","journal-title":"SAE Int. J. Connect. Autom. Veh."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1074","DOI":"10.1109\/TNANO.2011.2106797","article-title":"Robust repetitive controller for fast AFM imaging","volume":"10","author":"Necipoglu","year":"2011","journal-title":"IEEE Trans. Nanotechnol."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Guvenc, L., Guvenc, B.A., Demirel, B., and Emirler, M.T. (2017). Control of Mechatronic Systems, The Institution of Engineering and Technology.","DOI":"10.1049\/PBCE104E"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"46874","DOI":"10.1109\/ACCESS.2021.3066925","article-title":"Multi-objective digital PID controller design in parameter space and its application to automated path following","volume":"9","author":"Wang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Zhu, S., Wang, J., Yang, Y., and Aksun-Guvenc, B. (2024). Stability of Local Trajectory Planning for Level-2+ Semi-Autonomous Driving without Absolute Localization. Electronics, 13.","DOI":"10.3390\/electronics13193808"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1007\/s10846-020-01215-y","article-title":"Trajectory planning of autonomous vehicles based on parameterized control optimization in dynamic on-road environments","volume":"100","author":"Zhu","year":"2020","journal-title":"J. Intell. Robot. Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/11\/3409\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,28]],"date-time":"2026-05-28T09:59:30Z","timestamp":1779962370000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/11\/3409"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,28]]},"references-count":47,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["s26113409"],"URL":"https:\/\/doi.org\/10.3390\/s26113409","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,28]]}}}