{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T10:27:30Z","timestamp":1784284050686,"version":"3.55.0"},"reference-count":37,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2020,12,18]],"date-time":"2020-12-18T00:00:00Z","timestamp":1608249600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Reinforcement learning (RL) is a promising direction in automated parking systems (APSs), as integrating planning and tracking control using RL can potentially maximize the overall performance. However, commonly used model-free RL requires many interactions to achieve acceptable performance, and model-based RL in APS cannot continuously learn. In this paper, a data-efficient RL method is constructed to learn from data by use of a model-based method. The proposed method uses a truncated Monte Carlo tree search to evaluate parking states and select moves. Two artificial neural networks are trained to provide the search probability of each tree branch and the final reward for each state using self-trained data. The data efficiency is enhanced by weighting exploration with parking trajectory returns, an adaptive exploration scheme, and experience augmentation with imaginary rollouts. Without human demonstrations, a novel training pipeline is also used to train the initial action guidance network and the state value network. Compared with path planning and path-following methods, the proposed integrated method can flexibly co-ordinate the longitudinal and lateral motion to park a smaller parking space in one maneuver. Its adaptability to changes in the vehicle model is verified by joint Carsim and MATLAB simulation, demonstrating that the algorithm converges within a few iterations. Finally, experiments using a real vehicle platform are used to further verify the effectiveness of the proposed method. Compared with obtaining rewards using simulation, the proposed method achieves a better final parking attitude and success rate.<\/jats:p>","DOI":"10.3390\/s20247297","type":"journal-article","created":{"date-parts":[[2020,12,21]],"date-time":"2020-12-21T01:01:08Z","timestamp":1608512468000},"page":"7297","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Data Efficient Reinforcement Learning for Integrated Lateral Planning and Control in Automated Parking System"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7607-3167","authenticated-orcid":false,"given":"Shaoyu","family":"Song","sequence":"first","affiliation":[{"name":"School of Automotive Studies, Tongji University, Shanghai 201804, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hui","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Automotive Studies, Tongji University, Shanghai 201804, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongwei","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Automotive Studies, Tongji University, Shanghai 201804, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meicen","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Automotive Studies, Tongji University, Shanghai 201804, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,18]]},"reference":[{"key":"ref_1","first-page":"1","article-title":"Re-Plannable Automated Parking System With a Standalone Around View Monitor for Narrow Parking Lots","volume":"21","author":"Jang","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Banzhaf, H., Nienhuser, D., Knoop, S., and Zollner, J.M. (2017, January 11\u201314). The Future of Parking: A Survey on Automated Valet Parking with an Outlook on High Density Parking. Proceedings of the 2017 28th IEEE Intelligent Vehicles Symposium, Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995971"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Qin, T., Chen, T., Chen, Y., and Su, Q. (2020, January 25\u201329). AVP-SLAM: Semantic Visual Mapping and Localization for Autonomous Vehicles in the Parking Lot. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, IROS 2020, Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9340939"},{"key":"ref_4","unstructured":"Yan, W., Tao, Y., Junqiao, Z., Linting, G., and Wei, J. (2018, January 26\u201330). VH-HFCN based Parking Slot and Lane Markings Segmentation on Panoramic Surround View. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Piscataway, NJ, USA."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Yang, Q., Chen, H., Su, J., and Li, J. (2019, January 21\u201322). Towards High Accuracy Parking Slot Detection for Automated Valet Parking System. Proceedings of the SAE 2019 New Energy and Intelligent Connected Vehicle Technology Conference, Shanghai, China.","DOI":"10.4271\/2019-01-5061"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1177\/027836498600500304","article-title":"Toward Efficient Trajectory Planning\u2014The Path-Velocity Decomposition","volume":"5","author":"Kant","year":"1986","journal-title":"Int. J. Robot. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1007\/s12239-020-0085-9","article-title":"Trajectory Planning for Automated Parking Systems Using Deep Reinforcement Learning","volume":"21","author":"Du","year":"2020","journal-title":"Int. J. Automot. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhang, P., Xiong, L., Yu, Z., Fang, P., Yan, S., Yao, J., and Zhou, Y. (2019). Reinforcement Learning-Based End-to-End Parking for Automatic Parking System. Sensors, 19.","DOI":"10.3390\/s19183996"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bejar, E., and Moran, A. (2019, January 7\u20139). Reverse Parking a Car-Like Mobile Robot with Deep Reinforcement Learning and Preview Control. Proceedings of the 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, Las Vegas, NV, USA.","DOI":"10.1109\/CCWC.2019.8666613"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"154485","DOI":"10.1109\/ACCESS.2020.3017770","article-title":"Reinforcement Learning-Based Motion Planning for Automatic Parking System","volume":"8","author":"Zhang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_11","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_13","first-page":"1629","article-title":"Approximate Modified Policy Iteration and its Application to the Game of Tetris","volume":"16","author":"Scherrer","year":"2015","journal-title":"J. Mach. Learn. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1007\/s10472-011-9258-6","article-title":"Multi-armed bandits with episode context","volume":"61","author":"Rosin","year":"2011","journal-title":"Ann. Math. Artif. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TCIAIG.2012.2186810","article-title":"A Survey of Monte Carlo Tree Search Methods","volume":"4","author":"Browne","year":"2012","journal-title":"IEEE Trans. Comput. Intell. AI Games"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of Go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1557","DOI":"10.1049\/iet-its.2019.0049","article-title":"Laser-based SLAM automatic parallel parking path planning and tracking for passenger vehicle","volume":"13","author":"Song","year":"2019","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"42884","DOI":"10.1109\/ACCESS.2020.2978054","article-title":"Augmented Ship Tracking Under Occlusion Conditions From Maritime Surveillance Videos","volume":"8","author":"Chen","year":"2020","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Lee, S., Lim, W., and Sunwoo, M. (2020). Robust Parking Path Planning with Error-Adaptive Sampling under Perception Uncertainty. Sensors, 20.","DOI":"10.3390\/s20123560"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3263","DOI":"10.1109\/TITS.2016.2546386","article-title":"Time-Optimal Maneuver Planning in Automatic Parallel Parking Using a Simultaneous Dynamic Optimization Approach","volume":"17","author":"Li","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_21","unstructured":"Chao, C., Rickert, M., and Knoll, A. (July, January 28). Path planning with orientation-aware space exploration guided heuristic search for autonomous parking and maneuvering. Proceedings of the 2015 IEEE Intelligent Vehicles Symposium (IV), Piscataway, NJ, USA."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1053","DOI":"10.1109\/LRA.2019.2893975","article-title":"Learning to Predict Ego-Vehicle Poses for Sampling-Based Nonholonomic Motion Planning","volume":"4","author":"Banzhaf","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1109\/TITS.2014.2335054","article-title":"Automatic Parallel Parking in Tiny Spots: Path Planning and Control","volume":"16","author":"Vorobieva","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"41","DOI":"10.4271\/2016-01-1881","article-title":"Study on Path Following Control Method for Automatic Parking System Based on LQR","volume":"10","author":"Fan","year":"2016","journal-title":"SAE Int. J. Passeng. Cars Electron. Electr. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1225","DOI":"10.1109\/TITS.2014.2354423","article-title":"Autonomous Reverse Parking System Based on Robust Path Generation and Improved Sliding Mode Control","volume":"16","author":"Du","year":"2015","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_26","first-page":"447","article-title":"Automatic parallel parking algorithm for a car-like robot using fuzzy pd+i control","volume":"26","author":"Ballinas","year":"2018","journal-title":"Eng. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Bernhard, J., Gieselmann, R., Esterle, K., and Knoll, A. (2018, January 4\u20137). Experience-Based Heuristic Search: Robust Motion Planning with Deep Q-Learning. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems, Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569436"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Hu, F., Chen, H., and Zhang, J. (2019, January 21\u201322). Study on Robust Motion Planning Method for Automatic Parking Assist System Based on Neural Network and Tree Search. Proceedings of the SAE 2019 New Energy and Intelligent Connected Vehicle Technology Conference, Shanghai, China.","DOI":"10.4271\/2019-01-5059"},{"key":"ref_29","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016, January 2\u20134). Continuous control with deep reinforcement learning. Proceedings of the 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1007\/s10994-010-5223-6","article-title":"Policy search for motor primitives in robotics","volume":"84","author":"Kober","year":"2011","journal-title":"Mach. Learn."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1007\/s12239-014-0102-y","article-title":"Automatic Parking of Vehicles: A Review of Literatures","volume":"15","author":"Wang","year":"2014","journal-title":"Int. J. Automot. Technol."},{"key":"ref_32","unstructured":"Choi, S., Boussard, C., and D\u2019Andrea-Novel, B. (September, January 28). Easy path planning and robust control for automatic parallel parking. Proceedings of the 18th IFAC World Congress, Milano, Italy."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"3008","DOI":"10.1109\/TSG.2019.2962625","article-title":"Safe Off-Policy Deep Reinforcement Learning Algorithm for Volt-VAR Control in Power Distribution Systems","volume":"11","author":"Wang","year":"2020","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_34","unstructured":"Janner, M., Fu, J., Zhang, M., and Levine, S. (2019, January 8\u201314). When to trust your model: Model-based policy optimization. Proceedings of the 33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019, Vancouver, BC, Canada."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Hamada, K., Zhencheng, H., Mengyang, F., and Hui, C. (July, January 28). Surround view based parking lot detection and tracking. Proceedings of the 2015 IEEE Intelligent Vehicles Symposium (IV), Piscataway, NJ, USA.","DOI":"10.1109\/IVS.2015.7225832"},{"key":"ref_36","unstructured":"British Standards Institution (2017). Intelligent Transport Systems-Assisted Parking System (APS)-Performance Requirements and Test Procedures, BSI Standards Limited. BS ISO 16787:2017."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1663","DOI":"10.1109\/TWC.2019.2956044","article-title":"Robust Data Detection for MIMO Systems with One-Bit ADCs: A Reinforcement Learning Approach","volume":"19","author":"Jeon","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7297\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:47:07Z","timestamp":1760179627000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7297"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,18]]},"references-count":37,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["s20247297"],"URL":"https:\/\/doi.org\/10.3390\/s20247297","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,18]]}}}