{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T20:43:23Z","timestamp":1783025003738,"version":"3.54.6"},"reference-count":46,"publisher":"Cambridge University Press (CUP)","issue":"10","license":[{"start":{"date-parts":[[2024,12,2]],"date-time":"2024-12-02T00:00:00Z","timestamp":1733097600000},"content-version":"unspecified","delay-in-days":62,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotica"],"published-print":{"date-parts":[[2024,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Expert drivers possess the ability to execute high sideslip angle maneuvers, commonly known as drifting, during racing to navigate sharp corners and execute rapid turns. However, existing model-based controllers encounter challenges in handling the highly nonlinear dynamics associated with drifting along general paths. While reinforcement learning-based methods alleviate the reliance on explicit vehicle models, training a policy directly for autonomous drifting remains difficult due to multiple objectives. In this paper, we propose a control framework for autonomous drifting in the general case, based on curriculum reinforcement learning. The framework empowers the vehicle to follow paths with varying curvature at high speeds, while executing drifting maneuvers during sharp corners. Specifically, we consider the vehicle\u2019s dynamics to decompose the overall task and employ curriculum learning to break down the training process into three stages of increasing complexity. Additionally, to enhance the generalization ability of the learned policies, we introduce randomization into sensor observation noise, actuator action noise, and physical parameters. The proposed framework is validated using the CARLA simulator, encompassing various vehicle types and parameters. Experimental results demonstrate the effectiveness and efficiency of our framework in achieving autonomous drifting along general paths. The code is available at <jats:uri xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/github.com\/BIT-KaiYu\/drifting\">https:\/\/github.com\/BIT-KaiYu\/drifting<\/jats:uri>.<\/jats:p>","DOI":"10.1017\/s026357472400119x","type":"journal-article","created":{"date-parts":[[2024,12,2]],"date-time":"2024-12-02T09:10:05Z","timestamp":1733130605000},"page":"3263-3280","source":"Crossref","is-referenced-by-count":4,"title":["Curriculum reinforcement learning-based drifting along a general path for autonomous vehicles"],"prefix":"10.1017","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-1128-3491","authenticated-orcid":false,"given":"Kai","family":"Yu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mengyin","family":"Fu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0709-7875","authenticated-orcid":false,"given":"Xiaohui","family":"Tian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuaicong","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"56","published-online":{"date-parts":[[2024,12,2]]},"reference":[{"key":"S026357472400119X_ref25","first-page":"1","article-title":"Modeling and control for dynamic drifting trajectories","volume":"9","author":"Weber","year":"2023","journal-title":"IEEE Trans. Intell Veh"},{"key":"S026357472400119X_ref40","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.adi7566"},{"key":"S026357472400119X_ref6","doi-asserted-by":"publisher","DOI":"10.1080\/00423111003746140"},{"key":"S026357472400119X_ref31","unstructured":"[31] Schulman, J. , Wolski, F. , Dhariwal, P. , Radford, A. and Klimov, O. , \u201cProximal policy optimization algorithms,\u201d arXiv: 1707.06347, 1-12 (2017)"},{"key":"S026357472400119X_ref21","doi-asserted-by":"publisher","DOI":"10.1109\/IBCAST.2018.8312234"},{"key":"S026357472400119X_ref7","doi-asserted-by":"publisher","DOI":"10.1016\/j.conengprac.2011.07.010"},{"key":"S026357472400119X_ref12","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574721001600"},{"key":"S026357472400119X_ref19","doi-asserted-by":"crossref","unstructured":"[19] Hindiyeh, R. Y. and Gerdes, J. C. , \u201cEquilibrium analysis of drifting vehicles for control design,\u201d Dynamic Systems and Control Conference, Philadelphia,\u00a0USA (2009) pp. 181\u2013188.","DOI":"10.1115\/DSCC2009-2626"},{"key":"S026357472400119X_ref2","doi-asserted-by":"publisher","DOI":"10.1080\/00423114.2023.2297799"},{"key":"S026357472400119X_ref34","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2022.3150793"},{"key":"S026357472400119X_ref26","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2018.04.015"},{"key":"S026357472400119X_ref35","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2019.2958308"},{"key":"S026357472400119X_ref33","unstructured":"[33] Dosovitskiy, A. , Ros, G. , Codevilla, F. , Lopez, A. and Koltun, V. , \u201cCarla: An open urban driving simulator,\u201d Conference on Robot Learning, California, USA (PMLR, 2017) pp. 1\u201316."},{"key":"S026357472400119X_ref42","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2017.8202133"},{"key":"S026357472400119X_ref15","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9812249"},{"key":"S026357472400119X_ref23","doi-asserted-by":"publisher","DOI":"10.1109\/LCSYS.2021.3136142"},{"key":"S026357472400119X_ref4","first-page":"216","article-title":"Analysis on vehicle stability in critical cornering using phase-plane method","volume":"2","author":"Inagaki","year":"1995","journal-title":"JSAE Rev"},{"key":"S026357472400119X_ref8","doi-asserted-by":"publisher","DOI":"10.1115\/1.4045320"},{"key":"S026357472400119X_ref13","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574723001613"},{"key":"S026357472400119X_ref38","unstructured":"[38] Rudin, N. , Hoeller, D. , Reist, P. and Hutter, M. , \u201cLearning to walk in minutes using massively parallel deep reinforcement learning,\u201d Conference on Robot Learning, Auckland, New Zealand (PMLR, 2022). 91\u2013100."},{"key":"S026357472400119X_ref20","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2021.3085503"},{"key":"S026357472400119X_ref43","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8460528"},{"key":"S026357472400119X_ref37","doi-asserted-by":"crossref","unstructured":"[37] Bengio, Y. , Louradour, J. , Collobert, R. and Weston, J. , \u201cCurriculum learning,\u201d Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, Canada (2009) pp. 41\u201348.","DOI":"10.1145\/1553374.1553380"},{"key":"S026357472400119X_ref22","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2022.3171734"},{"key":"S026357472400119X_ref39","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2023.3251193"},{"key":"S026357472400119X_ref30","doi-asserted-by":"publisher","DOI":"10.1109\/OJITS.2022.3181510"},{"key":"S026357472400119X_ref24","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9812185"},{"key":"S026357472400119X_ref5","doi-asserted-by":"publisher","DOI":"10.1109\/87.668041"},{"key":"S026357472400119X_ref27","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC55140.2022.9921909"},{"key":"S026357472400119X_ref32","unstructured":"[32] Schulman, J. , Moritz, P. , Levine, S. , Jordan, M. and Abbeel, P. , \u201cHigh dimensional continuous control using generalized advantage estimation,\u201d arXiv: 1506.02438, 1-14 (2015)"},{"key":"S026357472400119X_ref14","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06419-4"},{"key":"S026357472400119X_ref29","volume-title":"Dynamics and Control of Drifting in Automobiles","author":"Hindiyeh","year":"2013"},{"key":"S026357472400119X_ref17","first-page":"7382","article-title":"Curriculum learning for reinforcement learning domains: a framework and survey","volume":"21","author":"Narvekar","year":"2020","journal-title":"J. Mach. Learn. Res"},{"key":"S026357472400119X_ref18","doi-asserted-by":"publisher","DOI":"10.1134\/S1560354709060069"},{"key":"S026357472400119X_ref36","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2013.6629659"},{"key":"S026357472400119X_ref3","first-page":"1","article-title":"Phase portrait analysis and drifting control of unmanned tracked vehicles","volume":"10","author":"Dong","year":"2024","journal-title":"IEEE Trans. Intell. Veh"},{"key":"S026357472400119X_ref9","doi-asserted-by":"publisher","DOI":"10.1016\/j.mechatronics.2018.05.009"},{"key":"S026357472400119X_ref11","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton","year":"2018"},{"key":"S026357472400119X_ref16","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2967299"},{"key":"S026357472400119X_ref1","first-page":"1","article-title":"A multi-layer drifting controller for all-wheel drive vehicles beyond driving limits","volume":"29","author":"Tian","year":"2023","journal-title":"IEEE\/ASME Trans. Mechatron"},{"key":"S026357472400119X_ref28","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487756"},{"key":"S026357472400119X_ref44","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574722001230"},{"key":"S026357472400119X_ref46","doi-asserted-by":"publisher","DOI":"10.1115\/1.4027471"},{"key":"S026357472400119X_ref45","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574724000092"},{"key":"S026357472400119X_ref10","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2023.3235007"},{"key":"S026357472400119X_ref41","first-page":"4555","article-title":"A survey on curriculum learning","volume":"44","author":"Wang","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"}],"container-title":["Robotica"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S026357472400119X","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T05:33:30Z","timestamp":1737437610000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S026357472400119X\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10]]},"references-count":46,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2024,10]]}},"alternative-id":["S026357472400119X"],"URL":"https:\/\/doi.org\/10.1017\/s026357472400119x","relation":{},"ISSN":["0263-5747","1469-8668"],"issn-type":[{"value":"0263-5747","type":"print"},{"value":"1469-8668","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10]]}}}