{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T14:23:58Z","timestamp":1763389438217,"version":"3.37.3"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T00:00:00Z","timestamp":1616457600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T00:00:00Z","timestamp":1616457600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2022,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This paper studies the online learning control of a truck-trailer parking problem via adaptive dynamic programming (ADP). The contribution is twofold. First, a novel ADP method is developed for systems with parametric nonlinearities. It learns the optimal control policy of the linearized system at the origin, while the learning process utilizes online measurements of the full system and is robust with respect to nonlinear disturbances. Second, a control strategy is formulated for a commonly seen truck-trailer parallel parking problem, and the proposed ADP method is integrated into the strategy to provide online learning capabilities and to handle uncertainties. A numerical simulation is conducted to demonstrate the effectiveness of the proposed methodology.<\/jats:p>","DOI":"10.1007\/s40747-021-00330-z","type":"journal-article","created":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T15:04:25Z","timestamp":1616511865000},"page":"1835-1845","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["A tractor-trailer parking control scheme using adaptive dynamic programming"],"prefix":"10.1007","volume":"8","author":[{"given":"Chenyong","family":"Guan","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8828-7832","authenticated-orcid":false,"given":"Yu","family":"Jiang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,3,23]]},"reference":[{"issue":"9","key":"330_CR1","doi-asserted-by":"publisher","first-page":"1551","DOI":"10.1109\/TAC.2002.802750","volume":"47","author":"C Altafini","year":"2002","unstructured":"Altafini C (2002) Following a path of varying curvature as an output regulation problem. IEEE Trans Autom Control 47(9):1551\u20131556","journal-title":"IEEE Trans Autom Control"},{"issue":"6","key":"330_CR2","doi-asserted-by":"publisher","first-page":"915","DOI":"10.1109\/70.976025","volume":"17","author":"C Altafini","year":"2001","unstructured":"Altafini C, Speranzon A, Wahlberg B (2001) A feedback control scheme for reversing a truck and trailer vehicle. IEEE Trans Robot Autom 17(6):915\u2013922","journal-title":"IEEE Trans Robot Autom"},{"issue":"4","key":"330_CR3","doi-asserted-by":"publisher","first-page":"913","DOI":"10.1109\/TSMCB.2008.926599","volume":"38","author":"SN Balakrishnan","year":"2008","unstructured":"Balakrishnan SN, Ding J, Lewis FL (2008) Issues on stability of ADP feedback controllers for dynamical systems. IEEE Trans Syst Man Cybern Part B: Cybern 38(4):913\u2013917","journal-title":"IEEE Trans Syst Man Cybern Part B: Cybern"},{"key":"330_CR4","volume-title":"Dynamic programming","author":"R Bellman","year":"1957","unstructured":"Bellman R (1957) Dynamic programming. Princeton University Press, Princeton"},{"key":"330_CR5","volume-title":"Dynamic programming and optimal control","author":"DP Bertsekas","year":"2007","unstructured":"Bertsekas DP (2007) Dynamic programming and optimal control, 4th edn. Athena Scientific Belmont, Belmonth","edition":"4"},{"key":"330_CR6","volume-title":"Neuro-dynamic programming","author":"DP Bertsekas","year":"1996","unstructured":"Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena Scientific, Nashua"},{"key":"330_CR7","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1016\/j.automatica.2016.05.003","volume":"71","author":"T Bian","year":"2016","unstructured":"Bian T, Jiang ZP (2016) Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design. Automatica 71:348\u2013360","journal-title":"Automatica"},{"issue":"6","key":"330_CR8","doi-asserted-by":"publisher","first-page":"4150","DOI":"10.1137\/18M1214147","volume":"57","author":"T Bian","year":"2019","unstructured":"Bian T, Jiang ZP (2019) Continuous-time robust dynamic programming. SIAM J Control Optim 57(6):4150\u20134174","journal-title":"SIAM J Control Optim"},{"issue":"11","key":"330_CR9","doi-asserted-by":"publisher","first-page":"4423","DOI":"10.1109\/TAC.2019.2905215","volume":"64","author":"C Chen","year":"2019","unstructured":"Chen C, Modares H, Xie K, Lewis FL, Wan Y, Xie S (2019) Reinforcement learning-based adaptive optimal exponential tracking control of linear systems with unknown dynamics. IEEE Trans Autom Control 64(11):4423\u20134438","journal-title":"IEEE Trans Autom Control"},{"key":"330_CR10","doi-asserted-by":"publisher","unstructured":"Hafner M, Pilutti T (2017) Control for automated trailer backup. In: SAE Technical Paper, 2017-01-0040 https:\/\/doi.org\/10.4271\/2017-01-0040","DOI":"10.4271\/2017-01-0040"},{"key":"330_CR11","doi-asserted-by":"crossref","unstructured":"Halgamuge SK, Runkler TA, Glesner M (1994) A hierarchical hybrid fuzzy controller for real-time reverse driving support of vehicles with long trailers. In: Proceedings of 1994 IEEE 3rd international fuzzy systems conference, vol\u00a02, pp 1207\u20131210","DOI":"10.1109\/FUZZY.1994.343907"},{"issue":"5","key":"330_CR12","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1016\/0893-6080(89)90020-8","volume":"2","author":"K Hornik","year":"1989","unstructured":"Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359\u2013366","journal-title":"Neural Netw"},{"issue":"10","key":"330_CR13","doi-asserted-by":"publisher","first-page":"4607","DOI":"10.1109\/TNNLS.2017.2771459","volume":"29","author":"Y Jiang","year":"2018","unstructured":"Jiang Y, Fan JL, Chai TY, Lewis FL, Li JN (2018) Tracking control for linear discrete-time networked control systems with unknown dynamics and dropout. IEEE Trans Neural Netw Learn Syst 29(10):4607\u20134620","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"issue":"11","key":"330_CR14","doi-asserted-by":"publisher","first-page":"2917","DOI":"10.1109\/TAC.2015.2414811","volume":"60","author":"Y Jiang","year":"2015","unstructured":"Jiang Y, Jiang ZP (2015) Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Trans Autom Control 60(11):2917\u20132929","journal-title":"IEEE Trans Autom Control"},{"key":"330_CR15","doi-asserted-by":"publisher","DOI":"10.1002\/9781119132677","volume-title":"Robust adaptive dynamic programming","author":"Y Jiang","year":"2017","unstructured":"Jiang Y, Jiang ZP (2017) Robust adaptive dynamic programming. Wiley, New York"},{"issue":"3","key":"330_CR16","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1561\/2600000023","volume":"8","author":"ZP Jiang","year":"2020","unstructured":"Jiang ZP, Bian T, Gao W (2020) Learning-based control: a tutorial and some recent results. Found Trends\u00ae Syst Control 8(3):176\u2013284. https:\/\/doi.org\/10.1561\/2600000023","journal-title":"Found Trends\u00ae Syst Control"},{"issue":"5","key":"330_CR17","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1016\/j.ejcon.2013.05.017","volume":"19","author":"ZP Jiang","year":"2013","unstructured":"Jiang ZP, Jiang Y (2013) Robust adaptive dynamic programming for linear and nonlinear systems: an overview. Eur J Control 19(5):417\u2013425","journal-title":"Eur J Control"},{"issue":"1","key":"330_CR18","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1109\/TAC.1968.1098829","volume":"13","author":"D Kleinman","year":"1968","unstructured":"Kleinman D (1968) On an iterative technique for Riccati equation computations. IEEE Trans Autom Control 13(1):114\u2013115","journal-title":"IEEE Trans Autom Control"},{"key":"330_CR19","doi-asserted-by":"crossref","unstructured":"Leng Z, Minor M (2010) A simple tractor-trailer backing control law for path following. In: 2010 IEEE\/RSJ international conference on intelligent robots and systems, Taipei, Taiwan, pp 5538\u20135542","DOI":"10.1109\/ICRA.2011.5979918"},{"volume-title":"Reinforcement learning and approximate dynamic programming for feedback control","year":"2013","key":"330_CR20","unstructured":"Lewis FL, Liu D (eds) (2013) Reinforcement learning and approximate dynamic programming for feedback control. Wiley, Hoboken"},{"issue":"3","key":"330_CR21","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1109\/MCAS.2009.933854","volume":"9","author":"FL Lewis","year":"2009","unstructured":"Lewis FL, Vrabie D (2009) Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits Syst Mag 9(3):32\u201350","journal-title":"IEEE Circuits Syst Mag"},{"key":"330_CR22","doi-asserted-by":"publisher","DOI":"10.1002\/9781118122631","volume-title":"Optimal control","author":"FL Lewis","year":"2012","unstructured":"Lewis FL, Vrabie D, Syrmos VL (2012) Optimal control, 3rd edn. Wiley, New York","edition":"3"},{"key":"330_CR23","doi-asserted-by":"crossref","unstructured":"Nguyen D, Widrow B (1989) The truck backer-upper: an example of self-learning in neural networks. In: International 1989 joint conference on neural networks, vol 2, pp 357\u2013363","DOI":"10.1109\/IJCNN.1989.118723"},{"key":"330_CR24","doi-asserted-by":"publisher","first-page":"108672","DOI":"10.1016\/j.automatica.2019.108672","volume":"112","author":"A Odekunle","year":"2020","unstructured":"Odekunle A, Gao W, Davari M, Jiang ZP (2020) Reinforcement learning and non-zero-sum game output regulation for multi-player linear uncertain systems. Automatica 112:108672","journal-title":"Automatica"},{"issue":"1\u20133","key":"330_CR25","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1016\/S0925-2312(97)00071-4","volume":"18","author":"GH Park","year":"1998","unstructured":"Park GH, Pao YH (1998) Training neural-net controllers with the help of trajectories generated with fuzzy rules (demonstrated with the truck backup task). Neurocomputing 18(1\u20133):91\u2013105","journal-title":"Neurocomputing"},{"issue":"2","key":"330_CR26","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1162\/neco.1991.3.2.246","volume":"3","author":"J Park","year":"1991","unstructured":"Park J, Sandberg IW (1991) Universal approximation using radial-basis-function networks. Neural Comput 3(2):246\u2013257","journal-title":"Neural Comput"},{"key":"330_CR27","doi-asserted-by":"publisher","DOI":"10.1002\/9780470182963","volume-title":"Approximate dynamic programming: solving the curses of dimensionality","author":"WB Powell","year":"2007","unstructured":"Powell WB (2007) Approximate dynamic programming: solving the curses of dimensionality. Wiley, New York"},{"key":"330_CR28","unstructured":"Prajna S, Papachristodoulou A, Wu F (2004) Nonlinear control synthesis by sum of squares optimization: a Lyapunov-based approach. In: Proceedings of the Asian control conference, pp 157\u2013165"},{"issue":"4","key":"330_CR29","doi-asserted-by":"publisher","first-page":"1240","DOI":"10.1109\/TCST.2015.2499699","volume":"24","author":"P Ritzen","year":"2016","unstructured":"Ritzen P, Roebroek E, Van De Wouw N, Jiang ZP, Nijmeijer H (2016) Trailer steering control of a tractor-trailer robot. IEEE Trans Control Syst Technol 24(4):1240\u20131252. https:\/\/doi.org\/10.1109\/TCST.2015.2499699","journal-title":"IEEE Trans Control Syst Technol"},{"issue":"4","key":"330_CR30","doi-asserted-by":"publisher","first-page":"1240","DOI":"10.1109\/TCST.2015.2499699","volume":"24","author":"P Ritzen","year":"2015","unstructured":"Ritzen P, Roebroek E, van de Wouw N, Jiang ZP, Nijmeijer H (2015) Trailer steering control of a tractor-trailer robot. IEEE Trans Control Syst Technol 24(4):1240\u20131252","journal-title":"IEEE Trans Control Syst Technol"},{"volume-title":"Handbook of learning and approximate dynamic programming","year":"2004","key":"330_CR31","unstructured":"Si J, Barto AG, Powell WB, Wunsch DC et al (eds) (2004) Handbook of learning and approximate dynamic programming. Wiley Inc, Hoboken"},{"key":"330_CR32","doi-asserted-by":"publisher","unstructured":"Song R, Wei Q, Zhang H, Lewis FL (2019) Discrete-time non-zero-sum games with completely unknown dynamics. IEEE Trans Cybern https:\/\/doi.org\/10.1109\/TCYB.2019.2957406","DOI":"10.1109\/TCYB.2019.2957406"},{"key":"330_CR33","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"1998","unstructured":"Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Cambridge University Press, Cambridge"},{"key":"330_CR34","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, Cambridge"},{"issue":"2","key":"330_CR35","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1109\/91.277961","volume":"2","author":"K Tanaka","year":"1994","unstructured":"Tanaka K, Sano M (1994) A robust stabilization problem of fuzzy control systems and its application to backing up control of a truck-trailer. IEEE Trans Fuzzy Syst 2(2):119\u2013134","journal-title":"IEEE Trans Fuzzy Syst"},{"issue":"5","key":"330_CR36","doi-asserted-by":"publisher","first-page":"878","DOI":"10.1016\/j.automatica.2010.02.018","volume":"46","author":"KG Vamvoudakis","year":"2010","unstructured":"Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878\u2013888","journal-title":"Automatica"},{"issue":"8","key":"330_CR37","doi-asserted-by":"publisher","first-page":"1556","DOI":"10.1016\/j.automatica.2011.03.005","volume":"47","author":"KG Vamvoudakis","year":"2011","unstructured":"Vamvoudakis KG, Lewis FL (2011) Multi-player non-zero-sum games: online adaptive learning solution of coupled Hamilton\u2013Jacobi equations. Automatica 47(8):1556\u20131569","journal-title":"Automatica"},{"key":"330_CR38","volume-title":"Optimal adaptive control and differential games by reinforcement learning principles","author":"D Vrabie","year":"2013","unstructured":"Vrabie D, Vamvoudakis KG, Lewis FL (2013) Optimal adaptive control and differential games by reinforcement learning principles. The Institution of Engineering and Technology, London"},{"issue":"2","key":"330_CR39","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1109\/MCI.2009.932261","volume":"4","author":"FY Wang","year":"2009","unstructured":"Wang FY, Zhang H, Liu D (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4(2):39\u201347","journal-title":"IEEE Comput Intell Mag"},{"issue":"2","key":"330_CR40","doi-asserted-by":"publisher","first-page":"444","DOI":"10.1109\/TNNLS.2015.2464080","volume":"27","author":"Q Wei","year":"2015","unstructured":"Wei Q, Song R, Yan P (2015) Data-driven zero-sum neuro-optimal control for a class of continuous-time unknown nonlinear systems with disturbance using ADP. IEEE Trans Neural Netw Learn Syst 27(2):444\u2013458","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"330_CR41","unstructured":"Werbos P (1974) Beyond regression: new tools for prediction and analysis in the behavioral sciences. Ph.D. thesis, Harvard University Comm Appl Math"},{"key":"330_CR42","first-page":"25","volume":"22","author":"P Werbos","year":"1977","unstructured":"Werbos P (1977) Advanced forecasting methods for global crisis warning and models of intelligence. Gen Syst Yearb 22:25\u201338","journal-title":"Gen Syst Yearb"},{"key":"330_CR43","first-page":"3","volume-title":"Reinforcement learning and approximate dynamic programming for feedback control","author":"P Werbos","year":"2013","unstructured":"Werbos P (2013) Reinforcement learning and approximate dynamic programming (RLADP)\u2014foundations, common misconceptions and the challenges ahead. In: Lewis FL, Liu D (eds) Reinforcement learning and approximate dynamic programming for feedback control. Wiley, Hoboken, pp 3\u201330"},{"key":"330_CR44","doi-asserted-by":"publisher","unstructured":"Yang Y, Vamvoudakis KG, Modares H, Yin Y, Wunsch DC (2020) Hamiltonian-driven hybrid adaptive dynamic programming. IEEE Trans Syst Man Cybern: Syst https:\/\/doi.org\/10.1109\/TSMC.2019.2962103.","DOI":"10.1109\/TSMC.2019.2962103."},{"issue":"12","key":"330_CR45","doi-asserted-by":"publisher","first-page":"2226","DOI":"10.1109\/TNN.2011.2168538","volume":"22","author":"H Zhang","year":"2011","unstructured":"Zhang H, Cui L, Zhang X, Luo Y (2011) Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method. IEEE Trans Neural Netw 22(12):2226\u20132236","journal-title":"IEEE Trans Neural Netw"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-021-00330-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-021-00330-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-021-00330-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,5,30]],"date-time":"2022-05-30T01:06:53Z","timestamp":1653872813000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-021-00330-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,23]]},"references-count":45,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,6]]}},"alternative-id":["330"],"URL":"https:\/\/doi.org\/10.1007\/s40747-021-00330-z","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"type":"print","value":"2199-4536"},{"type":"electronic","value":"2198-6053"}],"subject":[],"published":{"date-parts":[[2021,3,23]]},"assertion":[{"value":"28 November 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 March 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}