{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T18:00:31Z","timestamp":1790359231502,"version":"4.1.0"},"reference-count":49,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T00:00:00Z","timestamp":1790294400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100017724","name":"Henan Provincial Department of Transportation","doi-asserted-by":"publisher","award":["2020J-2-9"],"award-info":[{"award-number":["2020J-2-9"]}],"id":[{"id":"10.13039\/100017724","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>To improve the convergence issues of deep reinforcement learning in the control of vehicle semi-active suspension, and enhance the control effectiveness of the\u00a0vehicle\u2019s\u00a0semi-active suspension, a Twin Delayed Deep Deterministic-based control framework tailored for the high-dimensional nonlinear dynamics of the vehicle semi-active suspension model is proposed. Rather than merely applying standard deep reinforcement learning, this study introduces a domain-specific state-action mapping mechanism and a multi-objective reward function to physically constrain the neural network outputs and coordinate the full-vehicle dynamic responses. The simulation program for 7-degree-of-freedom vehicle semi-active suspension model embedded in the proposed control method is developed using MATLAB software. The validity of the proposed control method is verified using the developed program under different road roughness and speeds. The results indicate that compared to the passive suspension the classical linear Skyhook control method and the Fuzzy control method the proposed Twin Delayed Deep Deterministic based method achieves superior control effectiveness. Specifically in terms of the vehicle body vertical acceleration the proposed method achieves a reduction of 49.41% compared to the passive suspension and 9.33% compared to the Skyhook method on a Class A road at 60 km\/h a reduction of 24.82% compared to the passive suspension on a Class A road at 120 km\/h and a reduction of 3.19% compared to the Skyhook method on a Class B road at 60 km\/h. Furthermore compared to the traditional Fuzzy control method the proposed method demonstrates superior vibration suppression effectiveness across all evaluated operating conditions achieving significantly lower frequency weighted vertical root mean square accelerations. Furthermore the power spectral density analysis demonstrates that the proposed method significantly suppresses the low frequency resonance of the sprung mass across varying road conditions compared to the passive Skyhook and Fuzzy methods, further validating its superior broadband vibration isolation capability. This demonstrates that the Twin Delayed Deep Deterministic-based control method for the vehicle semi-active suspension model has the advantages of fast convergence speed and good control effectiveness under road surface excitation.<\/jats:p>","DOI":"10.1371\/journal.pone.0359102","type":"journal-article","created":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T17:31:57Z","timestamp":1790357517000},"page":"e0359102","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["A twin delayed deep deterministic-based control method for a full vehicle semi-active suspension system"],"prefix":"10.1371","volume":"21","author":[{"given":"Yongjun","family":"Wan","sequence":"first","affiliation":[{"name":"Henan College of Transportation","place":["Zhengzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoming","family":"Wang","sequence":"additional","affiliation":[{"name":"China Construction Fifth Bureau (HENAN) Construction Co., Ltd.","place":["Zhengzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gang","family":"Zhi","sequence":"additional","affiliation":[{"name":"Henan College of Transportation","place":["Zhengzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Yang","sequence":"additional","affiliation":[{"name":"China Construction Fifth Bureau (HENAN) Construction Co., Ltd.","place":["Zhengzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juye","family":"Wang","sequence":"additional","affiliation":[{"name":"Zhengzhou University","place":["Zhengzhou, China"],"department":["School of Civil Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangcheng","family":"Zhang","sequence":"additional","affiliation":[{"name":"Zhengzhou University","place":["Zhengzhou, China"],"department":["School of Civil Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3403-5633","authenticated-orcid":true,"given":"Yan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Zhengzhou University","place":["Zhengzhou, China"],"department":["School of Civil Engineering"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,25]]},"reference":[{"key":"pone.0359102.ref001","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1080\/00423114.2010.532223","article-title":"Editors\u2019 perspectives: road vehicle suspension design, dynamics, and control","volume":"49","author":"D Cao","year":"2011","journal-title":"Vehicle Syst Dyn"},{"issue":"2","key":"pone.0359102.ref002","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1139\/tcsme-2008-0019","article-title":"Design optimization of vehicle suspensions with a quarter-vehicle model","volume":"32","author":"Z Chi","year":"2008","journal-title":"Trans Can Soc Mech Eng"},{"issue":"01","key":"pone.0359102.ref003","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1142\/S0219455407002198","article-title":"Predictive control and signal on noise reduction for semi-active hydraulic damper","volume":"07","author":"M-H Shih","year":"2007","journal-title":"Int J Str Stab Dyn"},{"issue":"02","key":"pone.0359102.ref004","article-title":"Tracking active control forces by using a semi-active vehicle suspension integrated with negative stiffness","volume":"24","author":"X Shi","year":"2023","journal-title":"Int J Str Stab Dyn"},{"issue":"3","key":"pone.0359102.ref005","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1016\/0045-7949(89)90253-8","article-title":"On the statistical performance of active and semi-active car suspension systems","volume":"33","author":"MM ElMadany","year":"1989","journal-title":"Comput Struct"},{"issue":"4","key":"pone.0359102.ref006","article-title":"Application of semi-active inerter in semi-active suspensions via force tracking","volume":"138","author":"MZQ Chen","year":"2016","journal-title":"J Vibr Acoust"},{"issue":"14","key":"pone.0359102.ref007","doi-asserted-by":"crossref","first-page":"1811","DOI":"10.1177\/1045389X15577653","article-title":"Development of a linear damper working with magnetorheological shear thickening fluids","volume":"26","author":"J Yang","year":"2015","journal-title":"J Intell Mater Syst Struct"},{"issue":"7","key":"pone.0359102.ref008","article-title":"Tests and micro\u2013macro cross-scale model of high-performance acrylate viscoelastic dampers used in structural resistance","volume":"149","author":"Y-R Dong","year":"2023","journal-title":"J Struct Eng"},{"issue":"1","key":"pone.0359102.ref009","article-title":"Hyperelastic hybrid molecular chain model of thermal-oxidative aging viscoelastic damping materials based on physical\u2013chemical process","volume":"149","author":"Q-Q Li","year":"2023","journal-title":"J Eng Mech"},{"issue":"4","key":"pone.0359102.ref010","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1243\/095440905X8899","article-title":"Design and analysis of magnetorheological dampers for train suspension","volume":"219","author":"YK Lau","year":"2005","journal-title":"Proc Inst Mech Eng Part F: J Rail Rapid Transit"},{"issue":"2","key":"pone.0359102.ref011","doi-asserted-by":"crossref","first-page":"565","DOI":"10.2514\/1.24033","article-title":"Semi-active linear stroke magnetorheological fluid-elastic helicopter lag damper","volume":"30","author":"W Hu","year":"2007","journal-title":"J Guidance Control Dyn"},{"issue":"6","key":"pone.0359102.ref012","first-page":"956","article-title":"Design, performance test and analysis on magnetorheological damper for earthquake mitigation","volume":"20","author":"Z-D Xu","year":"2012","journal-title":"Struct Control Health Monit"},{"issue":"9","key":"pone.0359102.ref013","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1177\/1045389X13476151","article-title":"Unsteady analysis for oscillatory flow of magnetorheological fluid dampers based on Bingham plastic and Herschel\u2013Bulkley models","volume":"24","author":"M Yu","year":"2013","journal-title":"J Intell Mater Syst Struct"},{"key":"pone.0359102.ref014","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1016\/j.jmmm.2016.10.006","article-title":"Development of models of the magnetorheological fluid damper","volume":"431","author":"YuB Kazakov","year":"2017","journal-title":"J Magnetism Magnetic Mater"},{"issue":"1","key":"pone.0359102.ref015","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1177\/1045389X03014001004","article-title":"Intelligent Bi-State control for the structure with magnetorheological dampers","volume":"14","author":"Z-D Xu","year":"2003","journal-title":"J Intell Mater Syst Struct"},{"issue":"1","key":"pone.0359102.ref016","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1088\/0964-1726\/12\/1\/309","article-title":"Semi-active control of structures incorporated with magnetorheological dampers using neural networks","volume":"12","author":"Z-D Xu","year":"2003","journal-title":"Smart Mater Struct"},{"issue":"11","key":"pone.0359102.ref017","doi-asserted-by":"crossref","first-page":"2050120","DOI":"10.1142\/S0219455420501205","article-title":"Design and development of smart semi active suspension for nonlinear rail vehicle vibration reduction","volume":"20","author":"SK Sharma","year":"2020","journal-title":"Int J Str Stab Dyn"},{"issue":"6","key":"pone.0359102.ref018","doi-asserted-by":"crossref","first-page":"1002","DOI":"10.1177\/0954406214542840","article-title":"A non-linear vehicle dynamics model for accurate representation of suspension kinematics","volume":"229","author":"PK Vaddi","year":"2014","journal-title":"Proc Inst Mech Eng Part C: J Mech Eng Sci"},{"key":"pone.0359102.ref019","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1080\/00423110903401905","article-title":"Development of kineto-dynamic quarter-car model for synthesis of a double wishbone suspension","volume":"49","author":"KP Balike","year":"2011","journal-title":"Vehicle Syst Dyn"},{"issue":"2","key":"pone.0359102.ref020","first-page":"240","article-title":"Fuzzy control of active suspension system using full car model","volume":"24","author":"P Senthilkumar","year":"2018","journal-title":"J Mechanika"},{"key":"pone.0359102.ref021","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1109\/TASE.2025.3639251","article-title":"Integrating torque vectoring and active suspension systems using a game theory-based control framework","volume":"23","author":"J Liang","year":"2026","journal-title":"IEEE Trans Automat Sci Eng"},{"issue":"10","key":"pone.0359102.ref022","article-title":"An integrated control framework for torque vectoring and active suspension system","volume":"37","author":"J Feng","year":"2024","journal-title":"Chin J Mech Eng"},{"issue":"10","key":"pone.0359102.ref023","doi-asserted-by":"crossref","DOI":"10.1002\/stc.3040","article-title":"Analysis of inertance and damping double\u2010skyhook control strategies for a semi\u2010active device combining an adjustable inerter and damper","volume":"29","author":"X Zhang","year":"2022","journal-title":"Struct Contr Health Monit"},{"key":"pone.0359102.ref024","first-page":"1","article-title":"Research on GA-LQR of automotive semi-active suspension based on MRD","volume":"11","author":"S Shao","year":"2021","journal-title":"J Modern Manuf Eng"},{"key":"pone.0359102.ref025","doi-asserted-by":"crossref","first-page":"141722","DOI":"10.24425\/bpasts.2022.141722","article-title":"Dynamic characteristics of underframe semi-active inerter-based suspended device for high-speed train based on LQR control","author":"Y Wang","year":"2022","journal-title":"Bull Polish Acad Sci Tech Sci"},{"issue":"10","key":"pone.0359102.ref026","doi-asserted-by":"crossref","first-page":"4991","DOI":"10.1109\/TCYB.2025.3596063","article-title":"Minimum operator-based data-driven sliding mode control for a magnetorheological fluid dual clutch","volume":"55","author":"M Hou","year":"2025","journal-title":"IEEE Trans Cybern"},{"key":"pone.0359102.ref027","doi-asserted-by":"crossref","first-page":"9978","DOI":"10.1109\/ACCESS.2020.2964116","article-title":"Semi-active suspension control based on deep reinforcement learning","volume":"8","author":"L Ming","year":"2020","journal-title":"IEEE Access"},{"key":"pone.0359102.ref028","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1561\/2200000071","article-title":"An introduction to deep reinforcement learning","volume":"11","author":"V Fran\u00e7ois-Lavet","year":"2018","journal-title":"J Found Trends Mach Learn"},{"key":"pone.0359102.ref029","first-page":"298","article-title":"From Reinforcement Learning to Deep Reinforcement Learning: An Overview","author":"F Agostinelli","year":"2018","journal-title":"International Conference on Braverman Readings in Machine Learning - Key Ideas from Inception to Current State. Boston, MA: NE Univ"},{"issue":"5","key":"pone.0359102.ref030","doi-asserted-by":"crossref","first-page":"3215","DOI":"10.1007\/s10462-020-09938-y","article-title":"A survey on multi-agent deep reinforcement learning: from the perspective of challenges and applications","volume":"54","author":"W Du","year":"2020","journal-title":"Artif Intell Rev"},{"issue":"7587","key":"pone.0359102.ref031","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"D Silver","year":"2016","journal-title":"Nature"},{"issue":"2","key":"pone.0359102.ref032","first-page":"49","article-title":"Review of application prospect of deep reinforcement learning in intelligent manufacturing","volume":"57","author":"S Kong","year":"2021","journal-title":"J Comput Eng Appl"},{"issue":"7","key":"pone.0359102.ref033","doi-asserted-by":"crossref","first-page":"3520","DOI":"10.3390\/app12073520","article-title":"Multi-agent deep Q network to enhance the reinforcement learning for delayed reward system","volume":"12","author":"K Kim","year":"2022","journal-title":"Appl Sci"},{"key":"pone.0359102.ref034","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1109\/SMC.2017.8122622","article-title":"A novel DDPG method with prioritized experience replay.","author":"Y Hou","year":"2017","journal-title":"2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Banff, Canada"},{"issue":"5","key":"pone.0359102.ref035","doi-asserted-by":"crossref","first-page":"7356","DOI":"10.1109\/TVT.2025.3629701","article-title":"Efficient deep reinforcement learning with expert demonstrations for human\u2013machine shared steering control under emergency obstacle avoidance conditions","volume":"75","author":"F Xu","year":"2026","journal-title":"IEEE Trans Veh Technol"},{"key":"pone.0359102.ref036","doi-asserted-by":"crossref","first-page":"114613","DOI":"10.1016\/j.knosys.2025.114613","article-title":"Sample-efficient backtrack temporal difference deep reinforcement learning","volume":"330","author":"Q Liu","year":"2025","journal-title":"Knowl-Based Syst"},{"key":"pone.0359102.ref037","unstructured":"Fujimoto S, van Hoof H, Meger D. Addressing function approximation error in actor-critic methods. In: 35th International Conference on Machine Learning (ICML). Stockholm, Sweden; 2018."},{"key":"pone.0359102.ref038","doi-asserted-by":"crossref","first-page":"66864","DOI":"10.1109\/ACCESS.2021.3076299","article-title":"A new high-precision general inverse model based on deep learning for magnetorheological damper","volume":"9","author":"Y Han","year":"2021","journal-title":"IEEE Access"},{"key":"pone.0359102.ref039","doi-asserted-by":"crossref","first-page":"65671","DOI":"10.1109\/ACCESS.2022.3184417","article-title":"Design of virtual reference feedforward controller for an active suspension system","volume":"10","author":"Y Jeong","year":"2022","journal-title":"IEEE Access"},{"issue":"11","key":"pone.0359102.ref040","doi-asserted-by":"crossref","DOI":"10.1177\/1687814018811193","article-title":"Influence of response time of magnetorheological valve in Skyhook controlled three-parameter damping system","volume":"10","author":"Z Strecker","year":"2018","journal-title":"Adv Mech Eng"},{"issue":"1","key":"pone.0359102.ref041","doi-asserted-by":"crossref","first-page":"824","DOI":"10.1109\/TIE.2022.3153805","article-title":"Suspension control strategies using switched soft actor-critic models for real roads","volume":"70","author":"H Yong","year":"2023","journal-title":"IEEE Trans Ind Electron"},{"issue":"1","key":"pone.0359102.ref042","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1109\/TVT.2022.3207510","article-title":"Deep reinforcement learning of semi-active suspension controller for vehicle ride comfort","volume":"72","author":"D Lee","year":"2023","journal-title":"IEEE Trans Veh Technol"},{"issue":"8","key":"pone.0359102.ref043","doi-asserted-by":"crossref","first-page":"2586","DOI":"10.1109\/TFUZZ.2026.3695547","article-title":"Fuzzy game-theoretic tube model predictive control for integrated vehicle stability system","volume":"34","author":"G Cai","year":"2026","journal-title":"IEEE Trans Fuzzy Syst"},{"key":"pone.0359102.ref044","doi-asserted-by":"crossref","first-page":"106999","DOI":"10.1016\/j.ymssp.2020.106999","article-title":"Response time of magnetorheological dampers to current inputs in a semi-active suspension system: Modeling, control and sensitivity analysis","volume":"146","author":"D-S Yoon","year":"2021","journal-title":"Mech Syst Signal Process"},{"key":"pone.0359102.ref045","volume-title":"ISO 8608: Mechanical vibration-Road surface profiles-Reporting of measured data","year":"2016"},{"issue":"1","key":"pone.0359102.ref046","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1520\/JTE20160265","article-title":"Simulated road profiles according to ISO 8608 in vibration analysis","volume":"46","author":"P Mucka","year":"2018","journal-title":"J Test Eval"},{"key":"pone.0359102.ref047","author":"LH Zong","year":"2013"},{"key":"pone.0359102.ref048","volume-title":"ISO 2631-1:1997\/Amd 1:2010, Mechanical vibration and shock \u2014 Evaluation of human exposure to whole-body vibration \u2014 Part 1: General requirements \u2014 Amendment 1"},{"issue":"2","key":"pone.0359102.ref049","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1016\/S0022-460X(02)01567-5","article-title":"Low order continuous-time filters for approximation of the ISO 2631-1 human vibration sensitivity weightings","volume":"265","author":"L Zuo","year":"2003","journal-title":"J Sound Vibr"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0359102","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T17:32:08Z","timestamp":1790357528000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0359102"}},"subtitle":[],"editor":[{"given":"Jinhao","family":"Liang","sequence":"first","affiliation":[{"name":"National University of Singapore","place":["SINGAPORE"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,25]]},"references-count":49,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,25]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0359102","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,25]]}}}