{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T14:44:07Z","timestamp":1765377847166,"version":"3.46.0"},"reference-count":27,"publisher":"ASME International","issue":"2","license":[{"start":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T00:00:00Z","timestamp":1765324800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62103298"],"award-info":[{"award-number":["62103298"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,2,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>In the field of complex industrial control, the leveling task of multi-cylinder hydraulic presses imposes stringent requirements on control accuracy and system stability. Traditional control methods struggle to balance the performance and stability when facing unknown models due to their reliance on precise system modeling. In contrast, reinforcement learning optimizes policies through autonomous interaction, achieving both model-agnostic capability and multi-scenario adaptability. Based on the simplified dynamic model of the multi-cylinder hydraulic press, this study proposes a new control strategy based on reinforcement learning. The approach integrates the soft actor\u2013critic (SAC) algorithm with Lyapunov constraints and state-error integral compensation for leveling control. Embedding Lyapunov constraints within SAC ensures system stability, while the integral compensation minimizes steady-state error and enhances precision. Experimental results demonstrate that\u2014under simplified modeling assumptions\u2014the proposed method retains SAC\u2019s inherent advantages while significantly improving stability and leveling accuracy in specific complex scenarios (e.g., model-defined disturbances). By merging classical control theory with modern machine learning, this work offers new insights for designing reinforcement-learning controllers in complex settings and establishes a foundation for future validation on more realistic physical models. It aims to provide a reference for potential industrial deployment.<\/jats:p>","DOI":"10.1115\/1.4070439","type":"journal-article","created":{"date-parts":[[2025,11,20]],"date-time":"2025-11-20T17:33:55Z","timestamp":1763660035000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":0,"title":["Leveling Control of Multi-Cylinder Hydraulic Press: A Deep Reinforcement Learning Approach Based on Integral Compensation and Lyapunov Constraints"],"prefix":"10.1115","volume":"26","author":[{"given":"Chao","family":"Jia","sequence":"first","affiliation":[{"id":[{"id":"https:\/\/ror.org\/00zbe0w13","id-type":"ROR","asserted-by":"publisher"}],"name":"Tianjin University of Technology School of Electrical Engineering and Automation, , No. 391 Binshui West Road, Xiqing District, \u00a0 , ;","place":["Tianjin, China, 300384"]},{"name":"Tianjin University of Technology Tianjin Key Laboratory of New Energy Power Conversion Transmission and Intelligent Control, , No. 391 Binshui West Road, Xiqing District, \u00a0 ,","place":["Tianjin, China, 300384"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Yu","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/00zbe0w13","id-type":"ROR","asserted-by":"publisher"}],"name":"Tianjin University of Technology School of Electrical Engineering and Automation, , No. 391 Binshui West Road, Xiqing District, \u00a0 , ;","place":["Tianjin, China, 300384"]},{"name":"Tianjin University of Technology Tianjin Key Laboratory of New Energy Power Conversion Transmission and Intelligent Control, , No. 391 Binshui West Road, Xiqing District, \u00a0 ,","place":["Tianjin, China, 300384"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"33","published-online":{"date-parts":[[2025,12,10]]},"reference":[{"issue":"10","key":"2025121009395594000_CIT0001","doi-asserted-by":"publisher","first-page":"8023","DOI":"10.1109\/TIE.2017.2694382","article-title":"Active Disturbance Rejection Adaptive Control of Hydraulic Servo Systems","volume":"64","author":"Yao","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"issue":"4","key":"2025121009395594000_CIT0002","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1016\/j.cja.2020.07.007","article-title":"Vibration Analysis and Control Technologies of Hydraulic Pipeline System in Aircraft: A Review","volume":"34","author":"Gao","year":"2021","journal-title":"Chin. J. Aeronaut."},{"key":"2025121009395594000_CIT0003","doi-asserted-by":"publisher","first-page":"122540","DOI":"10.1016\/j.apenergy.2023.122540","article-title":"Optimal Power Distribution Control in Modular Power Architecture Using Hydraulic Free Piston Engines","volume":"358","author":"Fei","year":"2024","journal-title":"Appl. Energy"},{"key":"2025121009395594000_CIT0004","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1016\/j.ijheatmasstransfer.2017.10.015","article-title":"A Hierarchical Manifold Microchannel Heat Sink Array for High-Heat-Flux Two-Phase Cooling of Electronics","volume":"117","author":"Drummond","year":"2018","journal-title":"Int. J. Heat Mass Transfer"},{"issue":"A","key":"2025121009395594000_CIT0005","doi-asserted-by":"publisher","first-page":"472","DOI":"10.1016\/j.isatra.2021.12.044","article-title":"A New Adaptive Sliding Mode Controller Based on the Rbf Neural Network for an Electro-Hydraulic Servo System","volume":"129","author":"Feng","year":"2022","journal-title":"ISA Trans."},{"issue":"2","key":"2025121009395594000_CIT0006","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1109\/TPAMI.2013.102","article-title":"Half-Quadratic-Based Iterative Minimization for Robust Sparse Representation","volume":"36","author":"He","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"issue":"1","key":"2025121009395594000_CIT0007","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1109\/TSMC.2020.3042823","article-title":"A Survey on Intelligent Control for Multiagent Systems","volume":"51","author":"Shi","year":"2021","journal-title":"IEEE Trans. Syst. Man Cyber.-Syst."},{"key":"2025121009395594000_CIT0008","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1016\/j.automatica.2015.11.038","article-title":"Adaptive Continuous Higher Order Sliding Mode Control","volume":"65","author":"Edwards","year":"2016","journal-title":"Automatica"},{"key":"2025121009395594000_CIT0009","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1016\/j.buildenv.2013.11.016","article-title":"Theory and Applications of HVAC Control Systems - A Review of Model Predictive Control (MPC)","volume":"72","author":"Afram","year":"2014","journal-title":"Build. Environ."},{"issue":"1","key":"2025121009395594000_CIT0010","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/JAS.2023.123843","article-title":"Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications","volume":"11","author":"Wang","year":"2024","journal-title":"IEEE-Caa J. Auto. Sinica"},{"issue":"4","key":"2025121009395594000_CIT0011","doi-asserted-by":"publisher","first-page":"5064","DOI":"10.1109\/TNNLS.2022.3207346","article-title":"Deep Reinforcement Learning: A Survey","volume":"35","author":"Wang","year":"2024","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"issue":"9","key":"2025121009395594000_CIT0012","doi-asserted-by":"publisher","first-page":"091007","DOI":"10.1115\/1.4069035","article-title":"Medium Voltage Direct Current Shipboard Power Network Reconfiguration Using Graph-Based Reinforcement Learning","volume":"25","author":"Senemmar","year":"2025","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"2","key":"2025121009395594000_CIT0013","doi-asserted-by":"publisher","first-page":"787","DOI":"10.1016\/j.acha.2019.06.004","article-title":"Universality of Deep Convolutional Neural Networks","volume":"48","author":"Zhou","year":"2020","journal-title":"Appl. Comput. Harmonic Anal."},{"issue":"4","key":"2025121009395594000_CIT0014","doi-asserted-by":"publisher","first-page":"041006","DOI":"10.1115\/1.4067986","article-title":"Hybrid Physics-Infused One-Dimensional Convolutional Neural Network-Based Ensemble Learning Framework for Diesel Engine Fault Diagnostics","volume":"25","author":"Singh","year":"2025","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"4","key":"2025121009395594000_CIT0015","doi-asserted-by":"publisher","first-page":"2673","DOI":"10.1109\/TMECH.2023.3336070","article-title":"Data-Driven Control of Hydraulic Manipulators by Reinforcement Learning","volume":"29","author":"Yao","year":"2024","journal-title":"IEEE-Asme Trans. Mechatron."},{"issue":"12","key":"2025121009395594000_CIT0016","doi-asserted-by":"publisher","first-page":"1244","DOI":"10.3390\/machines10121244","article-title":"Reinforcement Learning Control of Hydraulic Servo System Based on TD3 Algorithm","volume":"10","author":"Yuan","year":"2022","journal-title":"Machines"},{"issue":"4","key":"2025121009395594000_CIT0017","doi-asserted-by":"publisher","first-page":"041006","DOI":"10.1115\/1.4064992","article-title":"Safe Reinforcement Learning-Based Balance Control for Multi-Cylinder Hydraulic Press","volume":"146","author":"Jia","year":"2024","journal-title":"ASME J. Dyn. Syst. Measure. Control"},{"issue":"10","key":"2025121009395594000_CIT0018","doi-asserted-by":"publisher","first-page":"3713","DOI":"10.1109\/TSMC.2018.2884725","article-title":"Deterministic Policy Gradient With Integral Compensator for Robust Quadrotor Control","volume":"50","author":"Wang","year":"2020","journal-title":"IEEE Trans. Syst. Man Cyber. Syst."},{"key":"2025121009395594000_CIT0019","doi-asserted-by":"publisher","first-page":"110999","DOI":"10.1016\/j.automatica.2023.110999","article-title":"Deep Reinforcement Learning Control Approach to Mitigating Actuator Attacks","volume":"152","author":"Wu","year":"2023","journal-title":"Automatica"},{"issue":"3","key":"2025121009395594000_CIT0020","doi-asserted-by":"publisher","first-page":"4757","DOI":"10.32604\/cmc.2024.055634","article-title":"Real-Time Implementation of Quadrotor UAV Control System Based on a Deep Reinforcement Learning Approach","volume":"81","author":"Trad","year":"2024","journal-title":"CMC-Comput. Mater. Continua"},{"issue":"10","key":"2025121009395594000_CIT0021","doi-asserted-by":"publisher","first-page":"12394","DOI":"10.1109\/TPEL.2023.3299979","article-title":"Stability-Oriented Multiobjective Control Design for Power Converters Assisted by Deep Reinforcement Learning","volume":"38","author":"Jiang","year":"2023","journal-title":"IEEE Trans. Power Electron."},{"issue":"5","key":"2025121009395594000_CIT0022","doi-asserted-by":"publisher","first-page":"3541","DOI":"10.1002\/asjc.3038","article-title":"Fault-Tolerant Control Strategy for Multi-Cylinder Hydraulic Press Machine Based on Dynamic Control Allocation and Adjustable Multi-Objective Optimization","volume":"25","author":"Jia","year":"2023","journal-title":"Asian J. Control"},{"key":"2025121009395594000_CIT0023","volume-title":"Automatic Control Principles","author":"Hu","year":"2007","edition":"5th ed."},{"key":"2025121009395594000_CIT0024","volume-title":"Nonlinear Systems","author":"Khalil","year":"2002","edition":"3rd ed."},{"key":"2025121009395594000_CIT0025","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.neucom.2022.05.006","article-title":"A Deep Reinforcement Learning Based Method for Real-Time Path Planning and Dynamic Obstacle Avoidance","volume":"497","author":"Chen","year":"2022","journal-title":"Neurocomputing"},{"issue":"2","key":"2025121009395594000_CIT0026","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10846-021-01367-5","article-title":"Soft Actor-Critic for Navigation of Mobile Robots","volume":"102","author":"de Jesus","year":"2021","journal-title":"J. Intell. Rob. Syst."},{"issue":"5","key":"2025121009395594000_CIT0027","doi-asserted-by":"publisher","first-page":"4706","DOI":"10.1109\/TVT.2022.3151651","article-title":"Highway Decision-Making and Motion Planning for Autonomous Driving Via Soft Actor-Critic","volume":"71","author":"Tang","year":"2022","journal-title":"IEEE Trans. Vehicular Technol."}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/26\/2\/021001\/7569668\/jcise-25-1241.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/26\/2\/021001\/7569668\/jcise-25-1241.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T14:40:03Z","timestamp":1765377603000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/26\/2\/021001\/1228373\/Leveling-Control-of-Multi-Cylinder-Hydraulic-Press"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,10]]},"references-count":27,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,2,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4070439","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"type":"print","value":"1530-9827"},{"type":"electronic","value":"1944-7078"}],"subject":[],"published":{"date-parts":[[2025,12,10]]},"article-number":"021001"}}