{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T19:36:02Z","timestamp":1771270562193,"version":"3.50.1"},"reference-count":45,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2022,11,18]],"date-time":"2022-11-18T00:00:00Z","timestamp":1668729600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We consider the problem of dimensionality reduction of state space in the variational approach to the optimal control problem, in particular, in the reinforcement learning method. The control problem is described by differential algebraic equations consisting of nonlinear differential equations and algebraic constraint equations interconnected with Lagrange multipliers. The proposed method is based on changing the Lagrange multipliers of one subset based on the Lagrange multipliers of another subset. We present examples of the application of the proposed method in robotics and vibration isolation in transport vehicles. The method is implemented in FRUND\u2014a multibody system dynamics software package.<\/jats:p>","DOI":"10.3390\/s22228920","type":"journal-article","created":{"date-parts":[[2022,11,18]],"date-time":"2022-11-18T06:22:28Z","timestamp":1668752548000},"page":"8920","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Optimal Greedy Control in Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5458-2240","authenticated-orcid":false,"given":"Alexander","family":"Gorobtsov","sequence":"first","affiliation":[{"name":"Higher Mathematics Department, Volgograd State Technical University, Lenin Ave, 28, Volgograd 400005, Russia"},{"name":"Mechanical Engineering Research Institute, Russian Academy of Sciences, Maly Kharitonyevsky Pereulok, 4, Moscow 101990, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7296-2538","authenticated-orcid":false,"given":"Oleg","family":"Sychev","sequence":"additional","affiliation":[{"name":"Software Engineering Department, Volgograd State Technical University, Lenin Ave, 28, Volgograd 400005, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4854-7462","authenticated-orcid":false,"given":"Yulia","family":"Orlova","sequence":"additional","affiliation":[{"name":"Software Engineering Department, Volgograd State Technical University, Lenin Ave, 28, Volgograd 400005, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8382-5299","authenticated-orcid":false,"given":"Evgeniy","family":"Smirnov","sequence":"additional","affiliation":[{"name":"Higher Mathematics Department, Volgograd State Technical University, Lenin Ave, 28, Volgograd 400005, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6268-9486","authenticated-orcid":false,"given":"Olga","family":"Grigoreva","sequence":"additional","affiliation":[{"name":"Higher Mathematics Department, Volgograd State Technical University, Lenin Ave, 28, Volgograd 400005, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5774-093X","authenticated-orcid":false,"given":"Alexander","family":"Bochkin","sequence":"additional","affiliation":[{"name":"Higher Mathematics Department, Volgograd State Technical University, Lenin Ave, 28, Volgograd 400005, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7816-3833","authenticated-orcid":false,"given":"Marina","family":"Andreeva","sequence":"additional","affiliation":[{"name":"Higher Mathematics Department, Volgograd State Technical University, Lenin Ave, 28, Volgograd 400005, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,18]]},"reference":[{"key":"ref_1","unstructured":"Bellman, R. (2010). Dynamic Programming, Princeton University Press. Princeton Landmarks in Mathematics and Physics."},{"key":"ref_2","unstructured":"Pontryagin, L. (1987). Mathematical Theory of Optimal Processes, Taylor & Francis. Classics of Soviet Mathematics."},{"key":"ref_3","unstructured":"Heess, N., Dhruva, T., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., and Eslami, S.M.A. (2017). Emergence of Locomotion Behaviours in Rich Environments. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Tassa, Y., Erez, T., and Todorov, E. (2012, January 7\u201312). Synthesis and stabilization of complex behaviors through online trajectory optimization. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, IROS 2012, Vilamoura, Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6386025"},{"key":"ref_5","unstructured":"Schulman, J., Moritz, P., Levine, S., Jordan, M.I., and Abbeel, P. (2016, January 2\u20134). High-Dimensional Continuous Control Using Generalized Advantage Estimation. Proceedings of the 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico."},{"key":"ref_6","unstructured":"Schulman, J., Levine, S., Moritz, P., Jordan, M.I., and Abbeel, P. (2015). Trust Region Policy Optimization. arXiv."},{"key":"ref_7","unstructured":"JMLR.org, Balcan, M., and Weinberger, K.Q. (2016). Benchmarking Deep Reinforcement Learning for Continuous Control. JMLR Workshop and Conference Proceedings, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York, NY, USA, 19\u201324 June 2016, Microtome Publishing."},{"key":"ref_8","unstructured":"Pelikan, M., and Branke, J. (2010, January 7\u201311). Investigating EA solutions for approximate KKT conditions in smooth problems. Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2010, Portland, OR, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Fu, J., Li, C., Teng, X., Luo, F., and Li, B. (2020). Compound Heuristic Information Guided Policy Improvement for Robot Motor Skill Acquisition. Appl. Sci., 10.","DOI":"10.3390\/app10155346"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cho, N.J., Lee, S.H., Kim, J.B., and Suh, I.H. (2020). Learning, Improving, and Generalizing Motor Skills for the Peg-in-Hole Tasks Based on Imitation Learning and Self-Learning. Appl. Sci., 10.","DOI":"10.3390\/app10082719"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1016\/j.neunet.2008.02.003","article-title":"Reinforcement learning of motor skills with policy gradients","volume":"21","author":"Peters","year":"2008","journal-title":"Neural Netw."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3420","DOI":"10.1109\/TCYB.2018.2844177","article-title":"Adaptive Tracking Control of Surface Vessel Using Optimized Backstepping Technique","volume":"49","author":"Wen","year":"2019","journal-title":"IEEE Trans. Cybern."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1016\/j.isatra.2022.03.027","article-title":"Disturbance observer-based adaptive reinforcement learning for perturbed uncertain surface vessels","volume":"130","author":"Pham","year":"2022","journal-title":"ISA Trans."},{"key":"ref_14","unstructured":"Sutton, R., and Barto, A.G. (2020). Reinforcement Learning, MIT Press."},{"key":"ref_15","unstructured":"Gorobtsov, A.S., Karcov, S.K., Pletnev, A.E., and Polyakov, Y.A. (2011). Komp\u2019yuternye Metody Postroeniya i Issledovaniya Matematicheskih Modelej Dinamiki Konstrukcij Avtomobilej, Nauchno-Tekhnicheskoe Izdatel\u2019stvo \u201cMashinostroenie\u201d. (In Russian)."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1023\/A:1019131212618","article-title":"Differential\u2013algebraic equations in multibody system modeling","volume":"19","author":"Pogorelov","year":"1998","journal-title":"newblock Numerical Algorithms"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wittenburg, J. (1977). Dynamics of Systems of Rigid Bodies, Vieweg+Teubner Verlag. Leitf\u00e4den der Angewandten Mathematik und Mechanik.","DOI":"10.1007\/978-3-322-90942-8"},{"key":"ref_18","unstructured":"Gorobtsov, A.S., Skorikov, A.V., Tarasov, P.S., Markov, A., and Dianskij, A. (2021, January 27). Metod sinteza programmnogo dvizheniya robotov s uchetom zadannyh ogranichenij reakcij v svyazyah. Proceedings of the XIII Vserosijskaia Nauchno Tekhnicheskaia Konferencia s Mezhdunarodnym Uchastiem \u201cRobototekhnika i Iskusstvennyj Intellekt\u201d, Krasnoyarsk, Russia. (In Russian)."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"9262","DOI":"10.1016\/j.ifacol.2020.12.2378","article-title":"Underactuated mechanical systems: Whether orbital stabilization is an adequate assignment for a controller design?","volume":"53","author":"Mamedov","year":"2020","journal-title":"IFAC-PapersOnLine"},{"key":"ref_20","unstructured":"(2022, October 24). FRUND\u2014A System for Solving Non-Linear Dynamic Equations. Available online: http:\/\/frund.vstu.ru\/."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1145\/5948.5950","article-title":"Legged Robots","volume":"29","author":"Raibert","year":"1986","journal-title":"Commun. ACM"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1163\/156855306777361622","article-title":"Experimental realization of dynamic walking of the biped humanoid robot KHR-2 using zero moment point feedback and inertial measurement","volume":"20","author":"Kim","year":"2006","journal-title":"Adv. Robot."},{"key":"ref_23","first-page":"85","article-title":"Features of solving the inverse dynamic method equations for the synthesis of stable walking robots controlled motion","volume":"18","author":"Gorobtsov","year":"2019","journal-title":"Inform. Autom."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Englsberger, J., Werner, A., Ott, C., Henze, B., Roa, M.A., Garofalo, G., Burger, R., Beyer, A., Eiberger, O., and Schmid, K. (2014, January 18\u201320). Overview of the torque-controlled humanoid robot TORO. Proceedings of the 14th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2014, Madrid, Spain.","DOI":"10.1109\/HUMANOIDS.2014.7041473"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Englsberger, J., Ott, C., and Albu-Sch\u00e4ffer, A. (2013, January 3\u20137). Three-dimensional bipedal walking control using Divergent Component of Motion. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696723"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Pratt, J.E., Carff, J., Drakunov, S.V., and Goswami, A. (2006, January 4\u20136). Capture Point: A Step toward Humanoid Push Recovery. Proceedings of the 2006 6th IEEE-RAS International Conference on Humanoid Robots, Genova, Italy.","DOI":"10.1109\/ICHR.2006.321385"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Englsberger, J., Koolen, T., Bertrand, S., Pratt, J.E., Ott, C., and Albu-Sch\u00e4ffer, A. (2014, January 14\u201318). Trajectory generation for continuous leg forces during double support and heel-to-toe shift based on divergent component of motion. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6943128"},{"key":"ref_28","first-page":"130","article-title":"Swing Leg Trajectory Optimization for a Humanoid Robot Locomotion","volume":"Volume 2","author":"Gusikhin","year":"2016","journal-title":"Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2016)"},{"key":"ref_29","first-page":"141","article-title":"Toward a Human-like Locomotion: Modelling Dynamically Stable Locomotion of an Anthropomorphic Robot in Simulink Environment","volume":"Volume 2","author":"Filipe","year":"2015","journal-title":"Proceedings of the ICINCO 2015\u201412th International Conference on Informatics in Control, Automation and Robotics"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"182","DOI":"10.2991\/jrnal.2016.3.3.9","article-title":"Bipedal robot locomotion modelling with virtual height inverted pendulum and preview control approaches in Simulink environment","volume":"3","author":"Khusainov","year":"2016","journal-title":"J. Robot. Netw. Artif. Life"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Khusainov, R., Afanasyev, I., and Magid, E. (2016, January 29). Anthropomorphic robot modelling with virtual height inverted pendulum approach in Simulink: Step length and robot height influence on walking stability. Proceedings of the ICAROB 2016\u2014International Conference on Artificial Life and Robotics, Okinawa Convention Center, Ginowan, Japan.","DOI":"10.5954\/ICAROB.2016.GS9-2"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1206","DOI":"10.1109\/TSMC.2012.2235426","article-title":"Central Pattern Generator Inspired Control for Adaptive Walking of Biped Robots","volume":"43","author":"Liu","year":"2013","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_33","unstructured":"Glazunov, V. (2018). Mekhanizmy Parallel\u2019noj Struktury i ih Primenenie: Robototekhnicheskie, Tekhnologicheskie. Medicinskie, Obuchayushchie Sistemy, Izhevskij Institut Komp\u2019yuternyh Issledovanij. (In Russian)."},{"key":"ref_34","first-page":"428","article-title":"Manipulyacionnye mekhanizmy parallel\u2019noj struktury i ih prilozheniya v sovremennoj tekhnike","volume":"459","author":"Ganiev","year":"2014","journal-title":"Dokl. Akad. Nauk."},{"key":"ref_35","unstructured":"Glazunov, V.A. (2020). Mekhanizmy Perspektivnyh Robototekhnicheskih Sistem, Tekhnosfera. (In Russian)."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Qi, Q., Lin, W., Guo, B., Chen, J., Deng, C., Lin, G., Sun, X., and Chen, Y. (2022). Augmented Lagrangian-Based Reinforcement Learning for Network Slicing in IIoT. Electronics, 11.","DOI":"10.3390\/electronics11203385"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kamikokuryo, K., Haga, T., Venture, G., and Hernandez, V. (2022). Adversarial Autoencoder and Multi-Armed Bandit for Dynamic Difficulty Adjustment in Immersive Virtual Reality for Rehabilitation: Application to Hand Movement. Sensors, 22.","DOI":"10.3390\/s22124499"},{"key":"ref_38","unstructured":"Pontryagin, L.S., Boltyanskij, V.G., Gamkrelidze, R.V., and Mishchenko, E.F. (1961). Matematicheskaya Teoriya Optimalnih Processov, Fizmatgiz. (In Russian)."},{"key":"ref_39","unstructured":"Bellman, R. (1957). Dynamic Programming, Princeton University Press. [1st ed.]."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"657","DOI":"10.17587\/mau.17.657-669","article-title":"Metody AKAR i AKOR v zadachah sinteza nelinejnyh sistem upravleniya","volume":"17","author":"Kolesnikov","year":"2016","journal-title":"Mekhatronika Avtomatizaciya Upravlenie"},{"key":"ref_41","first-page":"38","article-title":"Umen\u2019shenie amplitudy kolebanij rezonansnyh sistem putem upravlyaemogo izmeneniya parametrov","volume":"3","author":"Frolov","year":"1965","journal-title":"Mashinovedenie"},{"key":"ref_42","unstructured":"Dmitriev, A.A. (1976). Teoriya i Raschet Nelinejnyh Sistem Podressorivaniya Gusenichnyh Mashin, Mashinostroenie. (In Russian)."},{"key":"ref_43","unstructured":"Gorobtsov, A. (1982, January 18\u201320). Issledovanie vozmozhnostej sistemy vibrozashchity so stupenchato izmenyayushchimisya parametrami. Proceedings of the IV Vsesoyuz. Simpozium Vliyanie Vibracii na Organizm Cheloveka i Problemy Vibrozashchity, Moscow, Russia. (In Russian)."},{"key":"ref_44","unstructured":"Karnopp, D., and Rosenberg, R. (1968). Analysis and Simulation of Multiport Systems: The Bond Graph Approach to Physical System Dynamics, MIT Press."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Vukobratovic, M., Stoki\u0107, D., and Kir\u0107anski, N. (1985). Non-Adaptive and Adaptive Control of Manipulation Robots, Springer.","DOI":"10.1007\/978-3-642-82201-8"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8920\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:20:56Z","timestamp":1760145656000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8920"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,18]]},"references-count":45,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22228920"],"URL":"https:\/\/doi.org\/10.3390\/s22228920","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,18]]}}}