{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T17:46:08Z","timestamp":1789580768006,"version":"build-2803163510"},"reference-count":27,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,2,24]],"date-time":"2020-02-24T00:00:00Z","timestamp":1582502400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61573244, 61521063"],"award-info":[{"award-number":["61573244, 61521063"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>In this paper we study the optimization of the discrete-time stochastic linear-quadratic (LQ) control problem with conic control constraints on an infinite horizon, considering multiplicative noises. Stochastic control systems can be formulated as Markov Decision Problems (MDPs) with continuous state spaces and therefore we can apply the direct-comparison based optimization approach to solve the problem. We first derive the performance difference formula for the LQ problem by utilizing the state separation property of the system structure. Based on this, we successfully derive the optimality conditions and the stationary optimal feedback control. By introducing the optimization, we establish a general framework for infinite horizon stochastic control problems. The direct-comparison based approach is applicable to both linear and nonlinear systems. Our work provides a new perspective in LQ control problems; based on this approach, learning based algorithms can be developed without identifying all of the system parameters.<\/jats:p>","DOI":"10.3390\/a13020049","type":"journal-article","created":{"date-parts":[[2020,2,25]],"date-time":"2020-02-25T04:21:26Z","timestamp":1582604486000},"page":"49","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Optimization of Constrained Stochastic Linear-Quadratic Control on an Infinite Horizon: A Direct-Comparison Based Approach"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3745-2620","authenticated-orcid":false,"given":"Ruobing","family":"Xue","sequence":"first","affiliation":[{"name":"Department of Automation, Shanghai Jiaotong University, Shanghai 200240, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangshen","family":"Ye","sequence":"additional","affiliation":[{"name":"Department of Automation, Shanghai Jiaotong University, Shanghai 200240, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weiping","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Economics and Management, Fuzhou University, Fuzhou 350108, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1002\/rnc.1055","article-title":"Opitmal filtering for polynomial system wtates with polynomial multiplicative noise","volume":"16","author":"Basin","year":"2006","journal-title":"Int. J. Robust Nonlinear Control"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1016\/j.sysconle.2005.07.010","article-title":"Static H2 and Houtput-feedback of discrete-time LTI systems with state multiplicative noise","volume":"55","author":"Gershon","year":"2006","journal-title":"Syst. Control Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1359","DOI":"10.1109\/9.774108","article-title":"Stochastic optimal control LQR control with integral quadratic constraints and indefinite control weights","volume":"44","author":"Lim","year":"1999","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.sysconle.2004.07.003","article-title":"On stochastic riccati equations for the stochastic LQR problem","volume":"44","author":"Zhu","year":"2005","journal-title":"Syst. Control Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1137\/S0363012904441969","article-title":"Constrained stochastic LQ control with random coefficients, and application to portfolio selection","volume":"44","author":"Hu","year":"2005","journal-title":"SIAM J. Control Optim."},{"key":"ref_6","first-page":"102","article-title":"Contributions to the theory of optimal control","volume":"5","author":"Kalman","year":"1960","journal-title":"Bol. Soc. Mat. Mex."},{"key":"ref_7","unstructured":"Anderson, B.D., and Moore, J.B. (2007). Optimal Control: Linear Quadratic Methods, Courier Corporation."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2809","DOI":"10.1137\/120892477","article-title":"Linear-quadratic optimal control problems for mean-field stochastic differential equations","volume":"51","author":"Yong","year":"2013","journal-title":"SIAM J. Control Optim."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.automatica.2015.01.040","article-title":"Time cardinality constrained mean-variance dynamic portfolio selection and market timing: A stochastic control approach","volume":"54","author":"Gao","year":"2015","journal-title":"Automatica"},{"key":"ref_10","unstructured":"Costa, O.L.V., Fragoso, M.D., and Margues, R.P. (2007). Discrete-Time Markov Jump Linear Systems, Springer."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1109\/TAC.2008.2010886","article-title":"Stochastic receding horizon control of contrained linear systems with state and control multiplicative noise","volume":"54","author":"Primbs","year":"2009","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1007\/s11401-018-0099-z","article-title":"Constrained LQ problem with a random jump and application to portfolio selection","volume":"39","author":"Dong","year":"2019","journal-title":"Chin. Ann. Math."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1109\/TAC.2011.2140770","article-title":"Cardinality constrained linear quadratic optimal control","volume":"56","author":"Gao","year":"2011","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1999","DOI":"10.1109\/TAC.2018.2831641","article-title":"Explicit solution for constrained stochastic linear-quadratic control with multiplicative noise","volume":"64","author":"Wu","year":"2019","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"885","DOI":"10.1080\/00207178208932938","article-title":"On positive controllers and linear quadratic optimal control problems","volume":"36","author":"Campbell","year":"1982","journal-title":"Int. J. Control"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1080\/002071798222208","article-title":"Linear quadratic regulator problem with positive controls","volume":"70","author":"Heemels","year":"1998","journal-title":"Int. J. Control"},{"key":"ref_17","unstructured":"Cao, X.R. (2007). Stochastic Learning and Optimization: A Sensitivity-Based Approach, Springer."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Puterman, M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley.","DOI":"10.1002\/9780470316887"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Chen, R.C. (2004, January 14\u201317). Constrained stochastic control with probabilistic criteria and search optimization. Proceedings of the 43rd IEEE Conference on Decision and Control (CDC), Nassau, Bahamas.","DOI":"10.1109\/CDC.2004.1428926"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1016\/j.automatica.2007.08.014","article-title":"Policy iteration based feedback control","volume":"44","author":"Zhang","year":"2008","journal-title":"Automatica"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6136","DOI":"10.1109\/TAC.2017.2701412","article-title":"Stochastic feedback control with one-dimensional degenerate diffusions and nonsmooth value functions","volume":"62","author":"Cao","year":"2018","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1111\/mafi.12076","article-title":"Sensitivity analysis of nonlinear behavior with distorted probability","volume":"27","author":"Cao","year":"2017","journal-title":"Math. Financ."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.automatica.2017.11.012","article-title":"Mean-variance optimization of discrete time discounted Markov decision processes","volume":"88","author":"Xia","year":"2018","journal-title":"Automatica"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4318","DOI":"10.1109\/TAC.2017.2655487","article-title":"Optimality consitions for long-run average rewards with underselectivity and nonsmooth features","volume":"62","author":"Cao","year":"2017","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_25","unstructured":"Xue, R.B., Ye, X.S., and Cao, X.R. (2019). Optimization of stock trading with additional information by Limit Order Book. Automatica, submitted."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1016\/j.automatica.2018.04.036","article-title":"Optimization in curbing risk contagion among financial institutes","volume":"94","author":"Ye","year":"2018","journal-title":"Automatica"},{"key":"ref_27","first-page":"32","article-title":"A tutorial on event-based optimization with application in energy Internet","volume":"35","author":"Jia","year":"2018","journal-title":"J. Control Theory Appl."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/13\/2\/49\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:01:17Z","timestamp":1760173277000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/13\/2\/49"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,24]]},"references-count":27,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["a13020049"],"URL":"https:\/\/doi.org\/10.3390\/a13020049","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,24]]}}}