{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T02:58:41Z","timestamp":1780973921875,"version":"3.54.1"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T00:00:00Z","timestamp":1774569600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T00:00:00Z","timestamp":1774569600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000780","name":"European Commission","doi-asserted-by":"publisher","award":["739592"],"award-info":[{"award-number":["739592"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"European Commission","doi-asserted-by":"publisher","award":["739592"],"award-info":[{"award-number":["739592"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"European Commission","doi-asserted-by":"publisher","award":["RRF-2.3.1-21-2022-00002"],"award-info":[{"award-number":["RRF-2.3.1-21-2022-00002"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011019","name":"Nemzeti Kutat\u00e1si Fejleszt\u00e9si \u00e9s Innov\u00e1ci\u00f3s Hivatal","doi-asserted-by":"publisher","award":["ADVANCED 153390"],"award-info":[{"award-number":["ADVANCED 153390"]}],"id":[{"id":"10.13039\/501100011019","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011019","name":"Nemzeti Kutat\u00e1si Fejleszt\u00e9si \u00e9s Innov\u00e1ci\u00f3s Hivatal","doi-asserted-by":"publisher","award":["KKP 137490"],"award-info":[{"award-number":["KKP 137490"]}],"id":[{"id":"10.13039\/501100011019","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011019","name":"Nemzeti Kutat\u00e1si Fejleszt\u00e9si \u00e9s Innov\u00e1ci\u00f3s Hivatal","doi-asserted-by":"publisher","award":["KKP 137490"],"award-info":[{"award-number":["KKP 137490"]}],"id":[{"id":"10.13039\/501100011019","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003825","name":"Magyar Tudom\u00e1nyos Akad\u00e9mia","doi-asserted-by":"publisher","award":["J\u00e1nos Bolyai Research Scholarship"],"award-info":[{"award-number":["J\u00e1nos Bolyai Research Scholarship"]}],"id":[{"id":"10.13039\/501100003825","id-type":"DOI","asserted-by":"publisher"}]},{"name":"HUN-REN Institute for Computer Science and Control"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Math. Control Signals Syst."],"published-print":{"date-parts":[[2026,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>In this paper, we revisit a fundamental technical issue within the theory of stochastic approximation (SA) in a Markovian framework, first proposed in the book by Derevitskii and Fradkov (Applied theory of discrete adaptive control systems, Nauka, 1981), and further developed in much detail in the book by Benveniste, M\u00e9tivier, and Priouret (Adaptive algorithms and stochastic approximations, Springer, Berlin, 1990). This theory is instrumental in many application areas such as the statistical analysis of Hidden Markov Models arising in telecommunication, quantized linear stochastic systems, and reinforcement learning. The problem at hand is the verification of the existence, uniqueness, and Lipschitz continuity of the solution of a parameter-dependent Poisson equation, in an appropriate weighted sup-norm, associated with a collection of Markov chains on general state spaces. Verification of the above facts is vital in the analysis of SA processes presented in the cited book via the ODE (ordinary differential equations) method, requiring substantial technical effort. The motivation and focus of the paper is to address this technical issue, by presenting a simple set of conditions, under which the above properties of the Poisson equation at hand can be conveniently established. A distinctive feature of our work is that it is based on a remarkable result of Hairer and Mattingly (2011), proving that by tilting standard conditions of mainstream stability theory for Markov chains, the transition kernels prove to be contractions in the space of differences of probability measures in a suitable metric. To demonstrate the applicability of our results, the proposed conditions are verified for a class of queuing system with open-loop control.<\/jats:p>","DOI":"10.1007\/s00498-026-00436-0","type":"journal-article","created":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T18:09:47Z","timestamp":1774634987000},"page":"291-333","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Stochastic approximation in a Markovian framework revisited: Lipschitz continuity of the Poisson equation"],"prefix":"10.1007","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5658-8988","authenticated-orcid":false,"given":"Algo","family":"Car\u00e8","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7079-8343","authenticated-orcid":false,"given":"Bal\u00e1zs Csan\u00e1d","family":"Cs\u00e1ji","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3885-4146","authenticated-orcid":false,"given":"Bal\u00e1zs","family":"Gerencs\u00e9r","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4239-7950","authenticated-orcid":false,"given":"L\u00e1szl\u00f3","family":"Gerencs\u00e9r","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3105-4752","authenticated-orcid":false,"given":"Mikl\u00f3s","family":"R\u00e1sonyi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,3,27]]},"reference":[{"issue":"3","key":"436_CR1","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1214\/aoms\/1177729586","volume":"22","author":"H Robbins","year":"1951","unstructured":"Robbins H, Monro S (1951) A stochastic approximation method. Ann Math Stat 22(3):400\u2013407","journal-title":"Ann Math Stat"},{"key":"436_CR2","unstructured":"Derevitskii DP, Fradkov AL (1974) Application of the theory of Markov-processes to the analysis of the dynamics of adaptation algorithms. Autom Remote Control 2:39\u201348"},{"key":"436_CR3","unstructured":"Derevitskii DP, Fradkov AL (1981) Applied theory of discrete adaptive control systems. Nauka, Moscow"},{"key":"436_CR4","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-75894-2","volume-title":"Adaptive algorithms and stochastic approximations","author":"A Benveniste","year":"1990","unstructured":"Benveniste A, M\u00e9tivier M, Priouret P (1990) Adaptive algorithms and stochastic approximations, 2nd edn. Springer, Berlin","edition":"2"},{"issue":"3","key":"436_CR5","first-page":"279","volume":"8","author":"CJ Watkins","year":"1992","unstructured":"Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3):279\u2013292","journal-title":"Mach Learn"},{"issue":"1","key":"436_CR6","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1023\/A:1022633531479","volume":"3","author":"RS Sutton","year":"1988","unstructured":"Sutton RS (1988) Learning to predict by the methods of temporal differences. Mach Learn 3(1):9\u201344","journal-title":"Mach Learn"},{"key":"436_CR7","volume-title":"Theory and practice of recursive identification","author":"L Ljung","year":"1983","unstructured":"Ljung L, S\u00f6derstr\u00f6m T (1983) Theory and practice of recursive identification. MIT Press, Cambridge"},{"key":"436_CR8","doi-asserted-by":"publisher","unstructured":"Car\u00e8 A, Cs\u00e1ji B Cs, Gerencs\u00e9r B, Gerencs\u00e9r L, R\u00e1sonyi M (2019) Parameter-dependent Poisson equations: tools for stochastic approximation in a Markovian framework. In: IEEE 58th conference on decision and control (CDC), pp. 2259\u20132264 (2019). https:\/\/doi.org\/10.1109\/CDC40024.2019.9029383","DOI":"10.1109\/CDC40024.2019.9029383"},{"key":"436_CR9","unstructured":"Car\u00e8 A, Cs\u00e1ji B Cs, Gerencs\u00e9r B, Gerencs\u00e9r L, R\u00e1sonyi M (2019) Poisson equations, Lipschitz continuity and controlled queues. arXiv preprint arXiv:1906.09464"},{"key":"436_CR10","unstructured":"Cutkosky A, Orabona F (2018) Black-box reductions for parameter-free online learning in Banach spaces. In: Conference on learning theory (COLT), pp. 1493\u20131529. PMLR"},{"key":"436_CR11","unstructured":"Tam\u00e1s A, Cs\u00e1ji B Cs (2024) Recursive estimation of conditional kernel mean embeddings. J Mach Learn Res 25(264):1\u201335"},{"issue":"24","key":"436_CR12","first-page":"1","volume":"26","author":"SD Liu","year":"2025","unstructured":"Liu SD, Chen S, Zhang S (2025) The ODE method for stochastic approximation and reinforcement learning with Markovian noise. J Mach Learn Res 26(24):1\u201376","journal-title":"J Mach Learn Res"},{"key":"436_CR13","doi-asserted-by":"crossref","unstructured":"Gerencs\u00e9r L, Michaletzky Gy, Bokor J, Polcz P (2024) Notes on input design: from multi-sine design to data-driven procedures. IEEE Control Syst Lett 8:1943\u20131948","DOI":"10.1109\/LCSYS.2024.3416072"},{"issue":"4","key":"436_CR14","doi-asserted-by":"publisher","first-page":"551","DOI":"10.1109\/TAC.1977.1101561","volume":"22","author":"L Ljung","year":"1977","unstructured":"Ljung L (1977) Analysis of recursive stochastic algorithms. IEEE Trans Autom Control 22(4):551\u2013575","journal-title":"IEEE Trans Autom Control"},{"issue":"5","key":"436_CR15","doi-asserted-by":"publisher","first-page":"1200","DOI":"10.1137\/0330064","volume":"30","author":"L Gerencs\u00e9r","year":"1992","unstructured":"Gerencs\u00e9r L (1992) Rate of convergence of recursive estimators. SIAM J Control Optim 30(5):1200\u20131227","journal-title":"SIAM J Control Optim"},{"issue":"2","key":"436_CR16","doi-asserted-by":"publisher","first-page":"936","DOI":"10.1214\/24-AAP2132","volume":"35","author":"V Borkar","year":"2025","unstructured":"Borkar V, Chen S, Devraj A, Kontoyiannis I, Meyn S (2025) The ODE method for asymptotic statistics in stochastic approximation and reinforcement learning. Ann Appl Probab 35(2):936\u2013982","journal-title":"Ann Appl Probab"},{"key":"436_CR17","volume-title":"Stochastic approximation and recursive algorithms and applications","author":"HJ Kushner","year":"2003","unstructured":"Kushner HJ, Yin GG (2003) Stochastic approximation and recursive algorithms and applications. Springer, New York"},{"issue":"2","key":"436_CR18","doi-asserted-by":"publisher","first-page":"401","DOI":"10.2307\/3212261","volume":"5","author":"PJ Schweitzer","year":"1968","unstructured":"Schweitzer PJ (1968) Perturbation theory and finite Markov chains. J Appl Probab 5(2):401\u2013413","journal-title":"J Appl Probab"},{"key":"436_CR19","doi-asserted-by":"crossref","unstructured":"Hairer M, Mattingly JC (2011) Yet another look at Harris\u2019 ergodic theorem for Markov chains. In: Seminar on stochastic analysis, random fields and applications VI, pp. 109\u2013117. Springer","DOI":"10.1007\/978-3-0348-0021-1_7"},{"key":"436_CR20","volume-title":"Markov chains and stochastic stability","author":"SP Meyn","year":"2012","unstructured":"Meyn SP, Tweedie RL (2012) Markov chains and stochastic stability. Springer, London"},{"key":"436_CR21","doi-asserted-by":"crossref","unstructured":"Borkar V, Chen S, Devraj A, Kontoyiannis I, Meyn SP (2024) The ODE method for asymptotic statistics in stochastic approximation and reinforcement learning. arXiv preprint arXiv:2110.14427v6","DOI":"10.1214\/24-AAP2132"},{"key":"436_CR22","doi-asserted-by":"crossref","unstructured":"Finesso L, Gerencs\u00e9r L, Kmecs I (1999) Estimation of parameters from quantized noisy observations. In: 1999 European control conference (ECC), pp. 1648\u20131653 IEEE","DOI":"10.23919\/ECC.1999.7099550"},{"key":"436_CR23","doi-asserted-by":"crossref","unstructured":"Finesso L, Gerencs\u00e9r L, Kmecs I (1999) A randomized EM-algorithm for estimating quantized linear Gaussian regression. In: Proceedings of the 38th IEEE conference on decision and control (Cat. No. 99CH36304), vol. 5, pp. 5100\u20135101. IEEE","DOI":"10.1109\/CDC.1999.833359"},{"issue":"2","key":"436_CR24","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1109\/TAI.2021.3076021","volume":"2","author":"F Xia","year":"2021","unstructured":"Xia F, Sun K, Yu S, Aziz A, Wan L, Pan S, Liu H (2021) Graph learning: a survey. IEEE Trans Artif Intell 2(2):109\u2013127","journal-title":"IEEE Trans Artif Intell"},{"key":"436_CR25","volume-title":"Nonparametric functional data analysis: theory and practice","author":"F Ferraty","year":"2006","unstructured":"Ferraty F, Vieu P (2006) Nonparametric functional data analysis: theory and practice, vol 76. Springer, New York"},{"issue":"5","key":"436_CR26","doi-asserted-by":"publisher","first-page":"796","DOI":"10.1109\/TPAMI.2007.70735","volume":"30","author":"T Lin","year":"2008","unstructured":"Lin T, Zha H (2008) Riemannian manifold learning. IEEE Trans Pattern Anal Mach Intell 30(5):796\u2013809","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"436_CR27","doi-asserted-by":"publisher","DOI":"10.4171\/elm\/34","volume-title":"An introduction to singular stochastic PDEs: Allen-Cahn equations. Metastability and regularity structures","author":"N Berglund","year":"2022","unstructured":"Berglund N (2022) An introduction to singular stochastic PDEs: Allen-Cahn equations. Metastability and regularity structures. EMS Press, Berlin"},{"key":"436_CR28","volume-title":"Neuro-dynamic Programming","author":"D Bertsekas","year":"1996","unstructured":"Bertsekas D, Tsitsiklis JN (1996) Neuro-dynamic Programming. Athena Scientific, Belmont"},{"key":"436_CR29","doi-asserted-by":"publisher","DOI":"10.1109\/9780470544785","volume-title":"Handbook of learning and approximate dynamic programming","author":"J Si","year":"2004","unstructured":"Si J, Barto AG, Powell WB, Wunsch D (2004) Handbook of learning and approximate dynamic programming, vol 2. John Wiley & Sons, Hoboken, NJ"},{"key":"436_CR30","volume-title":"Reproducing Kernel Hilbert spaces in probability and statistics","author":"A Berlinet","year":"2011","unstructured":"Berlinet A, Thomas-Agnan C (2011) Reproducing Kernel Hilbert spaces in probability and statistics. Springer, New York"},{"key":"436_CR31","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-15142-3","volume-title":"Introduction to queueing systems with telecommunication applications","author":"L Lakatos","year":"2019","unstructured":"Lakatos L, Szeidl L, Telek M (2019) Introduction to queueing systems with telecommunication applications. Springer, Cham"},{"key":"436_CR32","doi-asserted-by":"publisher","first-page":"497","DOI":"10.1017\/S0305004100036781","volume":"58","author":"RM Loynes","year":"1962","unstructured":"Loynes RM (1962) The stability of a queue with non-independent inter-arrival and service times. Math Proc Cambridge Philos Soc 58:497\u2013520","journal-title":"Math Proc Cambridge Philos Soc"},{"key":"436_CR33","volume-title":"Egodicity and stability of stochastic processes","author":"AA Borovkov","year":"1998","unstructured":"Borovkov AA (1998) Egodicity and stability of stochastic processes. Wiley, New York"},{"issue":"1","key":"436_CR34","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1137\/S0036144598338446","volume":"41","author":"P Diaconis","year":"1999","unstructured":"Diaconis P, Freedman D (1999) Iterated random functions. SIAM Rev 41(1):45\u201376","journal-title":"SIAM Rev"},{"key":"436_CR35","unstructured":"Sparre\u00a0Andersen E (1957) On the collective theory of risk in the case of contagion between the claims. In: Transactions of the 15th international congress of actuaries, pp. 219\u2013229"}],"container-title":["Mathematics of Control, Signals, and Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00498-026-00436-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00498-026-00436-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00498-026-00436-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T02:22:31Z","timestamp":1780971751000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00498-026-00436-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,27]]},"references-count":35,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,6]]}},"alternative-id":["436"],"URL":"https:\/\/doi.org\/10.1007\/s00498-026-00436-0","relation":{},"ISSN":["0932-4194","1435-568X"],"issn-type":[{"value":"0932-4194","type":"print"},{"value":"1435-568X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,27]]},"assertion":[{"value":"30 September 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 January 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 March 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}