{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T23:16:15Z","timestamp":1776899775541,"version":"3.51.2"},"reference-count":27,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,5,2]],"date-time":"2024-05-02T00:00:00Z","timestamp":1714608000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>As an important mathematical model, the finite state machine (FSM) has been used in many fields, such as manufacturing system, health care, and so on. This paper analyzes the current development status of FSMs. It is pointed out that the traditional methods are often inconvenient for analysis and design, or encounter high computational complexity problems when studying FSMs.<\/jats:p><\/jats:sec><jats:sec><jats:title>Method<\/jats:title><jats:p>The deep Q-network (DQN) technique, which is a model-free optimization method, is introduced to solve the stabilization problem of probabilistic finite state machines (PFSMs). In order to better understand the technique, some preliminaries, including Markov decision process, \u03f5-greedy strategy, DQN, and so on, are recalled.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>First, a necessary and sufficient stabilizability condition for PFSMs is derived. Next, the feedback stabilization problem of PFSMs is transformed into an optimization problem. Finally, by using the stabilizability condition and deep Q-network, an algorithm for solving the optimization problem (equivalently, computing a state feedback stabilizer) is provided.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>Compared with the traditional Q learning, DQN avoids the limited capacity problem. So our method can deal with high-dimensional complex systems efficiently. The effectiveness of our method is further demonstrated through an illustrative example.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fncom.2024.1385047","type":"journal-article","created":{"date-parts":[[2024,5,2]],"date-time":"2024-05-02T11:14:10Z","timestamp":1714648450000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Feedback stabilization of probabilistic finite state machines based on deep Q-network"],"prefix":"10.3389","volume":"18","author":[{"given":"Hui","family":"Tian","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Su","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanfang","family":"Hou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,5,2]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"199254","DOI":"10.1109\/ACCESS.2020.3035152","article-title":"Double deep-Q learning-based output tracking of probabilistic Boolean control networks","volume":"8","author":"Acernese","year":"2020","journal-title":"IEEE Access"},{"key":"B2","doi-asserted-by":"publisher","first-page":"4066","DOI":"10.1109\/TCYB.2018.2856907","article-title":"Quasi-synchronization of delayed memristive neural networks via region-partitioning-dependent intermittent control","volume":"49","author":"Ding","year":"2019","journal-title":"IEEE Trans. Cybern"},{"key":"B3","doi-asserted-by":"publisher","first-page":"654","DOI":"10.1016\/j.microrel.2013.10.022","article-title":"A finite state machine based fault tolerance technique for sequential circuits","volume":"54","author":"El-Maleh","year":"2014","journal-title":"Microelectron. Reliab"},{"key":"B4","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1055\/s-0039-1688757","article-title":"Assistive conversational agent for health coaching: a validation study","volume":"58","author":"Fadhil","year":"2019","journal-title":"Methods Inf. Med"},{"key":"B5","doi-asserted-by":"publisher","first-page":"8642","DOI":"10.1016\/j.jfranklin.2018.09.009","article-title":"A matrix-based approach to verifying stability and synthesizing optimal stabilizing controllers for finite-state automata","volume":"355","author":"Han","year":"2018","journal-title":"J. Franklin Inst"},{"key":"B6","first-page":"628","article-title":"\u201cModeling of discrete dynamics for computational time reduction of model predictive control,\u201d","volume-title":"Proceedings of the 17th International Symposium on Mathematical Theory of Networks and Systems","author":"Kobayashi","year":"2006"},{"key":"B7","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1007\/978-3-540-71493-4_28","article-title":"\u201cMinimalilty of finite automata representation in hybrid systems control,\u201d","volume-title":"Hybrid Systems: Computation and Control","author":"Kobayashi","year":"2007"},{"key":"B8","doi-asserted-by":"publisher","first-page":"519","DOI":"10.1007\/s10626-011-0110-2","article-title":"Stabilization of finite automata with application to hybrid systems control","volume":"21","author":"Kobayashi","year":"2011","journal-title":"Discret. Event Dyn. Syst"},{"key":"B9","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1016\/j.asoc.2019.01.023","article-title":"A probabilistic finite state machine based strategy for multi-target search using swarm robotics","volume":"77","author":"Li","year":"2019","journal-title":"Appl. Soft Comput"},{"key":"B10","doi-asserted-by":"publisher","first-page":"729","DOI":"10.1145\/116825.116855","article-title":"Stability and stabilizability of discrete event dynamic systems","volume":"38","author":"\u00d6zveren","year":"1991","journal-title":"J. ACM"},{"key":"B11","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1109\/9.272323","article-title":"Lyapunov stability of a class of discrete event systems","volume":"39","author":"Passino","year":"1994","journal-title":"IEEE Trans. Automat. Contr"},{"key":"B12","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1016\/j.ifacol.2018.08.362","article-title":"Discrete event simulation for the reconfiguration of a flexible manufactuing plant","volume":"51","author":"Piccinini","year":"2018","journal-title":"IFAC-PapersOnLine"},{"key":"B13","doi-asserted-by":"publisher","first-page":"528","DOI":"10.1017\/S0269964818000451","article-title":"On deterministic finite state machines in random environments","volume":"33","author":"Ratsaby","year":"2019","journal-title":"Probab. Eng. Inf. Sci"},{"key":"B14","doi-asserted-by":"publisher","first-page":"e69","DOI":"10.2196\/jmir.7207","article-title":"Exacerbations in chronic obstructive pulmonary disease: identification and prediction using a digital health system","volume":"19","author":"Shah","year":"2017","journal-title":"J. Med. Internet Res"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1133","DOI":"10.1109\/TAC.2008.923658","article-title":"A framework for robust stability of systems over finite alphabets","volume":"53","author":"Tarraf","year":"2008","journal-title":"IEEE Trans. Automat. Contr"},{"key":"B16","doi-asserted-by":"publisher","first-page":"4358","DOI":"10.1016\/j.jfranklin.2018.12.027","article-title":"State feedback design for set stabilization of probabilistic boolean control networks","volume":"356","author":"Tian","year":"2019","journal-title":"J. Franklin Inst"},{"key":"B17","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1016\/j.automatica.2016.12.040","article-title":"Stabilization of k-valued logical control networks by open-loop control via the reverse-transfer method","volume":"83","author":"Tian","year":"2017","journal-title":"Automatica"},{"key":"B18","doi-asserted-by":"publisher","first-page":"106325","DOI":"10.1016\/j.aml.2020.106325","article-title":"A new multiple integral inequality and its application to stability analysis of time-delay systems","volume":"105","author":"Tian","year":"2020","journal-title":"Appl. Math. Lett"},{"key":"B19","doi-asserted-by":"publisher","first-page":"13","DOI":"10.26438\/ijcse\/v10i1.1317","article-title":"Simulation and testing of deterministic finite automata machine","volume":"10","author":"Vayadande","year":"2022","journal-title":"International Journal of Comput. Sci. Eng"},{"key":"B20","doi-asserted-by":"publisher","first-page":"1013","DOI":"10.1109\/TPAMI.2005.147","article-title":"Probabilistic finite-state machines - part I","volume":"27","author":"Vidal","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B21","doi-asserted-by":"publisher","first-page":"857","DOI":"10.1007\/s00170-016-9097-z","article-title":"Modeling of hot stamping process procedure based on finite state machine (FSM)","volume":"89","author":"Wang","year":"2017","journal-title":"Int. J. Adv. Manuf. Technol"},{"key":"B22","first-page":"3242","article-title":"\u201cMatrix approach to stabilizability of deterministic finite automata,\u201d","volume-title":"2013 American Control Conference","author":"Xu","year":"2013"},{"key":"B23","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1109\/JSEE.2015.00018","article-title":"Semi-tensor product approach to controllability and stabilizability of finite automata","volume":"26","author":"Yan","year":"","journal-title":"J. Syst. Eng. Electron"},{"key":"B24","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1016\/j.ifacol.2015.12.114","article-title":"Stp approach to controlliability of finite state machines","volume":"48","author":"Yan","year":"","journal-title":"IFAC-PapersOnLine"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12913-018-3456-4","article-title":"Application of discrete event simulation in health care: a systematic review","volume":"18","author":"Zhang","year":"2018","journal-title":"BMC Health Serv. Res"},{"key":"B26","doi-asserted-by":"publisher","first-page":"5173","DOI":"10.1016\/j.jfranklin.2020.02.028","article-title":"Stabilization of probabilistic finite automata based on semi-tensor product of matrices","volume":"357","author":"Zhang","year":"","journal-title":"J. Franklin Inst"},{"key":"B27","doi-asserted-by":"publisher","first-page":"124687","DOI":"10.1016\/j.amc.2019.124687","article-title":"On the stabilization of nondeterministic finite automata via static output feedback","volume":"365","author":"Zhang","year":"","journal-title":"Appl. Math. Comput"}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1385047\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,2]],"date-time":"2024-05-02T11:14:16Z","timestamp":1714648456000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1385047\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,2]]},"references-count":27,"alternative-id":["10.3389\/fncom.2024.1385047"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2024.1385047","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,2]]},"article-number":"1385047"}}