{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T01:06:04Z","timestamp":1779066364109,"version":"3.51.4"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T00:00:00Z","timestamp":1775433600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T00:00:00Z","timestamp":1779062400000},"content-version":"vor","delay-in-days":42,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"DGAPA UNAM","award":["PAPIIT IT101524"],"award-info":[{"award-number":["PAPIIT IT101524"]}]},{"name":"DGAPA UNAM","award":["PAPIIT TA102424"],"award-info":[{"award-number":["PAPIIT TA102424"]}]},{"name":"DGAPA UNAM","award":["PAPIME PE108624"],"award-info":[{"award-number":["PAPIME PE108624"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Intell Robot Syst"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Hidden Markov models (HMMs) have found myriad uses. When employed in robotics, however, a conspicuous difference compared with finite-state machine emerges: HMMs lack a direct way of incorporating inputs. Although a sequence of observations can be viewed as input, we argue that it is desirable to extend HMMs with inputs in the vein of finite-state machines. As robot controllers, HMMs exhibit another salient feature: such models primarily focus on capturing uncertainty and stochasticity in the operational environment while maintaining deterministic actions based on the robot\u2019s belief state. We propose extendin HMMs in this other direction as well, embracing uncertainty also in the reactive behavior of the robot, resulting in stochastic behavior, thus making the robot act differently under the same conditions. We argue that this produces rich information, often sufficient for effective navigation without explicit maps. In addition, this approach significantly reduces the computational burden on the agent and simultaneously mitigates undesirable behaviors arising from deterministic policies under uncertainty of environmental perception. An\n                    <jats:italic>input output extended HMM<\/jats:italic>\n                    (IOEHMM) can be regarded as a Moore machine with stochastic inputs and outputs. We build IOEHMM robot behaviors using a genetic algorithm to find the best models and compare this approach with robot behavior implemented with a Markov decision process (MDP), whose parameters are also found by a genetic algorithm. Our method outperforms MDPs in cases where obstacles are large or the robot is used for exploration.\n                  <\/jats:p>","DOI":"10.1007\/s10846-026-02370-4","type":"journal-article","created":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T12:55:21Z","timestamp":1775480121000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Stochastic Robot Behaviors through Hidden Markov Models with Inputs and Outputs"],"prefix":"10.1007","volume":"112","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6340-8375","authenticated-orcid":false,"given":"Jesus","family":"Savage","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8933-8267","authenticated-orcid":false,"given":"David A.","family":"Rosenblueth","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6259-1609","authenticated-orcid":false,"given":"Stan","family":"Mu\u00f1oz Guti\u00e9rrez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5468-2807","authenticated-orcid":false,"given":"Marco","family":"Negrete","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2026,4,6]]},"reference":[{"issue":"1","key":"2370_CR1","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1109\/JRA.1986.1087032","volume":"2","author":"R Brooks","year":"1986","unstructured":"Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Automat. 2(1), 14\u201323 (1986)","journal-title":"IEEE J. Robot. Automat."},{"key":"2370_CR2","unstructured":"Arkin, R.C.: Behavior-based robotics (MIT press, 1998)"},{"issue":"5","key":"2370_CR3","doi-asserted-by":"publisher","first-page":"1045","DOI":"10.1002\/j.1538-7305.1955.tb03788.x","volume":"34","author":"GH Mealy","year":"1955","unstructured":"Mealy, G.H.: A method for synthesizing sequential circuits. The Bell System Technical Journal 34(5), 1045\u20131079 (1955)","journal-title":"The Bell System Technical Journal"},{"key":"2370_CR4","first-page":"129","volume":"34","author":"EF Moore","year":"1956","unstructured":"Moore, E.F.: Gedanken-experiments on sequential machines. Automata studies 34, 129\u2013153 (1956)","journal-title":"Gedanken-experiments on sequential machines. Automata studies"},{"issue":"7","key":"2370_CR5","doi-asserted-by":"publisher","first-page":"1013","DOI":"10.1109\/TPAMI.2005.147","volume":"27","author":"E Vidal","year":"2005","unstructured":"Vidal, E., Thollard, F., De La Higuera, C., Casacuberta, F., Carrasco, R.C.: Probabilistic finite-state machines\u2013Part I. IEEE Trans. Pattern Anal. Mach. Intell. 27(7), 1013\u20131025 (2005)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"issue":"7","key":"2370_CR6","doi-asserted-by":"publisher","first-page":"1026","DOI":"10.1109\/TPAMI.2005.148","volume":"27","author":"E Vidal","year":"2005","unstructured":"Vidal, E., Thollard, F., De La Higuera, C., Casacuberta, F., Carrasco, R.C.: Probabilistic finite-state machines\u2013Part II. IEEE Trans. Pattern Anal. Mach. Intell. 27(7), 1026\u20131039 (2005)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"issue":"3","key":"2370_CR7","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1145\/504729.504754","volume":"45","author":"S Thrun","year":"2002","unstructured":"Thrun, S.: Probabilistic robotics. Commun. ACM 45(3), 52\u201357 (2002)","journal-title":"Commun. ACM"},{"issue":"3","key":"2370_CR8","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1007\/s42979-020-00185-0","volume":"1","author":"M Mohanan","year":"2020","unstructured":"Mohanan, M., Salgaonkar, A.: Probabilistic approach to robot motion planning in dynamic environments. SN Computer Science 1(3), 181 (2020)","journal-title":"SN Computer Science"},{"key":"2370_CR9","first-page":"427","volume":"7","author":"Y Bengio","year":"1994","unstructured":"Bengio, Y., Frasconi, P.: An input output HMM architecture. Adv. Neural. Inf. Process. Syst. 7, 427\u2013434 (1994)","journal-title":"Adv. Neural. Inf. Process. Syst."},{"issue":"1","key":"2370_CR10","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s10514-024-10187-z","volume":"49","author":"E Scheide","year":"2025","unstructured":"Scheide, E., Best, G., Hollinger, G.A.: Synthesizing compact behavior trees for probabilistic robotics domains. Auton. Robot. 49(1), 3 (2025)","journal-title":"Auton. Robot."},{"key":"2370_CR11","doi-asserted-by":"crossref","unstructured":"Savage, J., Munoz, S., Contreras-Toledo, L.A., Matamoros, M., Negrete, M., Rivera, C., Steinbabuer, G., Fuentes, O., Okada, H.: Generating Reactive Robots\u2019 Behaviors using Genetic Algorithms. In: ICAART (2) (2021), pp. 698\u2013707","DOI":"10.5220\/0010229306980707"},{"issue":"5","key":"2370_CR12","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1109\/72.536317","volume":"7","author":"Y Bengio","year":"1996","unstructured":"Bengio, Y., Frasconi, P.: Input-output HMM\u2019s for sequence processing. IEEE Trans. Neural Networks 7(5), 1231\u20131249 (1996)","journal-title":"IEEE Trans. Neural Networks"},{"issue":"3","key":"2370_CR13","doi-asserted-by":"publisher","first-page":"390","DOI":"10.1109\/70.88149","volume":"7","author":"Q Zhu","year":"1991","unstructured":"Zhu, Q.: Hidden Markov model for dynamic obstacle avoidance of mobile robot navigation. IEEE Trans. Robot. Autom. 7(3), 390\u2013397 (1991)","journal-title":"IEEE Trans. Robot. Autom."},{"key":"2370_CR14","doi-asserted-by":"crossref","unstructured":"Won, K.J., Hamelryck, T., Prugel-Bennett, A., Krogh, A.: Evolving hidden Markov models for protein secondary structure prediction. In: 2005 IEEE Congress on Evolutionary Computation, vol. 1 (IEEE, 2005), pp. 33\u201340","DOI":"10.1109\/CEC.2005.1554664"},{"key":"2370_CR15","doi-asserted-by":"crossref","unstructured":"Hanheide, M., G\u00f6belbecker, M., Horn, G.S., Pronobis, A., Sj\u00f6\u00f6, K., Aydemir, A., Jensfelt, P., Gretton, C., Dearden, R., Janicek, M., Zender, H., Kruijff, G.J., Hawes, N., Wyatt, J.L.: Robot task planning and explanation in open and uncertain worlds. Artif. Intell. 247, 119\u2013150 (2017). Special Issue on AI and Robotics","DOI":"10.1016\/j.artint.2015.08.008"},{"key":"2370_CR16","doi-asserted-by":"crossref","unstructured":"G\u00f6belbecker, M., Gretton, C., Dearden, R.: A Switching Planner for Combined Task and Observation Planning. In: Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence (AAAI Press, 2011), pp. 964\u2013970","DOI":"10.1609\/aaai.v25i1.8011"},{"key":"2370_CR17","doi-asserted-by":"crossref","unstructured":"Lv, S., Li, Y., Liu, Q., Gao, J., Pang, X., Chen, M.: A deep safe reinforcement learning approach for mapless navigation. In: 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO) (IEEE, 2021), pp. 1520\u20131525","DOI":"10.1109\/ROBIO54168.2021.9739251"},{"key":"2370_CR18","doi-asserted-by":"crossref","unstructured":"Zhou, Q., Lyu, L., Liu, H.: Deep reinforcement learning with long-time memory capability for robot mapless navigation. In: 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD) (IEEE, 2022), pp. 1215\u20131220","DOI":"10.1109\/CSCWD54268.2022.9776137"},{"key":"2370_CR19","doi-asserted-by":"publisher","DOI":"10.1016\/j.conengprac.2024.105991","volume":"150","author":"C Cheng","year":"2024","unstructured":"Cheng, C., Zhang, H., Sun, Y., Tao, H., Chen, Y.: A cross-platform deep reinforcement learning model for autonomous navigation without global information in different scenes. Control. Eng. Pract. 150, 105991 (2024)","journal-title":"Control. Eng. Pract."},{"key":"2370_CR20","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/j.neunet.2020.12.001","volume":"135","author":"J Wang","year":"2021","unstructured":"Wang, J., Elfwing, S., Uchibe, E.: Modular deep reinforcement learning from reward and punishment for robot navigation. Neural Netw. 135, 115\u2013126 (2021)","journal-title":"Neural Netw."},{"key":"2370_CR21","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1016\/j.neunet.2023.06.007","volume":"165","author":"H Li","year":"2023","unstructured":"Li, H., Luo, B., Song, W., Yang, C.: Predictive hierarchical reinforcement learning for path-efficient mapless navigation with moving target. Neural Netw. 165, 677\u2013688 (2023)","journal-title":"Neural Netw."},{"key":"2370_CR22","doi-asserted-by":"crossref","unstructured":"Gao, Y., Wu, J., Yang, X., Ji, Z.: Efficient hierarchical reinforcement learning for mapless navigation with predictive neighbouring space scoring. IEEE Trans. Automat. Sci. Eng. (2023)","DOI":"10.1109\/TASE.2023.3312237"},{"key":"2370_CR23","unstructured":"Bishop, C.M.: Mixture density networks. Tech. Rep. Neural Computing Research Group: NCRG\/94\/004, Dept. of Computer Science and Applied Mathematics, Aston University (1994)"},{"key":"2370_CR24","unstructured":"Ambrogioni, L., G\u00fc\u00e7l\u00fc, U., van Gerven, M.A., Maris, E.: The kernel mixture network: A nonparametric method for conditional density estimation of continuous random variables. arXiv preprint arXiv:1705.07111 (2017)"},{"key":"2370_CR25","unstructured":"Alday-Salazar, G., Gonz\u00e1lez-Jim\u00e9nez, I., Negrete, M.: Deep neural networks for grasp planning in domestic service robots. In: Handbook of Intelligent Robots: Theory, Methods and Applications. Mart\u00ednez-Carranza, J., Guzman-Zavaleta, Z., Negrete, M., Sossa-Azuela, H. (eds.), (Taylor & Francis CRC) Accepted for publication"},{"issue":"2","key":"2370_CR26","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1109\/TIT.1967.1054010","volume":"13","author":"A Viterbi","year":"1967","unstructured":"Viterbi, A.: Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans. Inf. Theory 13(2), 260\u2013269 (1967)","journal-title":"IEEE Trans. Inf. Theory"},{"issue":"1","key":"2370_CR27","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/MASSP.1986.1165342","volume":"3","author":"L Rabiner","year":"1986","unstructured":"Rabiner, L., Juang, B.: An introduction to hidden Markov models. IEEE ASSP Mag. 3(1), 4\u201316 (1986)","journal-title":"IEEE ASSP Mag."},{"issue":"1","key":"2370_CR28","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1109\/TCOM.1980.1094577","volume":"28","author":"Y Linde","year":"1980","unstructured":"Linde, Y., Buzo, A., Gray, R.: An algorithm for vector quantizer design. IEEE Trans. Commun. 28(1), 84\u201395 (1980)","journal-title":"IEEE Trans. Commun."},{"key":"2370_CR29","doi-asserted-by":"crossref","unstructured":"Ibe, O.: Markov Processes for Stochastic Modeling (Newnes, 2013)","DOI":"10.1016\/B978-0-12-407795-9.00015-3"},{"issue":"3","key":"2370_CR30","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1561\/2400000045","volume":"6","author":"Y Zhou","year":"2024","unstructured":"Zhou, Y., Ma, S.: Stochastic optimization methods for policy evaluation in reinforcement learning. Foundations and Trends in Optimization 6(3), 145\u2013192 (2024)","journal-title":"Foundations and Trends in Optimization"},{"key":"2370_CR31","unstructured":"Montenegro, A., Mussi, M., Metelli, A.M., Papini, M.: Learning Optimal Deterministic Policies with Stochastic Policy Gradients. In: International Conference on Machine Learning (PMLR, 2024), pp. 36160\u201336211 (2024)"},{"key":"2370_CR32","doi-asserted-by":"crossref","unstructured":"Nolfi, S., Bongard, J., Husbands, P., Floreano, D.: Evolutionary robotics. In: Handbook of Robotics. Siciliano, B., Khatib, O. (eds.), 2nd edn. (Springer, 2016), chap. 76, pp. 2035\u20132067 (2016)","DOI":"10.1007\/978-3-319-32552-1_76"},{"key":"2370_CR33","doi-asserted-by":"crossref","unstructured":"K\u00f6nig, L.: Complex Behavior in Evolutionary Robotics (Walter de Gruyter GmbH & Co KG, 2015) (2015)","DOI":"10.1515\/9783110408553"},{"key":"2370_CR34","unstructured":"Banzhaf, W., Nordin, P., Olmer, M.: Generating adaptive behavior using function regression within genetic programming and a real robot. In: 2nd International Conference on Genetic Programming (Citeseer, 1997), pp. 35\u201343 (1997)"},{"key":"2370_CR35","unstructured":"Seok, H.S., Lee, K.J., Zhang, B.T.: An on-line learning method for object-locating robots using genetic programming on evolvable hardware. In: 5th International Symposium on Artificial Life and Robotics, vol. 1 (Citeseer, 2000), pp. 321\u2013324 (2000)"},{"key":"2370_CR36","doi-asserted-by":"crossref","unstructured":"Nelson, A.L., Barlow, G.J., Doitsidis, L.: Fitness functions in evolutionary robotics: A survey and analysis. Robot. Auton. Syst. 57(4), 345\u2013370 (2009)","DOI":"10.1016\/j.robot.2008.09.009"},{"key":"2370_CR37","doi-asserted-by":"crossref","unstructured":"Szymanski, M., Winkler, L., Laneri, D., Schlachter, F., van Rossum, A.C., Schmickl, T., Thenius, R.: Symbricatorrtos: a flexible and dynamic framework for bio-inspired robot control systems and evolution. In: 2009 IEEE Congress on Evolutionary Computation (IEEE), pp. 3314\u20133321 (2009)","DOI":"10.1109\/CEC.2009.4983365"},{"key":"2370_CR38","doi-asserted-by":"crossref","unstructured":"Marocco, D., Floreano, D.: Active vision and feature selection in evolutionary behavioral systems. In: 7th International Conference on Simulation of Adaptive Behavior (EPFL), pp. 247\u2013255 (2002)","DOI":"10.7551\/mitpress\/3121.003.0042"},{"key":"2370_CR39","doi-asserted-by":"crossref","unstructured":"Zagal, J.C., Ruiz-del-Solar, J., Guerrero, P., Palma, R.: Evolving visual object recognition for legged robots. In: RoboCup 2003: Robot Soccer World Cup VII 7 (Springer), pp. 181\u2013191 (2004)","DOI":"10.1007\/978-3-540-25940-4_16"},{"key":"2370_CR40","doi-asserted-by":"crossref","unstructured":"Nolfi, S., Floreano, D.: Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines (MIT press) (2000)","DOI":"10.7551\/mitpress\/2889.001.0001"},{"key":"2370_CR41","unstructured":"Kuri, A., Villegas, C.: A universal eclectic genetic algorithm for constrained optimization. In: 6th European Congress on Intelligent Techniques and Soft Computing, 1, 518\u2013522 (1998)"},{"issue":"9","key":"2370_CR42","doi-asserted-by":"publisher","first-page":"1349","DOI":"10.1016\/j.patcog.2004.03.020","volume":"38","author":"P Dupont","year":"2005","unstructured":"Dupont, P., Denis, F., Esposito, Y.: Links between probabilistic automata and hidden Markov models: probability distributions, learning models and induction algorithms. Pattern Recogn. 38(9), 1349\u20131371 (2005)","journal-title":"Pattern Recogn."},{"key":"2370_CR43","doi-asserted-by":"crossref","unstructured":"Zhang, L., Hermanns, H., Jansen, D.N.: Logic and model checking for hidden Markov models. In: Formal Techniques for Networked and Distributed Systems-FORTE 2005: 25th IFIP WG 6.1 International Conference, Taipei, Taiwan, October 2\u20135, 2005. Proceedings 25 (Springer), pp. 98\u2013112 (2005)","DOI":"10.1007\/11562436_9"},{"issue":"4","key":"2370_CR44","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1080\/10618600.2020.1743295","volume":"29","author":"J Buckby","year":"2020","unstructured":"Buckby, J., Wang, T., Zhuang, J., Obara, K.: Model checking for hidden Markov models. J. Comput. Graph. Stat. 29(4), 859\u2013874 (2020)","journal-title":"J. Comput. Graph. Stat."},{"issue":"11","key":"2370_CR45","doi-asserted-by":"publisher","first-page":"1036","DOI":"10.3390\/machines13111036","volume":"13","author":"L Marseglia","year":"2025","unstructured":"Marseglia, L., Vale, A., Di Gironimo, G.: Genetic algorithm-based optimization of velocity profiles for multi-robot collision avoidance. Machines 13(11), 1036 (2025)","journal-title":"Machines"},{"key":"2370_CR46","doi-asserted-by":"publisher","DOI":"10.1016\/j.swevo.2025.102215","volume":"99","author":"R Ghanem","year":"2025","unstructured":"Ghanem, R., Ali, I.M., Kasmarik, K., Garratt, M.: Simulation-based multi-objective optimization for mission specific tuning of swarming robots. Swarm Evol. Comput. 99, 102215 (2025)","journal-title":"Swarm Evol. Comput."}],"container-title":["Journal of Intelligent &amp; Robotic Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10846-026-02370-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10846-026-02370-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10846-026-02370-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T00:09:04Z","timestamp":1779062944000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10846-026-02370-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,6]]},"references-count":46,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["2370"],"URL":"https:\/\/doi.org\/10.1007\/s10846-026-02370-4","relation":{},"ISSN":["1573-0409"],"issn-type":[{"value":"1573-0409","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,6]]},"assertion":[{"value":"2 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 February 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 April 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"There are no conflicts of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of Interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}}],"article-number":"56"}}