{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:38:02Z","timestamp":1760060282115,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T00:00:00Z","timestamp":1755734400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The dynamics of signal transmission in neuronal networks remain incompletely understood. In this study, we propose a novel Rulkov neuronal network model that incorporates Q-learning, a reinforcement learning method, to establish efficient signal transmission pathways. Using a simulated neuronal network, we focused on a key parameter that modulates both the intrinsic dynamics of individual neurons and the input signals received from active neighbors. We investigated how variations in this parameter affect signal transmission efficiency by analyzing changes in attenuation rate, as well as the maximum and minimum firing intervals of the start and goal neurons. Our simulations revealed that signal transmission efficiency between distant neurons was significantly impaired in the parameter region, where a chaotic attractor and an attractor of the eight-periodic points are observed to co-exist. A key finding was that low-frequency oscillatory bursts, while failing long-distance transmission, were capable of amplifying signals in neighboring neurons. Furthermore, we observed variation in signal transmission even when individual neuron dynamics remained similar. This variability, despite similar presynaptic activity, is a biologically significant phenomenon, and it is argued that it may contribute to the flexibility and robustness of information processing. These findings are discussed in the context of their biological implications.<\/jats:p>","DOI":"10.3390\/e27080884","type":"journal-article","created":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T07:41:45Z","timestamp":1755848505000},"page":"884","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Investigation of Signal Transmission Dynamics in Rulkov Neuronal Networks with Q-Learned Pathways"],"prefix":"10.3390","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3200-2129","authenticated-orcid":false,"given":"Mio","family":"Kobayashi","sequence":"first","affiliation":[{"name":"Institute of Humanities, Shinshu University, 3-1-1 Asahi, Matsumoto 390-8621, Nagano, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/S0092-8240(05)80004-7","article-title":"A quantitative description of membrane current and its application to conduction and excitation in nerve","volume":"52","author":"Hodgkin","year":"1990","journal-title":"Bull. Math. Biol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1016\/0375-9601(82)90674-0","article-title":"Chaotic behavior in the Onchidium giant neuron under sinusoidal stimulation","volume":"88","author":"Hayashi","year":"1982","journal-title":"Phys. Lett. A"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1016\/S0022-5193(05)80676-9","article-title":"Chaotic nature of bursting discharges in the Onchidium pacemaker neuron","volume":"156","author":"Hayashi","year":"1992","journal-title":"J. Theor. Biol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1195","DOI":"10.1152\/physrev.00035.2008","article-title":"Neurophysiological and Computational Principles of Cortical Rhythms in Cognition","volume":"90","author":"Wang","year":"2010","journal-title":"Physiol. Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1016\/S0006-3495(61)86902-6","article-title":"Impulses and Physiological States in Theoretical Models of Nerve Membrane","volume":"1","author":"FitzHugh","year":"1961","journal-title":"Biophys. J."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hunt, B.R., Li, T.Y., Kennedy, J.A., and Nusse, H.E. (2004). Deterministic Nonperiodic Flow. The Theory of Chaotic Attractors, Springer.","DOI":"10.1007\/978-0-387-21830-4"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1569","DOI":"10.1007\/s11071-017-3565-3","article-title":"A review for dynamics in neuron and neuronal network","volume":"89","author":"Ma","year":"2017","journal-title":"Nonlinear Dyn."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1016\/0375-9601(90)90136-C","article-title":"Chaotic neural networks","volume":"144","author":"Aihara","year":"1990","journal-title":"Phys. Lett. A"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"041922","DOI":"10.1103\/PhysRevE.65.041922","article-title":"Modeling of spiking-bursting neural behavior using two-dimensional map","volume":"65","author":"Rulkov","year":"2002","journal-title":"Phys. Rev. E"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1103\/PhysRevLett.86.183","article-title":"Regularization of Synchronized Chaotic Bursts","volume":"86","author":"Rulkov","year":"2001","journal-title":"Phys. Rev. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1023\/B:JCNS.0000037683.55688.7e","article-title":"Oscillations in Large-Scale Cortical Networks: Map-Based Model","volume":"17","author":"Rulkov","year":"2004","journal-title":"J. Comput. Neurosci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2901","DOI":"10.1016\/j.jfranklin.2013.01.026","article-title":"Electronic circuit implementation of the chaotic Rulkov neuron model","volume":"350","author":"Wagemakers","year":"2013","journal-title":"J. Frankl. Inst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"155283","DOI":"10.1016\/j.aeue.2024.155283","article-title":"Firing behaviors of memristor-based Rulkov neuron map using energy method","volume":"178","author":"Liu","year":"2024","journal-title":"AEU - Int. J. Electron. Commun."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"114466","DOI":"10.1016\/j.chaos.2024.114466","article-title":"A discrete Chialvo\u2013Rulkov neuron network coupled with a novel memristor model: Design, Dynamical analysis, DSP implementation and its application","volume":"179","author":"Cao","year":"2024","journal-title":"Chaos Solitons Fractals"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"114800","DOI":"10.1016\/j.chaos.2024.114800","article-title":"A joint image encryption based on a memristive Rulkov neuron with controllable multistability and compressive sensing","volume":"182","author":"Li","year":"2024","journal-title":"Chaos Solit. Fractals"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/j.strueco.2022.02.003","article-title":"Simulating heterogeneous corporate dynamics via the Rulkov map","volume":"61","author":"Orlando","year":"2022","journal-title":"Struct. Chang. Econ. Dyn."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"536","DOI":"10.1016\/j.cnsns.2014.06.015","article-title":"Stability and chaos of Rulkov map-based neuron network with electrical synapse","volume":"20","author":"Wang","year":"2015","journal-title":"Commun. Nonlinear Sci. Numer. Simul."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"116012","DOI":"10.1016\/j.chaos.2025.116012","article-title":"Dynamics and synchronization of fractional-order Rulkov neuron coupled with discrete fracmemristor","volume":"192","author":"Tang","year":"2025","journal-title":"Chaos Solitons Fractals"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"107383","DOI":"10.1016\/j.cnsns.2023.107383","article-title":"Multistability and stochastic dynamics of Rulkov neurons coupled via a chemical synapse","volume":"125","author":"Bashkirtseva","year":"2023","journal-title":"Commun. Nonlinear Sci. Numer. Simul."},{"key":"ref_20","first-page":"279","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zanatta, L., Barchi, F., Manoni, S., Tolu, S., Bartolini, A., and Acquaviva, A. (2024). Exploring spiking neural networks for deep reinforcement learning in robotic tasks. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-77779-8"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Park, Y., Lee, J., Sim, D., Cho, Y., and Park, C. (2025). Designing Spiking Neural Network-Based Reinforcement Learning for 3D Robotic Arm Applications. Electronics, 14.","DOI":"10.3390\/electronics14030578"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Li, S., Zhang, Q., and Lin, F. (2024, January 8\u201311). DEP-SNN-RL:Spiking Neural Networks Reinforcement Learning in Musculoskeletal Systems. Proceedings of the 2024 IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE International Conference on Robotics, Automation and Mechatronics (RAM), Hangzhou, China.","DOI":"10.1109\/CIS-RAM61939.2024.10672803"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"109415","DOI":"10.1016\/j.engappai.2024.109415","article-title":"Spiking neural networks for autonomous driving: A review","volume":"138","author":"Subirats","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/S0166-2236(96)10070-9","article-title":"Bursts as a unit of neural information: Making unreliable synapses reliable","volume":"20","author":"Lisman","year":"1997","journal-title":"Trends Neurosci."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1016\/S0166-2236(00)01714-8","article-title":"Tonic and burst firing: Dual modes of thalamocortical relay","volume":"24","author":"Sherman","year":"2001","journal-title":"Trends Neurosci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"14406","DOI":"10.1523\/JNEUROSCI.2487-13.2013","article-title":"Stochastic Properties of Neurotransmitter Release Expand the Dynamic Range of Synapses","volume":"33","author":"Yang","year":"2013","journal-title":"J. Neurosci."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e69884","DOI":"10.7554\/eLife.69884","article-title":"Presynaptic stochasticity improves energy efficiency and helps alleviate the stability-plasticity dilemma","volume":"10","author":"Schug","year":"2021","journal-title":"eLife"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1016\/S0361-9230(99)00161-6","article-title":"Lapicque\u2019s introduction of the integrate-and-fire model neuron (1907)","volume":"50","author":"Abbott","year":"1999","journal-title":"Brain Res. Bull."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1569","DOI":"10.1109\/TNN.2003.820440","article-title":"Simple model of spiking neurons","volume":"14","author":"Izhikevich","year":"2003","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_31","unstructured":"Hebb, D.O. (2025, July 27). The Organization of Behavior: A Neuropsychological Theory. Available online: https:\/\/archive.org\/details\/in.ernet.dli.2015.226341\/mode\/2up."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1016\/S0361-9230(99)00182-3","article-title":"D.O. Hebb: The Organization of Behavior, Wiley: New York; 1949","volume":"50","author":"Morris","year":"1999","journal-title":"Brain Res. Bull."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1017\/S0033583500002894","article-title":"Voltage gating of ion channels","volume":"27","author":"Sigworth","year":"1994","journal-title":"Q. Rev. Biophys."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1016\/S0896-6273(00)80981-2","article-title":"Voltage-gated ion channels and electrical excitability","volume":"20","author":"Armstrong","year":"1998","journal-title":"Neuron"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.cell.2006.09.014","article-title":"Dendritic Protein Synthesis, Synaptic Plasticity, and Memory","volume":"127","author":"Sutton","year":"2006","journal-title":"Cell"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/sj.npp.1301559","article-title":"Synaptic Plasticity: Multiple Forms, Functions, and Mechanisms","volume":"33","author":"Citri","year":"2008","journal-title":"Neuropsychopharmacology"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/8\/884\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:32:34Z","timestamp":1760034754000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/8\/884"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,21]]},"references-count":36,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8]]}},"alternative-id":["e27080884"],"URL":"https:\/\/doi.org\/10.3390\/e27080884","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2025,8,21]]}}}