{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T13:10:04Z","timestamp":1789564204889,"version":"build-2803163510"},"reference-count":72,"publisher":"AIP Publishing","issue":"9","license":[{"start":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T00:00:00Z","timestamp":1789516800000},"content-version":"vor","delay-in-days":15,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["LI 4412\/2-1"],"award-info":[{"award-number":["LI 4412\/2-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["pubs.aip.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,9,1]]},"abstract":"<jats:p>Life-threatening arrhythmias are the leading cause of sudden cardiac death. The standard clinical treatment involves delivering a high-energy defibrillation shock, which comes along with severe side effects like potential tissue damage or adverse psychological outcomes for patients with implantable cardioverter-defibrillators. We examine whether Reinforcement Learning (RL) can be used to derive low-energy pulse sequences to control the chaotic myocardial excitation dynamics during cardiac arrhythmias. We demonstrate that policies can be trained using numerical simulations and that the resulting multi-pulse protocols offer amplitude reductions of up to 34% compared to state-of-the-art multi-pulse methods. We further show that RL-trained policies increasingly focus on phase singularity dynamics throughout the pulse sequence and that established system observables can be used to explain RL-induced pulse timings. These observations indicate that RL-trained policies are more flexible than previously published protocols and may open up the way toward patient-specific defibrillation strategies.<\/jats:p>","DOI":"10.1063\/5.0346786","type":"journal-article","created":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T12:50:31Z","timestamp":1789563031000},"update-policy":"https:\/\/doi.org\/10.1063\/aip-crossmark-policy-page","source":"Crossref","is-referenced-by-count":0,"title":["Deep reinforcement learning to control cardiac arrhythmias via pulse sequences"],"prefix":"10.1063","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-0018-353X","authenticated-orcid":false,"given":"Daniel","family":"Fr\u00fchwald","sequence":"first","affiliation":[{"name":"Computational Physics for Life Science, Nuremberg Institute of Technology Georg Simon Ohm 1 , Ke\u00dflerplatz 12, Nuremberg 90486,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"Odefey","sequence":"additional","affiliation":[{"name":"Computational Physics for Life Science, Nuremberg Institute of Technology Georg Simon Ohm 1 , Ke\u00dflerplatz 12, Nuremberg 90486,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1117-6733","authenticated-orcid":false,"given":"Thomas","family":"Lilienkamp","sequence":"additional","affiliation":[{"name":"Computational Physics for Life Science, Nuremberg Institute of Technology Georg Simon Ohm 1 , Ke\u00dflerplatz 12, Nuremberg 90486,","place":["Germany"]},{"name":"Max Planck Institute for Dynamics and Self-Organization 2 , Am Fa\u00dfberg 17, G\u00f6ttingen 37077,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"317","published-online":{"date-parts":[[2026,9,16]]},"reference":[{"key":"2026091608502775000_c1","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1016\/j.hlc.2018.08.026","article-title":"Epidemiology of sudden cardiac death: Global and regional perspectives","volume":"28","year":"2019","journal-title":"Heart, Lung Circ."},{"key":"2026091608502775000_c2","doi-asserted-by":"publisher","first-page":"10F","DOI":"10.1016\/S0002-9149(97)00477-3","article-title":"Frequency of sudden cardiac death and profiles of risk","volume":"80","year":"1997","journal-title":"Am. J. Cardiol."},{"key":"2026091608502775000_c3","doi-asserted-by":"publisher","first-page":"S118","DOI":"10.1016\/j.jelectrocard.2007.06.023","article-title":"Global public health problem of sudden cardiac death","volume":"40","year":"2007","journal-title":"J. Electrocardiol."},{"key":"2026091608502775000_c4","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1038\/32164","article-title":"Spatial and temporal organization during cardiac fibrillation","volume":"392","year":"1998","journal-title":"Nature"},{"key":"2026091608502775000_c5","doi-asserted-by":"publisher","first-page":"043119","DOI":"10.1063\/1.4829632","article-title":"Shock-induced termination of reentrant cardiac arrhythmias: Comparing monophasic and biphasic shock protocols","volume":"23","year":"2013","journal-title":"Chaos"},{"key":"2026091608502775000_c6","doi-asserted-by":"publisher","first-page":"163","DOI":"10.5114\/amsad.2020.97103","article-title":"Implantable cardioverter defibrillators\u2013the past, present and future","volume":"5","year":"2020","journal-title":"Arch. Med. Sci.-Atheroscler. Dis."},{"key":"2026091608502775000_c7","doi-asserted-by":"publisher","first-page":"1947","DOI":"10.1056\/NEJMoa2207304","article-title":"Defibrillation strategies for refractory ventricular fibrillation","volume":"387","year":"2022","journal-title":"N. Engl. J. Med."},{"key":"2026091608502775000_c8","volume-title":"Updates on Cardiac Defibrillation, Cardioversion and AED Development","year":"2024"},{"key":"2026091608502775000_c9","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1161\/01.CIR.96.2.683","article-title":"High-energy defibrillation increases the severity of postresuscitation myocardial dysfunction","volume":"96","year":"1997","journal-title":"Circulation"},{"key":"2026091608502775000_c10","doi-asserted-by":"publisher","first-page":"N222","DOI":"10.1097\/00003246-200011001-00014","article-title":"Low-energy biphasic waveform defibrillation reduces the severity of postresuscitation myocardial dysfunction","volume":"28","year":"2000","journal-title":"Crit. Care Med."},{"key":"2026091608502775000_c11","doi-asserted-by":"publisher","first-page":"euad130","DOI":"10.1093\/europace\/euad130","article-title":"Burden of mood symptoms and disorders in implantable cardioverter defibrillator patients: A systematic review and meta-analysis of 39 954 patients","volume":"25","year":"2023","journal-title":"Europace"},{"key":"2026091608502775000_c12","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1161\/CIRCEP.110.957670","article-title":"Posttraumatic stress and the implantable cardioverter-defibrillator patient: What the electrophysiologist needs to know","volume":"4","year":"2011","journal-title":"Circ.: Arrhythmia Electrophysiol."},{"key":"2026091608502775000_c13","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1002\/clc.4960270604","article-title":"Panic disorders and agoraphobia: Side effects of treatment with an implantable cardioverter\/defibrillator","volume":"27","year":"2004","journal-title":"Clin. Cardiol."},{"key":"2026091608502775000_c14","doi-asserted-by":"publisher","first-page":"051108","DOI":"10.1063\/5.0011506","article-title":"Terminating transient chaos in spatially extended systems","volume":"30","year":"2020","journal-title":"Chaos"},{"key":"2026091608502775000_c15","doi-asserted-by":"publisher","first-page":"e2117568119","DOI":"10.1073\/pnas.2117568119","article-title":"Terminating spiral waves with a single designed stimulus: Teleportation as the mechanism for defibrillation","volume":"119","year":"2022","journal-title":"Proc. Natl. Acad. Sci."},{"key":"2026091608502775000_c16","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1113\/expphysiol.2005.030973","article-title":"Defibrillation of the heart: Insights into mechanisms from modelling studies","volume":"91","year":"2006","journal-title":"Exp. Physiol."},{"key":"2026091608502775000_c17","doi-asserted-by":"publisher","first-page":"1007585","DOI":"10.3389\/fnetp.2022.1007585","article-title":"The role of pulse timing in cardiac defibrillation","volume":"2","year":"2023","journal-title":"Front. Netw. Physiol."},{"key":"2026091608502775000_c18","doi-asserted-by":"publisher","first-page":"113110","DOI":"10.1063\/5.0222247","article-title":"Ultra-low-energy defibrillation through adjoint optimization","volume":"34","year":"2024","journal-title":"Chaos"},{"key":"2026091608502775000_c19","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1161\/CIRCULATIONAHA.108.825091","article-title":"Termination of atrial fibrillation using pulsed low-energy far-field stimulation","volume":"120","year":"2009","journal-title":"Circulation"},{"key":"2026091608502775000_c20","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1016\/j.jacc.2013.07.098","article-title":"Multistage electrotherapy delivered through chronically-implanted leads terminates atrial fibrillation with lower energy than a single biphasic shock","volume":"63","year":"2014","journal-title":"J. Am. Coll. Cardiol."},{"key":"2026091608502775000_c21","doi-asserted-by":"publisher","first-page":"113110","DOI":"10.1063\/1.5010787","article-title":"Control of electrical turbulence by periodic excitation of cardiac tissue","volume":"27","year":"2017","journal-title":"Chaos"},{"key":"2026091608502775000_c22","doi-asserted-by":"publisher","first-page":"170024","DOI":"10.1098\/rsos.170024","article-title":"Mechanisms of vortices termination in the cardiac muscle","volume":"4","year":"2017","journal-title":"R. Soc. Open. Sci."},{"key":"2026091608502775000_c23","doi-asserted-by":"publisher","first-page":"12043","DOI":"10.1038\/s41598-022-16068-8","article-title":"Non-monotonous dose response function of the termination of spiral wave chaos","volume":"12","year":"2022","journal-title":"Sci. Rep."},{"key":"2026091608502775000_c24","doi-asserted-by":"publisher","first-page":"121105","DOI":"10.1063\/5.0126682","article-title":"Taming cardiac arrhythmias: Terminating spiral wave chaos by adaptive deceleration pacing","volume":"32","year":"2022","journal-title":"Chaos"},{"key":"2026091608502775000_c25","doi-asserted-by":"publisher","first-page":"1572834","DOI":"10.3389\/fnetp.2025.1572834","article-title":"Success rates of simulated multi-pulse defibrillation protocols are sensitive to application timing with individual, protocol-specific optimal timings","volume":"5","year":"2025","journal-title":"Front. Netw. Physiol."},{"key":"2026091608502775000_c26","unstructured":"C. P.\n            Buran\n          , \u201cThe mechanism of fibrillation control by low-energy anti-fibrillation pacing,\u201d Doctoral thesis (Technische Universit\u00e4t Berlin, Inst. Theoretische Physik, Berlin, 2023).10.14279\/depositonce-19952"},{"key":"2026091608502775000_c27","doi-asserted-by":"publisher","first-page":"1401661","DOI":"10.3389\/fnetp.2024.1401661","article-title":"Chaos control in cardiac dynamics: Terminating chaotic states with local minima pacing","volume":"4","year":"2024","journal-title":"Front. Netw. Physiol."},{"key":"2026091608502775000_c28","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","year":"2015","journal-title":"Nature"},{"key":"2026091608502775000_c29","doi-asserted-by":"publisher","first-page":"1238","DOI":"10.1177\/0278364913495721","article-title":"Reinforcement learning in robotics: A survey","volume":"32","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"2026091608502775000_c30","doi-asserted-by":"publisher","first-page":"775","DOI":"10.1063\/1.166451","article-title":"Optimal chaos control through reinforcement learning","volume":"9","year":"1999","journal-title":"Chaos"},{"key":"2026091608502775000_c31","doi-asserted-by":"publisher","first-page":"116727","DOI":"10.1016\/j.chaos.2025.116727","article-title":"Generalized synchronization between two distinct chaotic systems through deep reinforcement learning","volume":"199","year":"2025","journal-title":"Chaos, Solitons Fractals"},{"key":"2026091608502775000_c32","doi-asserted-by":"publisher","first-page":"5415","DOI":"10.1007\/s12206-019-1036-0","article-title":"Automatic control of cardiac ablation catheter with deep reinforcement learning method","volume":"33","year":"2019","journal-title":"J. Mech. Sci. Technol."},{"key":"2026091608502775000_c33","doi-asserted-by":"publisher","first-page":"1418","DOI":"10.1111\/aor.14845","article-title":"Physiological control for left ventricular assist devices based on deep reinforcement learning","volume":"48","year":"2024","journal-title":"Artif. Organs"},{"key":"2026091608502775000_c34"},{"key":"2026091608502775000_c35","unstructured":"J.\n            Schulman\n          , F.Wolski, P.Dhariwal, A.Radford, and O.Klimov, \u201cProximal policy optimization algorithms,\u201d arXiv:1707.06347 (2017)."},{"key":"2026091608502775000_c36","volume-title":"Cambridge Studies in Mathematical Biology Introduction to Theoretical Neurobiology: Series Number 8: Linear Cable Theory and Dendritic Structure Volume 1","year":"1988"},{"key":"2026091608502775000_c37","volume-title":"Numerical Recipes 3rd Edition: The Art of Scientific Computing","year":"2007"},{"key":"2026091608502775000_c38","doi-asserted-by":"publisher","first-page":"767","DOI":"10.1016\/S0092-8240(03)00041-7","article-title":"A two-current model for the dynamics of cardiac membrane","volume":"65","year":"2003","journal-title":"Bull. Math. Biol."},{"key":"2026091608502775000_c39","doi-asserted-by":"publisher","first-page":"1770","DOI":"10.1016\/j.mcm.2011.11.025","article-title":"Shape reconstruction of cardiac ischemia from non-contact intracardiac recordings: A model study","volume":"55","year":"2012","journal-title":"Math. Comput. Model."},{"key":"2026091608502775000_c40","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1063\/1.166311","article-title":"Vortex dynamics in three-dimensional continuous myocardium with fiber rotation: Filament instability and fibrillation","volume":"8","year":"1998","journal-title":"Chaos"},{"key":"2026091608502775000_c41","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1016\/j.jtbi.2008.03.029","article-title":"Minimal model for human ventricular action potentials in tissue","volume":"253","year":"2008","journal-title":"J. Theor. Biol."},{"key":"2026091608502775000_c42","volume-title":"Reinforcement Learning: An Introduction","year":"1998"},{"key":"2026091608502775000_c43","first-page":"1","article-title":"Stable-baselines3: Reliable reinforcement learning implementations","volume":"22","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"2026091608502775000_c44"},{"key":"2026091608502775000_c45","doi-asserted-by":"publisher","first-page":"1172454","DOI":"10.3389\/fnetp.2023.1172454","article-title":"Optimising low-energy defibrillation in 2D cardiac tissue with a genetic algorithm","volume":"3","year":"2023","journal-title":"Front. Netw. Physiol."},{"key":"2026091608502775000_c46","doi-asserted-by":"publisher","first-page":"e82756","DOI":"10.7759\/cureus.82756","article-title":"Reinforcement learning in personalized medicine: A comprehensive review of treatment optimization strategies","volume":"17","year":"2025","journal-title":"Cureus"},{"key":"2026091608502775000_c47"},{"key":"2026091608502775000_c48","unstructured":"K.\n            Simonyan\n          , A.Vedaldi, and A.Zisserman, \u201cDeep inside convolutional networks: Visualising image classification models and saliency maps,\u201d arXiv:1312.6034 (2013)."},{"key":"2026091608502775000_c49"},{"key":"2026091608502775000_c50","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1038\/nature10216","article-title":"Low-energy control of electrical turbulence in the heart","volume":"475","year":"2011","journal-title":"Nature"},{"key":"2026091608502775000_c51"},{"key":"2026091608502775000_c52"},{"key":"2026091608502775000_c53","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/j.yjmcc.2015.11.018","article-title":"Uncertainty and variability in models of the cardiac action potential: Can we build trustworthy models?","volume":"96","year":"2016","journal-title":"J. Mol. Cell. Cardiol."},{"key":"2026091608502775000_c54","doi-asserted-by":"publisher","first-page":"941","DOI":"10.1063\/1.1497836","article-title":"Termination of spiral wave breakup in a Fitzhugh\u2013Nagumo model via short and long duration stimuli","volume":"12","year":"2002","journal-title":"Chaos"},{"key":"2026091608502775000_c55","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1016\/j.pbiomolbio.2007.07.004","article-title":"A guide to modelling cardiac electrical activity in anatomically detailed ventricles","volume":"96","year":"2008","journal-title":"Prog. Biophys. Mol. Biol."},{"key":"2026091608502775000_c56","doi-asserted-by":"publisher","first-page":"H1055","DOI":"10.1152\/ajpheart.2000.279.3.H1055","article-title":"Virtual electrode polarization in the far field: Implications for external defibrillation","volume":"279","year":"2000","journal-title":"Am. J. Physiol.-Heart Circ. Physiol."},{"key":"2026091608502775000_c57","doi-asserted-by":"publisher","first-page":"208101","DOI":"10.1103\/PhysRevLett.99.208101","article-title":"Wave emission from heterogeneities opens a way to controlling chaos in the heart","volume":"99","year":"2007","journal-title":"Phys. Rev. Lett."},{"key":"2026091608502775000_c58","doi-asserted-by":"publisher","first-page":"118106","DOI":"10.1103\/PhysRevLett.109.118106","article-title":"Negative curvature boundaries as wave emitting sites for the control of biological excitable media","volume":"109","year":"2012","journal-title":"Phys. Rev. Lett."},{"key":"2026091608502775000_c59","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1007\/s11071-025-11992-6","article-title":"Leading mechanisms of defibrillation: A computational approach to study differences between monophasic and biphasic waveforms","volume":"114","year":"2026","journal-title":"Nonlinear Dyn."},{"key":"2026091608502775000_c60","doi-asserted-by":"publisher","first-page":"1439","DOI":"10.1016\/S0002-9149(98)00210-0","article-title":"Frequency analysis of human atrial fibrillation using the surface electrocardiogram and its response to ibutilide","volume":"81","year":"1998","journal-title":"Am. J. Cardiol."},{"key":"2026091608502775000_c61","doi-asserted-by":"publisher","first-page":"H891","DOI":"10.1152\/ajpheart.1988.255.4.H891","article-title":"Influence of shock strength and timing on induction of ventricular arrhythmias in dogs","volume":"255","year":"1988","journal-title":"Am. J. Physiol.-Heart Circ. Physiol."},{"key":"2026091608502775000_c62","doi-asserted-by":"publisher","first-page":"068102","DOI":"10.1103\/PhysRevLett.123.068102","article-title":"Termination of scroll waves by surface impacts","volume":"123","year":"2019","journal-title":"Phys. Rev. Lett."},{"key":"2026091608502775000_c63","doi-asserted-by":"publisher","first-page":"021052","DOI":"10.1103\/PhysRevX.12.021052","article-title":"Spatiotemporal organization of electromechanical phase singularities during high-frequency cardiac arrhythmias","volume":"12","year":"2022","journal-title":"Phys. Rev. X"},{"key":"2026091608502775000_c64","doi-asserted-by":"publisher","first-page":"014404","DOI":"10.1103\/PhysRevE.105.014404","article-title":"Intermittent trapping of spiral waves in a cardiac model","volume":"105","year":"2022","journal-title":"Phys. Rev. E"},{"key":"2026091608502775000_c65","doi-asserted-by":"publisher","first-page":"093139","DOI":"10.1063\/5.0277113","article-title":"Particle model of creation and annihilation captures termination dynamics of spiral defect chaos","volume":"35","year":"2025","journal-title":"Chaos"},{"key":"2026091608502775000_c66","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1137\/100788379","article-title":"Electromechanical coupling in cardiac dynamics: The active strain approach","volume":"71","year":"2011","journal-title":"SIAM J. Appl. Math."},{"key":"2026091608502775000_c67","doi-asserted-by":"publisher","first-page":"e103","DOI":"10.1161\/CIRCRESAHA.107.161075","article-title":"Laminar arrangement of ventricular myocytes influences electrical behavior of the heart","volume":"101","year":"2007","journal-title":"Circ. Res."},{"key":"2026091608502775000_c68","doi-asserted-by":"publisher","first-page":"e60287","DOI":"10.1371\/journal.pone.0060287","article-title":"Effects of mechano-electric feedback on scroll wave stability in human ventricular fibrillation","volume":"8","year":"2013","journal-title":"PLoS One"},{"key":"2026091608502775000_c69","doi-asserted-by":"publisher","first-page":"044207","DOI":"10.1103\/PhysRevE.110.044207","article-title":"Influence of conduction heterogeneities on transient spatiotemporal chaos in cardiac excitable media","volume":"110","year":"2024","journal-title":"Phys. Rev. E"},{"key":"2026091608502775000_c70"},{"key":"2026091608502775000_c71","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1109\/TASE.2022.3144347","article-title":"Physics-constrained deep learning for robust inverse ECG modeling","volume":"20","year":"2023","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"2026091608502775000_c72","first-page":"73","article-title":"Replacement, reduction, refinement","volume":"19","year":"2002","journal-title":"ALTEX-Altern. Anim. Ex."}],"container-title":["Chaos: An Interdisciplinary Journal of Nonlinear Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.aip.org\/aip\/cha\/article-pdf\/doi\/10.1063\/5.0346786\/21142401\/091101_1_5.0346786.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pubs.aip.org\/aip\/cha\/article-pdf\/doi\/10.1063\/5.0346786\/21142401\/091101_1_5.0346786.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T12:50:35Z","timestamp":1789563035000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.aip.org\/cha\/article\/36\/9\/091101\/3404691\/Deep-reinforcement-learning-to-control-cardiac"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,1]]},"references-count":72,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2026,9,1]]}},"URL":"https:\/\/doi.org\/10.1063\/5.0346786","relation":{},"ISSN":["1054-1500","1089-7682"],"issn-type":[{"value":"1054-1500","type":"print"},{"value":"1089-7682","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,9]]},"published":{"date-parts":[[2026,9,1]]},"article-number":"091101"}}