{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T01:02:29Z","timestamp":1788915749072,"version":"build-2803163510"},"reference-count":71,"publisher":"AIP Publishing","issue":"8","funder":[{"DOI":"10.13039\/501100001804","name":"Canada Research Chairs","doi-asserted-by":"publisher","award":["950-231970"],"award-info":[{"award-number":["950-231970"]}],"id":[{"id":"10.13039\/501100001804","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["pubs.aip.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,8,1]]},"abstract":"<jats:p>In recent years, the artificial intelligence community has seen a continuous interest in research aimed at investigating dynamical aspects of both training procedures and machine learning models. Of particular interest among recurrent neural networks, we have the Reservoir Computing (RC) paradigm characterized by conceptual simplicity and a fast training scheme. Yet, the guiding principles under which RC operates are only partially understood. In this work, we analyze the role played by Generalized Synchronization (GS) when training a RC to solve a generic task. In particular, we show how GS allows the reservoir to correctly encode the system generating the input signal into its dynamics. We also discuss necessary and sufficient conditions for the learning to be feasible in this approach. Moreover, we explore the role that ergodicity plays in this process, showing how its presence allows the learning outcome to apply to multiple input trajectories. Finally, we show that satisfaction of the GS can be measured by means of the mutual false nearest neighbors index, which makes effective to practitioners theoretical derivations.<\/jats:p>","DOI":"10.1063\/5.0056425","type":"journal-article","created":{"date-parts":[[2021,8,13]],"date-time":"2021-08-13T16:25:19Z","timestamp":1628871919000},"update-policy":"https:\/\/doi.org\/10.1063\/aip-crossmark-policy-page","source":"Crossref","is-referenced-by-count":29,"title":["Learn to synchronize, synchronize to learn"],"prefix":"10.1063","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4470-7182","authenticated-orcid":false,"given":"Pietro","family":"Verzelli","sequence":"first","affiliation":[{"name":"Faculty of Informatics, Universit\u00e0 della Svizzera Italiana 1 , Lugano 69000, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3819-0025","authenticated-orcid":false,"given":"Cesare","family":"Alippi","sequence":"additional","affiliation":[{"name":"Faculty of Informatics, Universit\u00e0 della Svizzera Italiana 1 , Lugano 69000, Switzerland"},{"name":"Department of Electronics, Information and Bioengineering, Politecnico di Milano 2 , Milan 20133, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6384-4743","authenticated-orcid":false,"given":"Lorenzo","family":"Livi","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Mathematics, University of Manitoba 3 , Winnipeg, Manitoba R3T 2N2, Canada"},{"name":"Department of Computer Science, College of Engineering, Mathematics and Physical Sciences, University of Exeter 4 , Exeter EX4 4QF, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"317","published-online":{"date-parts":[[2021,8,13]]},"reference":[{"key":"2023080720140610600_c1","unstructured":"G.-H.\u2008Liu and E. A.\u2008Theodorou, \u201cDeep learning theory review: An optimal control and dynamical systems perspective,\u201d arXiv:1908.10920 (2019)."},{"key":"2023080720140610600_c2","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1109\/TNNLS.2016.2630802","article-title":"Investigating echo state networks dynamics by means of recurrence analysis","volume":"29","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"2023080720140610600_c3","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1016\/j.neuron.2009.07.018","article-title":"Generating coherent patterns of activity from chaotic neural networks","volume":"63","year":"2009","journal-title":"Neuron"},{"key":"2023080720140610600_c4","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning long-term dependencies with gradient descent is difficult","volume":"5","year":"1994","journal-title":"IEEE Trans. Neural Networks"},{"key":"2023080720140610600_c5","doi-asserted-by":"publisher","first-page":"2460","DOI":"10.1137\/14096815X","article-title":"Kernel methods for the approximation of nonlinear systems","volume":"55","year":"2017","journal-title":"SIAM J. Control Optim."},{"key":"2023080720140610600_c6","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1073\/pnas.1917285117","article-title":"Using machine learning to predict extreme events in complex systems","volume":"117","year":"2020","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"2023080720140610600_c7","unstructured":"W.\u2008Gilpin, \u201cDeep learning of dynamical attractors from time series measurements,\u201d arXiv:2002.05909 (2020)."},{"key":"2023080720140610600_c8","unstructured":"J. H.\u2008Tu, C. W.\u2008Rowley, D. M.\u2008Luchtenburg, S. L.\u2008Brunton, and J. N.\u2008Kutz, \u201cOn dynamic mode decomposition: Theory and applications,\u201d arXiv:1312.0041 (2013)."},{"key":"2023080720140610600_c9","doi-asserted-by":"crossref","unstructured":"T.\u2008Berry, D.\u2008Giannakis, and J.\u2008Harlim, \u201cBridging data science and dynamical systems theory,\u201d arXiv:2002.07928 (2020).","DOI":"10.1090\/noti2151"},{"key":"2023080720140610600_c10","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1016\/j.neunet.2007.04.003","article-title":"An experimental unification of reservoir computing methods","volume":"20","year":"2007","journal-title":"Neural Netw."},{"key":"2023080720140610600_c11","doi-asserted-by":"publisher","first-page":"041102","DOI":"10.1063\/1.4979665","article-title":"Reservoir observers: Model-free inference of unmeasured variables in chaotic systems","volume":"27","year":"2017","journal-title":"Chaos"},{"key":"2023080720140610600_c12","doi-asserted-by":"publisher","first-page":"061104","DOI":"10.1063\/1.5039508","article-title":"Attractor reconstruction by machine learning","volume":"28","year":"2018","journal-title":"Chaos"},{"key":"2023080720140610600_c13","doi-asserted-by":"publisher","first-page":"373","DOI":"10.5194\/npg-27-373-2020","article-title":"Data-driven predictions of a multiscale lorenz 96 chaotic system using machine-learning methods: Reservoir computing, artificial neural network, and long short-term memory network","volume":"27","year":"2020","journal-title":"Nonlinear Process. Geophys."},{"key":"2023080720140610600_c14","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1016\/j.neunet.2020.02.016","article-title":"Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics","volume":"126","year":"2020","journal-title":"Neural Netw."},{"key":"2023080720140610600_c15","doi-asserted-by":"crossref","unstructured":"S.\u2008Bompas, B.\u2008Georgeot, and D.\u2008Gu\u00e9ry-Odelin, \u201cAccuracy of neural networks for the simulation of chaotic dynamics: Precision of training data vs precision of the algorithm,\u201d arXiv:2008.04222 (2020).","DOI":"10.1063\/5.0021264"},{"key":"2023080720140610600_c16"},{"key":"2023080720140610600_c17","doi-asserted-by":"publisher","first-page":"2531","DOI":"10.1162\/089976602760407955","article-title":"Real-time computing without stable states: A new framework for neural computation based on perturbations","volume":"14","year":"2002","journal-title":"Neural Comput."},{"key":"2023080720140610600_c18","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1023\/A:1010972803901","article-title":"Predicting the future of discrete sequences from fractal representations of the past","volume":"45","year":"2001","journal-title":"Mach. Learn."},{"key":"2023080720140610600_c19","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1016\/j.neunet.2018.08.025","article-title":"Echo state networks are universal","volume":"108","year":"2018","journal-title":"Neural Netw."},{"key":"2023080720140610600_c20","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1016\/j.neunet.2020.05.013","article-title":"Embedding and approximation theorems for echo state networks","volume":"128","year":"2020","journal-title":"Neural Netw."},{"key":"2023080720140610600_c21","unstructured":"A. G.\u2008Hart, J. L.\u2008Hook, and J. H.\u2008Dawes, \u201cEcho state networks trained by Tikhonov least squares are l2(\u03bc) approximators of ergodic dynamical systems,\u201d arXiv:2005.06967 (2020)."},{"key":"2023080720140610600_c22","doi-asserted-by":"publisher","first-page":"132721","DOI":"10.1016\/j.physd.2020.132721","article-title":"Memory and forecasting capacities of nonlinear recurrent networks","volume":"414","year":"2020","journal-title":"Physica D"},{"key":"2023080720140610600_c23","doi-asserted-by":"publisher","first-page":"042809","DOI":"10.1103\/PhysRevE.87.042809","article-title":"Mean-field theory of echo state networks","volume":"87","year":"2013","journal-title":"Phys. Rev. E"},{"key":"2023080720140610600_c24","doi-asserted-by":"publisher","first-page":"1139","DOI":"10.1162\/neco_a_01187","article-title":"A geometrical analysis of global stability in trained feedback networks","volume":"31","year":"2019","journal-title":"Neural Comput."},{"key":"2023080720140610600_c25","doi-asserted-by":"publisher","first-page":"258101","DOI":"10.1103\/PhysRevLett.118.258101","article-title":"Local dynamics in trained recurrent neural networks","volume":"118","year":"2017","journal-title":"Phys. Rev. Lett."},{"key":"2023080720140610600_c26"},{"key":"2023080720140610600_c27","unstructured":"A.\u2008Goudarzi, S.\u2008Marzen, P.\u2008Banda, G.\u2008Feldman, C.\u2008Teuscher, and D.\u2008Stefanovic, \u201cMemory and information processing in recurrent neural networks,\u201d arXiv:1604.06929 (2016)."},{"key":"2023080720140610600_c28","doi-asserted-by":"publisher","first-page":"032308","DOI":"10.1103\/PhysRevE.96.032308","article-title":"Difference between memory and prediction in linear recurrent networks","volume":"96","year":"2017","journal-title":"Phys. Rev. E"},{"key":"2023080720140610600_c29","first-page":"1","article-title":"Dynamical systems as temporal feature spaces","volume":"21","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"2023080720140610600_c30","unstructured":"P.\u2008Verzelli, C.\u2008Alippi, L.\u2008Livi, and P.\u2008Tino, \u201cInput representation in recurrent neural networks dynamics,\u201d arXiv:2003.10585 (2020)."},{"key":"2023080720140610600_c31","doi-asserted-by":"publisher","first-page":"18970","DOI":"10.1073\/pnas.0804451105","article-title":"Memory traces in dynamical systems","volume":"105","year":"2008","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"2023080720140610600_c32","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.neunet.2019.03.005","article-title":"Recent advances in physical reservoir computing: A review","volume":"115","year":"2019","journal-title":"Neural Netw."},{"key":"2023080720140610600_c33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.neunet.2012.07.005","article-title":"Re-visiting the echo state property","volume":"35","year":"2012","journal-title":"Neural Netw."},{"key":"2023080720140610600_c34","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1109\/TNNLS.2011.2178562","article-title":"Nonlinear system modeling with random matrices: Echo state networks revisited","volume":"23","year":"2011","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"2023080720140610600_c35"},{"key":"2023080720140610600_c36","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1162\/NECO_a_00411","article-title":"Echo state property linked to an input: Exploring a fundamental characteristic of recurrent neural networks","volume":"25","year":"2013","journal-title":"Neural Comput."},{"key":"2023080720140610600_c37"},{"key":"2023080720140610600_c38","doi-asserted-by":"publisher","first-page":"132609","DOI":"10.1016\/j.physd.2020.132609","article-title":"The echo index and multistability in input-driven recurrent neural networks","volume":"412","year":"2020","journal-title":"Physica D"},{"key":"2023080720140610600_c39","doi-asserted-by":"publisher","first-page":"063133","DOI":"10.1063\/5.0004344","article-title":"Invertible generalized synchronization: A putative mechanism for implicit learning in neural systems","volume":"30","year":"2020","journal-title":"Chaos"},{"key":"2023080720140610600_c40","doi-asserted-by":"publisher","first-page":"042203","DOI":"10.1103\/PhysRevE.99.042203","article-title":"Synchronization of chaotic systems and their machine-learning models","volume":"99","year":"2019","journal-title":"Phys. Rev. E"},{"key":"2023080720140610600_c41","doi-asserted-by":"publisher","first-page":"093133","DOI":"10.1063\/1.5120733","article-title":"The reservoir\u2019s perspective on generalized synchronization","volume":"29","year":"2019","journal-title":"Chaos"},{"key":"2023080720140610600_c42","doi-asserted-by":"crossref","unstructured":"L.\u2008Grigoryeva, A.\u2008Hart, and J.-P.\u2008Ortega, \u201cChaos on compact manifolds: Differentiable synchronizations beyond the Takens theorem,\u201d arXiv:2010.03218 (2020).","DOI":"10.1103\/PhysRevE.103.062204"},{"key":"2023080720140610600_c43","doi-asserted-by":"publisher","first-page":"795","DOI":"10.1007\/BF01034476","article-title":"Stochastic synchronization of oscillation in dissipative systems","volume":"29","year":"1986","journal-title":"Radiophys. Quantum Electron."},{"key":"2023080720140610600_c44","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1103\/PhysRevE.51.980","article-title":"Generalized synchronization of chaos in directionally coupled chaotic systems","volume":"51","year":"1995","journal-title":"Phys. Rev. E"},{"key":"2023080720140610600_c45","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1063\/1.166278","article-title":"Fundamentals of synchronization in chaotic systems, concepts, and applications","volume":"7","year":"1997","journal-title":"Chaos"},{"key":"2023080720140610600_c46","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1587\/nolta.3.113","article-title":"Detecting generalized synchronization","volume":"3","year":"2012","journal-title":"Nonlinear Theory Appl. IEICE"},{"key":"2023080720140610600_c47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/S0370-1573(02)00137-0","article-title":"The synchronization of chaotic systems","volume":"366","year":"2002","journal-title":"Phys. Rep."},{"key":"2023080720140610600_c48"},{"key":"2023080720140610600_c49"},{"key":"2023080720140610600_c50"},{"key":"2023080720140610600_c51"},{"key":"2023080720140610600_c52","year":"2012"},{"key":"2023080720140610600_c53","doi-asserted-by":"publisher","first-page":"R4508","DOI":"10.1103\/PhysRevE.54.R4508","article-title":"Weak and strong synchronization of chaos","volume":"54","year":"1996","journal-title":"Phys. Rev. E"},{"key":"2023080720140610600_c54"},{"key":"2023080720140610600_c55"},{"key":"2023080720140610600_c56","volume-title":"Kernel Methods for Pattern Analysis","year":"2004"},{"key":"2023080720140610600_c57","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1109\/TNN.2006.885113","article-title":"Support vector echo-state machine for chaotic time-series prediction","volume":"18","year":"2007","journal-title":"IEEE Trans. Neural Netw."},{"key":"2023080720140610600_c58","volume-title":"Understanding Machine Learning: From Theory to Algorithms","year":"2014"},{"key":"2023080720140610600_c59","doi-asserted-by":"publisher","first-page":"656","DOI":"10.1073\/pnas.17.2.656","article-title":"Proof of the ergodic theorem","volume":"17","year":"1931","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"2023080720140610600_c60"},{"key":"2023080720140610600_c61","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1016\/S0167-2789(99)00140-2","article-title":"A robust method for detecting interdependences: Application to intracranially recorded EEG","volume":"134","year":"1999","journal-title":"Physica D"},{"key":"2023080720140610600_c62","doi-asserted-by":"publisher","first-page":"332","DOI":"10.1016\/0167-2789(91)90130-2","article-title":"Estimation of interrelation between chaotic observables","volume":"52","year":"1991","journal-title":"Physica D"},{"key":"2023080720140610600_c63","unstructured":"J. A.\u2008Platt, A. S.\u2008Wong, R.\u2008Clark, S. G.\u2008Penny, and H. D.\u2008Abarbanel, \u201cForecasting using reservoir computing: The role of generalized synchronization,\u201d arXiv:2103.00362 (2021)."},{"key":"2023080720140610600_c64","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/j.neucom.2016.12.089","article-title":"Deep reservoir computing: A critical experimental analysis","volume":"268","year":"2017","journal-title":"Neurocomputing"},{"key":"2023080720140610600_c65","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1007\/s12559-017-9450-z","article-title":"Training echo state networks with regularization through dimensionality reduction","volume":"9","year":"2017","journal-title":"Cognit. Comput."},{"key":"2023080720140610600_c66","doi-asserted-by":"publisher","first-page":"023118","DOI":"10.1063\/1.5079686","article-title":"Consistency in echo-state networks","volume":"29","year":"2019","journal-title":"Chaos"},{"key":"2023080720140610600_c67","volume-title":"Chaos in Dynamical Systems","year":"2002"},{"key":"2023080720140610600_c68","doi-asserted-by":"publisher","first-page":"821","DOI":"10.1103\/PhysRevLett.64.821","article-title":"Synchronization in chaotic systems","volume":"64","year":"1990","journal-title":"Phys. Rev. Lett."},{"key":"2023080720140610600_c69","doi-asserted-by":"publisher","first-page":"1816","DOI":"10.1103\/PhysRevLett.76.1816","article-title":"Generalized synchronization, predictability, and equivalence of unidirectionally coupled dynamical systems","volume":"76","year":"1996","journal-title":"Phys. Rev. Lett."},{"key":"2023080720140610600_c70","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1175\/1520-0469(1963)020&lt;0130:DNF&gt;2.0.CO;2","article-title":"Deterministic nonperiodic flow","volume":"20","year":"1963","journal-title":"J. Atmos. Sci."},{"key":"2023080720140610600_c71","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1016\/0375-9601(76)90101-8","article-title":"An equation for continuous chaos","volume":"57","year":"1976","journal-title":"Phys. Lett. A"}],"container-title":["Chaos: An Interdisciplinary Journal of Nonlinear Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.aip.org\/aip\/cha\/article-pdf\/doi\/10.1063\/5.0056425\/14634197\/083119_1_online.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pubs.aip.org\/aip\/cha\/article-pdf\/doi\/10.1063\/5.0056425\/14634197\/083119_1_online.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,7]],"date-time":"2023-08-07T20:14:17Z","timestamp":1691439257000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.aip.org\/cha\/article\/31\/8\/083119\/1059696\/Learn-to-synchronize-synchronize-to-learn"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,1]]},"references-count":71,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2021,8,1]]}},"URL":"https:\/\/doi.org\/10.1063\/5.0056425","relation":{},"ISSN":["1054-1500","1089-7682"],"issn-type":[{"value":"1054-1500","type":"print"},{"value":"1089-7682","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,8]]},"published":{"date-parts":[[2021,8,1]]},"article-number":"083119"}}