{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T01:30:34Z","timestamp":1773797434090,"version":"3.50.1"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,1,18]],"date-time":"2024-01-18T00:00:00Z","timestamp":1705536000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,1,18]],"date-time":"2024-01-18T00:00:00Z","timestamp":1705536000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/R018472\/1"],"award-info":[{"award-number":["EP\/R018472\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Categor Struct"],"published-print":{"date-parts":[[2024,2]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We introduce an algebraic analogue of dynamical systems, based on term rewriting. We show that a recursive function applied to the output of an iterated rewriting system defines a formal class of models into which all the main architectures for dynamic machine learning models (including recurrent neural networks, graph neural networks, and diffusion models) can be embedded. Considered in category theory, we also show that these algebraic models are a natural language for describing the compositionality of dynamic models. Furthermore, we propose that these models provide a template for the generalisation of the above dynamic models to learning problems on structured or non-numerical data, including \u2018hybrid symbolic-numeric\u2019 models.<\/jats:p>","DOI":"10.1007\/s10485-023-09762-9","type":"journal-article","created":{"date-parts":[[2024,1,18]],"date-time":"2024-01-18T11:43:09Z","timestamp":1705578189000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Algebraic Dynamical Systems in Machine Learning"],"prefix":"10.1007","volume":"32","author":[{"given":"Iolo","family":"Jones","sequence":"first","affiliation":[]},{"given":"Jerry","family":"Swan","sequence":"additional","affiliation":[]},{"given":"Jeffrey","family":"Giansiracusa","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,1,18]]},"reference":[{"key":"9762_CR1","doi-asserted-by":"publisher","unstructured":"Aloysius, N., Geetha, M. : A review on deep convolutional neural networks. In: 2017 International Conference on Communication and Signal Processing (ICCSP). (pp. 0588\u20130592). (2017). https:\/\/doi.org\/10.1109\/ICCSP.2017.8286426","DOI":"10.1109\/ICCSP.2017.8286426"},{"key":"9762_CR2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139172752","volume-title":"Term Rewriting and All That","author":"F Baader","year":"1998","unstructured":"Baader, F., Tobias, N.: Term Rewriting and All That. Cambridge University Press (1998). https:\/\/doi.org\/10.1017\/CBO9781139172752"},{"key":"9762_CR3","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1007\/BF00122683","volume":"4","author":"L Stephen","year":"1996","unstructured":"Stephen, L., Bloom, N.S., Walters, R.F.C.: Matrices, machines and behaviors. Appl. Categorical Struct. 4, 343\u2013360 (1996). https:\/\/doi.org\/10.1007\/BF00122683","journal-title":"Appl. Categorical Struct."},{"key":"9762_CR4","unstructured":"Michael, M., Bronstein, J.B., Taco C., et al. Geometric deep learning: grids, groups, graphs, geodesics, and gauges. (2021). arXiv:2104.13478 [cs.LG]"},{"key":"9762_CR5","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4613-8130-3","volume-title":"A Course in Universal Algebra","author":"S Burris","year":"1981","unstructured":"Burris, S., Sankappanavar, H.P.: A Course in Universal Algebra. Springer, Cham (1981)"},{"key":"9762_CR6","unstructured":"Cao, H., Tan, C., Gao, Z., et al. A survey on generative diffusion model. (2022). arXiv:2209.02646 [cs.AI]"},{"key":"9762_CR7","unstructured":"Chamberlain, B.P, Rowbottom, J., Gorinova, M., et al. GRAND: graph neural diffusion. (2021). arXiv:2106.10934 [cs.LG]"},{"key":"9762_CR8","doi-asserted-by":"publisher","first-page":"2227","DOI":"10.3390\/math11081777","volume":"11","author":"S Chen","year":"2023","unstructured":"Chen, S., Guo, W.: Auto-encoders in deep learning;a review with new perspectives\u2019. Mathematics 11, 2227\u20137390 (2023). https:\/\/doi.org\/10.3390\/math11081777","journal-title":"Mathematics"},{"key":"9762_CR9","doi-asserted-by":"crossref","unstructured":"Cruttwell, G.S.H., Gavranovi\u0107, B., Ghani, N., et al. Categorical foundations of gradient-based learning. (2021). arXiv:2103.01931 [cs.LG]","DOI":"10.1007\/978-3-030-99336-8_1"},{"key":"9762_CR10","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/BF02551274","volume":"2","author":"G Cybenko","year":"1989","unstructured":"Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. (MCSS) 2, 303\u2013314 (1989). https:\/\/doi.org\/10.1007\/BF02551274","journal-title":"Math. Control Signals Syst. (MCSS)"},{"key":"9762_CR11","unstructured":"D\u2019Amour, A., Heller, K., Moldovan, D., et al. Underspecification presents challenges for credibility in modern machine learning. (2020). arXiv: 2011.03395 [cs.LG]"},{"key":"9762_CR12","doi-asserted-by":"publisher","first-page":"608","DOI":"10.1109\/JSTSP.2016.2539100","volume":"10","author":"MA Davenport","year":"2016","unstructured":"Davenport, M.A., Romberg, J.: An overview of low-rank matrix recovery from incomplete observations. IEEE J. Sel. Topics Signal Process. 10, 608\u2013622 (2016). https:\/\/doi.org\/10.1109\/JSTSP.2016.2539100","journal-title":"IEEE J. Sel. Topics Signal Process."},{"key":"9762_CR13","unstructured":"Davison, A. J.: FutureMapping: the computational structure of spatial AI systems. In: CoRR abs\/1803.11288 (2018). arXiv:1803.11288"},{"key":"9762_CR14","unstructured":"Diskin, Z.: Supervised categorical learning as change propagation with delta lenses. In: CoRR abs\/1911.12904 (2019). arXiv:1911.12904"},{"key":"9762_CR15","doi-asserted-by":"publisher","unstructured":"Elliott, C. Compiling to categories. In: Proc. ACM Program. Lang. 1.ICFP (2017). https:\/\/doi.org\/10.1145\/3110271","DOI":"10.1145\/3110271"},{"key":"9762_CR16","doi-asserted-by":"crossref","unstructured":"Elliott, C.: The simple essence of automatic differentiation. In: Proceedings of the ACM on Programming Languages (ICFP). (2018). http:\/\/conal.net\/papers\/essence-of-ad\/","DOI":"10.1145\/3236765"},{"key":"9762_CR17","unstructured":"Fong, B., Johnson, M.: Lenses and learners. (2019). arXiv:1903.03671 [cs.LG]"},{"key":"9762_CR18","doi-asserted-by":"crossref","unstructured":"Fong, B., Spivak, D.I., Tuy\u00e9ras, R.: Backprop as Functor: a compositional perspective on supervised learning. (2019). arXiv:1711.10455 [math.CT]","DOI":"10.1109\/LICS.2019.8785665"},{"key":"9762_CR19","unstructured":"Galor, O., et al. Discrete dynamical systems. In: GE, Growth, Math Methods, Econ-WPA. available at http:\/\/ideas.repec.org\/p\/wpa\/wuwpge\/0504001.html (2005)"},{"key":"9762_CR20","unstructured":"Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and Harnessing Adversarial Example\u201d. In: CoRR abs\/1412.6572 (2014). arXiv:1412.6572"},{"key":"9762_CR21","doi-asserted-by":"crossref","unstructured":"Hughes, N., Chang Y., Hu, S., et al. Foundations of spatial perception for robotics: hierarchical representations and real-time systems. (2023). arXiv: 2305.07154 [cs.RO]","DOI":"10.1177\/02783649241229725"},{"key":"9762_CR22","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1115\/1.3662552","volume":"82","author":"RE Kalman","year":"1960","unstructured":"Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82, 35\u201345 (1960)","journal-title":"J. Basic Eng."},{"key":"9762_CR23","unstructured":"Kondor, I.R.: Group Theoretical Methods in Machine Learning. PhD thesis. Columbia University, (2008)"},{"key":"9762_CR24","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1007\/BF01110627","volume":"103","author":"J Lambek","year":"1968","unstructured":"Lambek, J.: A fixpoint theorem for complete categories. Math. Zeitschrift 103, 151\u2013161 (1968). (http:\/\/eudml.org\/doc\/170906)","journal-title":"Math. Zeitschrift"},{"key":"9762_CR25","unstructured":"Lipton, Z.C., Berkowitz J., Elkan C., A critical review of recurrent neural networks for sequence learning. (2015). arXiv:1506.00019 [cs.LG]"},{"key":"9762_CR26","unstructured":"Malcolm, G.: Algebraic data types and program transformation. PhD thesis. University of Groningen: Faculty of Science and Engineering, (1990)"},{"key":"9762_CR27","volume-title":"Marcus. The Algebraic Mind","author":"F Gary","year":"2001","unstructured":"Gary, F.: Marcus. The Algebraic Mind. MIT Press (2001)"},{"key":"9762_CR28","unstructured":"Master, J.: Composing behaviors of networks. (2021). arXiv:2105.12905 [math.CT]"},{"key":"9762_CR29","unstructured":"McCarthy, J., Minsky, M. L., Rochester, N.: et al. A Proposal for the dartmouth summer research project on artificial intelligence. (1955). http:\/\/www-formal.stanford.edu\/jmc\/history\/dartmouth\/dartmouth.html"},{"key":"9762_CR30","unstructured":"Myers, D.Z.: Categorical systems theory. http:\/\/davidjaz.com\/Papers\/DynamicalBook.pdf. In preparation. (2022)"},{"key":"9762_CR31","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1002\/for.3980020104","volume":"2","author":"N Paul","year":"1983","unstructured":"Paul, N.: ARIMA model building and the time series analysis approach to forecasting. J. Forecast. 2, 23\u201335 (1983). https:\/\/doi.org\/10.1002\/for.3980020104","journal-title":"J. Forecast."},{"key":"9762_CR32","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1038\/323533a0","volume":"323","author":"DE Rumelhart","year":"1986","unstructured":"Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533\u2013536 (1986)","journal-title":"Nature"},{"key":"9762_CR33","volume-title":"Russell, Peter, Norvig: Artificial Intelligence: A Modern Approach","author":"J Stuart","year":"2010","unstructured":"Stuart, J.: Russell, Peter, Norvig: Artificial Intelligence: A Modern Approach. Prentice Hall (2010)"},{"key":"9762_CR34","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/S0893-6080(97)00097-X","volume":"11","author":"S Franco","year":"1998","unstructured":"Franco, S., Chung, T.A.: Universal approximation using feedforward neural networks: a survey of some existing methods, and some new results. Neural Netw. 11, 15\u201337 (1998). https:\/\/doi.org\/10.1016\/S0893-6080(97)00097-X","journal-title":"Neural Netw."},{"key":"9762_CR35","doi-asserted-by":"publisher","unstructured":"Sch\u00f6lkopf, B.: Causality for machine learning. In: Probabilistic and Causal Inference. ACM, (pp. 765\u2013804). (2022). https:\/\/doi.org\/10.1145\/3501714.3501755","DOI":"10.1145\/3501714.3501755"},{"key":"9762_CR36","unstructured":"Dan, S., Bruno, G., Paul, W.W.: Category theory in machine learning. In: CoRR abs\/2106.07032 (2021). arXiv:2106.07032"},{"key":"9762_CR37","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1177\/027836498600500404","volume":"5","author":"SC Randall","year":"1986","unstructured":"Randall, S.C., Peter, C.: On the representation and estimation of spatial uncertainty. Int. J. Robot. Res. 5, 56\u201368 (1986). https:\/\/doi.org\/10.1177\/027836498600500404","journal-title":"Int. J. Robot. Res."},{"key":"9762_CR38","unstructured":"Smithe, T.S.C.: Bayesian updates compose optically. (2020). arXiv:2006.01631 [math.CT]"},{"key":"9762_CR39","doi-asserted-by":"publisher","first-page":"14","DOI":"10.4204\/eptcs.372.2","volume":"372","author":"DI Spivak","year":"2022","unstructured":"Spivak, D.I.: Learners\u2019 languages. Electron. Proc. Theor. Comput. Sci. 372, 14\u201328 (2022). https:\/\/doi.org\/10.4204\/eptcs.372.2","journal-title":"Electron. Proc. Theor. Comput. Sci."},{"key":"9762_CR40","doi-asserted-by":"publisher","unstructured":"Spivak, D.I.: Poly: an abundant categorical setting for mode-dependent dynamics. (2020). https:\/\/doi.org\/10.48550\/ARXIV.2005.01894","DOI":"10.48550\/ARXIV.2005.01894"},{"key":"9762_CR41","doi-asserted-by":"publisher","unstructured":"Jerry, S.: Harmonic analysis and resynthesis of Sliding-Tile Puzzle heuristics\u2019. In: 2017 IEEE Congress on Evolutionary Computation (CEC). (pp. 516\u2013524)). (2017). https:\/\/doi.org\/10.1109\/CEC.2017.7969355","DOI":"10.1109\/CEC.2017.7969355"},{"key":"9762_CR42","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1007\/s10710-019-09347-3","volume":"20","author":"S Jerry","year":"2019","unstructured":"Jerry, S., Krzysztof, K., Zoltan, K.A.: Stochastic synthesis of recursive functions made easy with bananas, lenses, envelopes and barbed wire\u2019. Genetic Program Evol Mach 20, 327\u2013350 (2019). https:\/\/doi.org\/10.1007\/s10710-019-09347-3","journal-title":"Genetic Program Evol Mach"},{"key":"9762_CR43","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-08020-3","volume-title":"The Road to General Intelligence","author":"J Swan","year":"2022","unstructured":"Swan, J., Nivel, E., Kant, N., et al.: The Road to General Intelligence. Springer, Cham (2022). https:\/\/doi.org\/10.1007\/978-3-031-08020-3"},{"key":"9762_CR44","doi-asserted-by":"crossref","unstructured":"Takens, F.: Detecting strange attractors in turbulence. In: Dynamical Systems and Turbulence, Warwick 1980: proceedings of a symposium held at the University of Warwick 1979\/80. Springer. (pp. 366\u2013381). (2006)","DOI":"10.1007\/BFb0091924"},{"key":"9762_CR45","unstructured":"Vaswani, A., Shazeer N.,, Parmar, N., et al. Attention is All you Need\u201d. In: Advances in Neural Information Processing Systems. Ed. by I. Guyon, U. Von Luxburg, S. Bengio, et al. Vol. 30. Curran Associates, Inc., (2017). https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf"},{"key":"9762_CR46","doi-asserted-by":"publisher","first-page":"1235","DOI":"10.1162\/neco_a_01199","volume":"31","author":"Y Yu","year":"2019","unstructured":"Yu, Y., Si, X., Hu, C., et al.: A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput. 31, 1235\u20131270 (2019). https:\/\/doi.org\/10.1162\/neco_a_01199","journal-title":"Neural Comput."},{"key":"9762_CR47","unstructured":"Yun, S., Jeong, M., Kim, R., et al. Graph transformer networks\u2019. In: Advances in Neural Information Processing Systems. Ed. by H. Wallach, H. Larochelle, A. Beygelzimer, et al. Vol. 32. Curran Associates, Inc., 2019. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/9d63484abb477c97640154d40595a3bb-Paper.pdf"},{"key":"9762_CR48","unstructured":"Zhou, J., Cui, G., Hu, S., et al. Graph neural networks: a review of methods and applications. (2021). arXiv:1812.08434 [cs.LG]"}],"container-title":["Applied Categorical Structures"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10485-023-09762-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10485-023-09762-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10485-023-09762-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T18:09:53Z","timestamp":1708970993000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10485-023-09762-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,18]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,2]]}},"alternative-id":["9762"],"URL":"https:\/\/doi.org\/10.1007\/s10485-023-09762-9","relation":{},"ISSN":["0927-2852","1572-9095"],"issn-type":[{"value":"0927-2852","type":"print"},{"value":"1572-9095","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,18]]},"assertion":[{"value":"16 June 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 December 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 January 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"None.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"All authors consent to the publication of this work.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}}],"article-number":"4"}}