{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T20:57:27Z","timestamp":1780952247745,"version":"3.54.1"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"PLDI","license":[{"start":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T00:00:00Z","timestamp":1780876800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2026,6,8]]},"abstract":"<jats:p>Discrete structures are currently second-class in differentiable programming. Since functions over discrete structures lack overt derivatives, differentiable programs do not differentiate through them and limit where they can be used. For example, when programming a neural network, conditionals and iteration cannot be used everywhere; they can break the derivatives necessary for gradient-based learning to work. This limits the class of differentiable algorithms we can directly express, imposing restraints on how we build neural networks and differentiable programs more generally. However, these restraints are not fundamental. Recent work shows conditionals can be first-class, by compiling them into differentiable form as linear neurons. Similarly, this work shows iteration can be first-class---by compiling to linear recurrent neurons. We present a minimal typed, higher-order and linear programming language with iteration called Cajal(N). We prove its programs compile correctly to recurrent neurons, allowing discrete algorithms to be expressed in a differentiable form compatible with gradient-based learning. With our implementation, we conduct two experiments where we link these recurrent neurons against a neural network solving an iterative image transformation task. This determines part of its function prior to learning. As a result, the network learns faster and with greater data-efficiency relative to a neural network programmed without first-class iteration. A key lesson is that recurrent neurons enable a rich interplay between learning and the discrete structures of ordinary programming.<\/jats:p>","DOI":"10.1145\/3808345","type":"journal-article","created":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T18:04:09Z","timestamp":1780941849000},"page":"2452-2473","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Compiling to Recurrent Neurons"],"prefix":"10.1145","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-6451-5107","authenticated-orcid":false,"given":"Joey","family":"Velez-Ginorio","sequence":"first","affiliation":[{"name":"University of Pennsylvania, Philadelphia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0830-7248","authenticated-orcid":false,"given":"Nada","family":"Amin","sequence":"additional","affiliation":[{"name":"Harvard University, Cambridge, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8408-4499","authenticated-orcid":false,"given":"Konrad","family":"Kording","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, Philadelphia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3516-1512","authenticated-orcid":false,"given":"Steve","family":"Zdancewic","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, Philadelphia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,8]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3371106"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-8176-4661-5"},{"key":"e_1_2_2_3_1","volume-title":"International conference on machine learning. 547\u2013556","author":"Bosnjak Matko","year":"2017","unstructured":"Matko Bosnjak, Tim Rocktaschel, Jason Naradowsky, and Sebastian Riedel. 2017. Programming with a differentiable forth interpreter. In International conference on machine learning. 547\u2013556. https:\/\/dl.acm.org\/doi\/10.5555\/3305381.3305438"},{"key":"e_1_2_2_4_1","volume-title":"Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang.","author":"Bradbury James","year":"2018","unstructured":"James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. 2018. JAX: composable transformations of Python+NumPy programs. http:\/\/github.com\/jax-ml\/jax"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0960129502003729"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236765"},{"key":"e_1_2_2_7_1","volume-title":"International Conference on Machine Learning. 1213\u20131222","author":"Gaunt Alexander L","year":"2017","unstructured":"Alexander L Gaunt, Marc Brockschmidt, Nate Kushman, and Daniel Tarlow. 2017. Differentiable programs with neural libraries. In International Conference on Machine Learning. 1213\u20131222. https:\/\/dl.acm.org\/doi\/abs\/10.5555\/3305381.3305507"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","unstructured":"Alex Graves Greg Wayne and Ivo Danihelka. 2014. Neural turing machines. arXiv preprint arXiv:1410.5401 https:\/\/doi.org\/10.48550\/arXiv.1410.5401 10.48550\/arXiv.1410.5401","DOI":"10.48550\/arXiv.1410.5401"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","unstructured":"David Ha Andrew Dai and Quoc V Le. 2016. Hypernetworks. arXiv preprint arXiv:1609.09106 https:\/\/doi.org\/10.48550\/arXiv.1609.09106 10.48550\/arXiv.1609.09106","DOI":"10.48550\/arXiv.1609.09106"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781316576892"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2010.579"},{"key":"e_1_2_2_12_1","volume-title":"Deep recursive neural networks for compositionality in language. Advances in neural information processing systems, 27","author":"Irsoy Ozan","year":"2014","unstructured":"Ozan Irsoy and Claire Cardie. 2014. Deep recursive neural networks for compositionality in language. Advances in neural information processing systems, 27 (2014), https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2014\/hash\/cb954cd53b6527a80248cc0732366871-Abstract.html"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/4932.001.0001"},{"key":"e_1_2_2_14_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, arxiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, arxiv:1412.6980"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3498710"},{"key":"e_1_2_2_16_1","volume-title":"Introductory functional analysis with applications","author":"Kreyszig Erwin","unstructured":"Erwin Kreyszig. 1991. Introductory functional analysis with applications. John Wiley & Sons."},{"key":"e_1_2_2_17_1","volume-title":"Handwritten digit recognition with a back-propagation network. Advances in neural information processing systems, 2","author":"LeCun Yann","year":"1989","unstructured":"Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard, and Lawrence Jackel. 1989. Handwritten digit recognition with a back-propagation network. Advances in neural information processing systems, 2 (1989)."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i9.28934"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3591280"},{"key":"e_1_2_2_20_1","volume-title":"Categorical semantics of linear logic. Panoramas et syntheses, 27","author":"Mellies Paul-Andr\u00e9","year":"2009","unstructured":"Paul-Andr\u00e9 Mellies. 2009. Categorical semantics of linear logic. Panoramas et syntheses, 27 (2009), 15\u2013215."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","unstructured":"Razvan Pascanu Caglar Gulcehre Kyunghyun Cho and Yoshua Bengio. 2013. How to construct deep recurrent neural networks. arXiv preprint arXiv:1312.6026 https:\/\/doi.org\/10.48550\/arXiv.1312.6026 10.48550\/arXiv.1312.6026","DOI":"10.48550\/arXiv.1312.6026"},{"key":"e_1_2_2_22_1","volume-title":"PyTorch: An Imperative Style","author":"Paszke Adam","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alch\u00e9-Buc, E. Fox, and R. Garnett (Eds.). 32, Curran Associates, Inc.. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1511.06279"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1201\/9780429501906"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","unstructured":"Beno\u00eet Valiron and Steve Zdancewic. 2014. Finite vector spaces as model of simply-typed lambda-calculi. In International Colloquium on Theoretical Aspects of Computing. 442\u2013459. https:\/\/doi.org\/10.48550\/arXiv.1406.1310 10.48550\/arXiv.1406.1310","DOI":"10.48550\/arXiv.1406.1310"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","unstructured":"Joey Velez-Ginorio. 2026. Compiling to recurrent neurons. https:\/\/doi.org\/10.5281\/zenodo.18963154 10.5281\/zenodo.18963154","DOI":"10.5281\/zenodo.18963154"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3776677"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","unstructured":"David Walker. 2005. Substructural type systems. Advanced topics in types and programming languages 3\u201344. https:\/\/doi.org\/10.7551\/mitpress\/1104.003.0003 10.7551\/mitpress\/1104.003.0003","DOI":"10.7551\/mitpress\/1104.003.0003"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/151145"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3808345","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T19:58:01Z","timestamp":1780948681000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3808345"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,8]]},"references-count":29,"journal-issue":{"issue":"PLDI","published-print":{"date-parts":[[2026,6,8]]}},"alternative-id":["10.1145\/3808345"],"URL":"https:\/\/doi.org\/10.1145\/3808345","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,8]]},"assertion":[{"value":"2025-11-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-03","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}