{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T18:19:49Z","timestamp":1781893189468,"version":"3.54.5"},"reference-count":74,"publisher":"Association for Computing Machinery (ACM)","issue":"ICFP","license":[{"start":{"date-parts":[[2019,7,26]],"date-time":"2019-07-26T00:00:00Z","timestamp":1564099200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2019,7,26]]},"abstract":"<jats:p>We present a system for the automatic differentiation (AD) of a higher-order functional array-processing language. The core functional language underlying this system simultaneously supports both source-to-source forward-mode AD and global optimisations such as loop transformations. In combination, gradient computation with forward-mode AD can be as efficient as reverse mode, and that the Jacobian matrices required for numerical algorithms such as Gauss-Newton and Levenberg-Marquardt can be efficiently computed.<\/jats:p>","DOI":"10.1145\/3341701","type":"journal-article","created":{"date-parts":[[2019,7,29]],"date-time":"2019-07-29T20:55:51Z","timestamp":1564433751000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":33,"title":["Efficient differentiable programming in a functional array-processing language"],"prefix":"10.1145","volume":"3","author":[{"given":"Amir","family":"Shaikhha","sequence":"first","affiliation":[{"name":"University of Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Fitzgibbon","sequence":"additional","affiliation":[{"name":"Microsoft Research, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dimitrios","family":"Vytiniotis","sequence":"additional","affiliation":[{"name":"DeepMind, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simon","family":"Peyton Jones","sequence":"additional","affiliation":[{"name":"Microsoft Research, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,7,26]]},"reference":[{"key":"e_1_2_2_1_1","first-page":"265","article-title":"TensorFlow: A System for Large-Scale Machine Learning","volume":"16","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi , Paul Barham , Jianmin Chen , Zhifeng Chen , Andy Davis , Jeffrey Dean , Matthieu Devin , Sanjay Ghemawat , Geoffrey Irving , Michael Isard , 2016 . TensorFlow: A System for Large-Scale Machine Learning .. In OSDI , Vol. 16. 265 \u2013 283 . Mart\u00edn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. TensorFlow: A System for Large-Scale Machine Learning.. In OSDI, Vol. 16. 265\u2013283.","journal-title":"OSDI"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/1888028.1888032"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148177"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2503778.2503789"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30192-9_58"},{"key":"e_1_2_2_6_1","volume-title":"Alexey Andreyevich Radul, and Jeffrey Mark Siskind","author":"Baydin Atilim Gunes","year":"2015","unstructured":"Atilim Gunes Baydin , Barak A Pearlmutter , Alexey Andreyevich Radul, and Jeffrey Mark Siskind . 2015 b. Automatic differentiation in machine learning: a survey. arXiv preprint arXiv:1502.05767 (2015). Atilim Gunes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. 2015b. Automatic differentiation in machine learning: a survey. arXiv preprint arXiv:1502.05767 (2015)."},{"key":"e_1_2_2_7_1","volume-title":"Diffsharp: Automatic differentiation library. arXiv preprint arXiv:1511.07727","author":"Baydin Atilim Gunes","year":"2015","unstructured":"Atilim Gunes Baydin , Barak A Pearlmutter , and Jeffrey Mark Siskind . 2015 a. Diffsharp: Automatic differentiation library. arXiv preprint arXiv:1511.07727 (2015). Atilim Gunes Baydin, Barak A Pearlmutter, and Jeffrey Mark Siskind. 2015a. Diffsharp: Automatic differentiation library. arXiv preprint arXiv:1511.07727 (2015)."},{"key":"e_1_2_2_8_1","volume-title":"Proc. 9th Python in Science Conf. 1\u20137.","author":"Bergstra James","year":"2010","unstructured":"James Bergstra , Olivier Breuleux , Fr\u00e9d\u00e9ric Bastien , Pascal Lamblin , Razvan Pascanu , Guillaume Desjardins , Joseph Turian , David Warde-Farley , and Yoshua Bengio . 2010 . Theano: A CP U and GP U math compiler in Python . In Proc. 9th Python in Science Conf. 1\u20137. James Bergstra, Olivier Breuleux, Fr\u00e9d\u00e9ric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. 2010. Theano: A CP U and GP U math compiler in Python. In Proc. 9th Python in Science Conf. 1\u20137."},{"key":"e_1_2_2_9_1","volume-title":"Plemmons","author":"Berry Michael W.","year":"2006","unstructured":"Michael W. Berry , Murray Browne , Amy N. Langville , V. Paul Pauca , and Robert J . Plemmons . 2006 . Algorithms and applications for approximate nonnegative matrix factorization. In Computational Statistics and Data Analysis . Michael W. Berry, Murray Browne, Amy N. Langville, V. Paul Pauca, and Robert J. Plemmons. 2006. Algorithms and applications for approximate nonnegative matrix factorization. In Computational Statistics and Data Analysis."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/99.537089"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/827253.827739"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2254064.2254079"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2103736.2103740"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/223428.207154"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1291151.1291199"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/91556.91622"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553468"},{"key":"e_1_2_2_18_1","volume-title":"Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging. arXiv preprint arXiv:1604.06525","author":"DeVito Zachary","year":"2016","unstructured":"Zachary DeVito , Michael Mara , Michael Zollh\u00f6fer , Gilbert Bernstein , Jonathan Ragan-Kelley , Christian Theobalt , Pat Hanrahan , Matthew Fisher , and Matthias Nie\u00dfner . 2016 . Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging. arXiv preprint arXiv:1604.06525 (2016). Zachary DeVito, Michael Mara, Michael Zollh\u00f6fer, Gilbert Bernstein, Jonathan Ragan-Kelley, Christian Theobalt, Pat Hanrahan, Matthew Fisher, and Matthias Nie\u00dfner. 2016. Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging. arXiv preprint arXiv:1604.06525 (2016)."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236765"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1631687.1596579"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/173262.155113"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141885.1141888"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/11783596_12"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/165180.165214"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10766-006-0018-x"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276512"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2450153.2450158"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3062341.3062354"},{"key":"e_1_2_2_29_1","first-page":"70","article-title":"MuPAD-Combinat, an open-source package for research in algebraic combinatorics","volume":"51","author":"Hivert Florent","year":"2004","unstructured":"Florent Hivert and N Thi\u00e9ry . 2004 . MuPAD-Combinat, an open-source package for research in algebraic combinatorics . S\u00e9m. Lothar. Combin 51 (2004), 70 . Florent Hivert and N Thi\u00e9ry. 2004. MuPAD-Combinat, an open-source package for research in algebraic combinatorics. S\u00e9m. Lothar. Combin 51 (2004), 70.","journal-title":"S\u00e9m. Lothar. Combin"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2560359"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/242224.242477"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0956796899003500"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1460833.1460872"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/503104.503108"},{"key":"e_1_2_2_35_1","volume-title":"Haskell workshop","volume":"1","author":"Jones Simon Peyton","year":"2001","unstructured":"Simon Peyton Jones , Andrew Tolmach , and Tony Hoare . 2001 . Playing by the rules: rewriting as a practical optimisation technique in GHC . In Haskell workshop , Vol. 1 . 203\u2013233. Simon Peyton Jones, Andrew Tolmach, and Tony Hoare. 2001. Playing by the rules: rewriting as a practical optimisation technique in GHC. In Haskell workshop, Vol. 1. 203\u2013233."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2774975.2774981"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/291251.289442"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1080\/10556788.2015.1025400"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3009837.3009880"},{"key":"e_1_2_2_40_1","volume-title":"XLA: TensorFlow, compiled. TensorFlow Dev Summit","author":"Leary Chris","year":"2017","unstructured":"Chris Leary and Todd Wang . 2017 . XLA: TensorFlow, compiled. TensorFlow Dev Summit (2017). Chris Leary and Todd Wang. 2017. XLA: TensorFlow, compiled. TensorFlow Dev Summit (2017)."},{"key":"e_1_2_2_41_1","volume-title":"A method for the solution of certain non-linear problems in least squares. Quarterly of applied mathematics 2, 2","author":"Levenberg Kenneth","year":"1944","unstructured":"Kenneth Levenberg . 1944. A method for the solution of certain non-linear problems in least squares. Quarterly of applied mathematics 2, 2 ( 1944 ), 164\u2013168. Kenneth Levenberg. 1944. A method for the solution of certain non-linear problems in least squares. Quarterly of applied mathematics 2, 2 (1944), 164\u2013168."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772760"},{"key":"e_1_2_2_43_1","volume-title":"ICML 2015 AutoML Workshop.","author":"Maclaurin Dougal","year":"2015","unstructured":"Dougal Maclaurin , David Duvenaud , and Ryan P Adams . 2015 . Autograd: Effortless gradients in numpy . In ICML 2015 AutoML Workshop. Dougal Maclaurin, David Duvenaud, and Ryan P Adams. 2015. Autograd: Effortless gradients in numpy. In ICML 2015 AutoML Workshop."},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1137\/0111030"},{"key":"e_1_2_2_45_1","volume-title":"Numerical analysis","author":"Mor\u00e9 Jorge J","unstructured":"Jorge J Mor\u00e9 . 1978. The Levenberg-Marquardt algorithm: implementation and theory . In Numerical analysis . Springer , 105\u2013116. Jorge J Mor\u00e9. 1978. The Levenberg-Marquardt algorithm: implementation and theory. In Numerical analysis. Springer, 105\u2013116."},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2010.04.206"},{"key":"e_1_2_2_47_1","doi-asserted-by":"crossref","unstructured":"Peter Norvig. 1992. Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp. Morgan Kaufmann.   Peter Norvig. 1992. Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp. Morgan Kaufmann.","DOI":"10.1016\/B978-0-08-057115-7.50024-8"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3136040.3136043"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3158101"},{"key":"e_1_2_2_50_1","unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in pytorch. (2017).  Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in pytorch. (2017)."},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1190215.1190242"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1330017.1330018"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2004.840306"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491956.2462176"},{"key":"e_1_2_2_55_1","volume-title":"Forward-mode automatic differentiation in Julia. arXiv preprint arXiv:1607.07892","author":"Revels Jarrett","year":"2016","unstructured":"Jarrett Revels , Miles Lubin , and Theodore Papamarkou . 2016. Forward-mode automatic differentiation in Julia. arXiv preprint arXiv:1607.07892 ( 2016 ). Jarrett Revels, Miles Lubin, and Theodore Papamarkou. 2016. Forward-mode automatic differentiation in Julia. arXiv preprint arXiv:1607.07892 (2016)."},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/1868294.1868314"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2429069.2429128"},{"key":"e_1_2_2_58_1","volume-title":"Glow: Graph lowering compiler techniques for neural networks. arXiv preprint arXiv:1805.00907","author":"Rotem Nadav","year":"2018","unstructured":"Nadav Rotem , Jordan Fix , Saleem Abdulrasool , Garret Catron , Summer Deng , Roman Dzhabarov , Nick Gibson , James Hegeman , Meghan Lele , Roman Levenstein , 2018 . Glow: Graph lowering compiler techniques for neural networks. arXiv preprint arXiv:1805.00907 (2018). Nadav Rotem, Jordan Fix, Saleem Abdulrasool, Garret Catron, Summer Deng, Roman Dzhabarov, Nick Gibson, James Hegeman, Meghan Lele, Roman Levenstein, et al. 2018. Glow: Graph lowering compiler techniques for neural networks. arXiv preprint arXiv:1805.00907 (2018)."},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0956796818000102"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3122948.3122949"},{"key":"e_1_2_2_61_1","volume-title":"Proceedings of the 33rd European Conference on Object-Oriented Programming (ECOOP\u201919)","author":"Shaikhha Amir","year":"2019","unstructured":"Amir Shaikhha and Lionel Parreaux . 2019 . Finally, a Polymorphic Linear Algebra Language . In Proceedings of the 33rd European Conference on Object-Oriented Programming (ECOOP\u201919) . Amir Shaikhha and Lionel Parreaux. 2019. Finally, a Polymorphic Linear Algebra Language. In Proceedings of the 33rd European Conference on Object-Oriented Programming (ECOOP\u201919)."},{"key":"e_1_2_2_62_1","unstructured":"Jeffrey Mark Siskind and Barak A Pearlmutter. 2005. Perturbation confusion and referential transparency: Correct functional implementation of forward-mode AD. (2005).  Jeffrey Mark Siskind and Barak A Pearlmutter. 2005. Perturbation confusion and referential transparency: Correct functional implementation of forward-mode AD. (2005)."},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10990-008-9037-1"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/2854038.2854060"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2784731.2784754"},{"key":"e_1_2_2_67_1","unstructured":"Arvind Sujeeth HyoukJoong Lee Kevin Brown Tiark Rompf Hassan Chafi Michael Wu Anand Atreya Martin Odersky and Kunle Olukotun. 2011. OptiML: An Implicitly Parallel Domain-Specific Language for Machine Learning (ICML \u201911). 609\u2013616.   Arvind Sujeeth HyoukJoong Lee Kevin Brown Tiark Rompf Hassan Chafi Michael Wu Anand Atreya Martin Odersky and Kunle Olukotun. 2011. OptiML: An Implicitly Parallel Domain-Specific Language for Machine Learning (ICML \u201911). 609\u2013616."},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/581478.581491"},{"key":"e_1_2_2_69_1","volume-title":"Defunctionalizing Push Arrays (FHPC \u201914). ACM","author":"Svensson Bo Joel","unstructured":"Bo Joel Svensson and Josef Svenningsson . 2014. Defunctionalizing Push Arrays (FHPC \u201914). ACM , NY , USA , 43\u201352. Bo Joel Svensson and Josef Svenningsson. 2014. Defunctionalizing Push Arrays (FHPC \u201914). ACM, NY, USA, 43\u201352."},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0304-3975(00)00053-0"},{"key":"e_1_2_2_71_1","volume-title":"Inter. workshop on vision algorithms. Springer, 298\u2013372","author":"Triggs Bill","year":"1999","unstructured":"Bill Triggs , Philip F McLauchlan , Richard I Hartley , and Andrew W Fitzgibbon . 1999 . Bundle adjustment\u2014a modern synthesis . In Inter. workshop on vision algorithms. Springer, 298\u2013372 . Bill Triggs, Philip F McLauchlan, Richard I Hartley, and Andrew W Fitzgibbon. 1999. Bundle adjustment\u2014a modern synthesis. In Inter. workshop on vision algorithms. Springer, 298\u2013372."},{"key":"e_1_2_2_72_1","volume-title":"Deforestation: Transforming programs to eliminate trees. In ESOP\u201988","author":"Wadler Philip","year":"1988","unstructured":"Philip Wadler . 1988 . Deforestation: Transforming programs to eliminate trees. In ESOP\u201988 . Springer , 344\u2013358. Philip Wadler. 1988. Deforestation: Transforming programs to eliminate trees. In ESOP\u201988. Springer, 344\u2013358."},{"key":"e_1_2_2_73_1","unstructured":"Fei Wang James Decker Xilun Wu Gregory Essertel and Tiark Rompf. 2018. Backpropagation with Callbacks: Foundations for Efficient and Expressive Differentiable Programming. In Advances in Neural Information Processing Systems. 10200\u2013 10211.   Fei Wang James Decker Xilun Wu Gregory Essertel and Tiark Rompf. 2018. Backpropagation with Callbacks: Foundations for Efficient and Expressive Differentiable Programming. In Advances in Neural Information Processing Systems. 10200\u2013 10211."},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/3104990"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_50"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3341701","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3341701","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:43:23Z","timestamp":1750207403000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3341701"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,26]]},"references-count":74,"journal-issue":{"issue":"ICFP","published-print":{"date-parts":[[2019,7,26]]}},"alternative-id":["10.1145\/3341701"],"URL":"https:\/\/doi.org\/10.1145\/3341701","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,26]]},"assertion":[{"value":"2019-07-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}