{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,27]],"date-time":"2026-09-27T02:43:29Z","timestamp":1790477009455,"version":"4.1.0"},"reference-count":82,"publisher":"Association for Computing Machinery (ACM)","issue":"ICFP","license":[{"start":{"date-parts":[[2019,7,26]],"date-time":"2019-07-26T00:00:00Z","timestamp":1564099200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF award","award":["1553471 and 1564207"],"award-info":[{"award-number":["1553471 and 1564207"]}]},{"name":"DOE award","award":["DE-SC0018050"],"award-info":[{"award-number":["DE-SC0018050"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2019,7,26]]},"abstract":"<jats:p>Deep learning has seen tremendous success over the past decade in computer vision, machine translation, and gameplay. This success rests crucially on gradient-descent optimization and the ability to \u201clearn\u201d parameters of a neural network by backpropagating observed errors. However, neural network architectures are growing increasingly sophisticated and diverse, which motivates an emerging quest for even more general forms of differentiable programming, where arbitrary parameterized computations can be trained by gradient descent. In this paper, we take a fresh look at automatic differentiation (AD) techniques, and especially aim to demystify the reverse-mode form of AD that generalizes backpropagation in neural networks. We uncover a tight connection between reverse-mode AD and delimited continuations, which permits implementing reverse-mode AD purely via operator overloading and without managing any auxiliary data structures. We further show how this formulation of AD can be fruitfully combined with multi-stage programming (staging), leading to an efficient implementation that combines the performance benefits of deep learning frameworks based on explicit reified computation graphs (e.g., TensorFlow) with the expressiveness of pure library approaches (e.g., PyTorch).<\/jats:p>","DOI":"10.1145\/3341700","type":"journal-article","created":{"date-parts":[[2019,7,29]],"date-time":"2019-07-29T16:55:51Z","timestamp":1564419351000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":34,"title":["Demystifying differentiable programming: shift\/reset the penultimate backpropagator"],"prefix":"10.1145","volume":"3","author":[{"given":"Fei","family":"Wang","sequence":"first","affiliation":[{"name":"Purdue University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Zheng","sequence":"additional","affiliation":[{"name":"Purdue University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Decker","sequence":"additional","affiliation":[{"name":"Purdue University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xilun","family":"Wu","sequence":"additional","affiliation":[{"name":"Purdue University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gr\u00e9gory M.","family":"Essertel","sequence":"additional","affiliation":[{"name":"Purdue University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tiark","family":"Rompf","sequence":"additional","affiliation":[{"name":"Purdue University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,7,26]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3088525.3088527"},{"key":"e_1_2_2_2_1","volume-title":"Alexey Andreyevich Radul, and Jeffrey Mark Siskind","author":"Baydin Atilim Gunes","year":"2018","unstructured":"Atilim Gunes Baydin , Barak A. Pearlmutter , Alexey Andreyevich Radul, and Jeffrey Mark Siskind . 2018 . Automatic differentiation in machine learning: a survey. CoRR abs\/1502.05767 (2018). Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. 2018. Automatic differentiation in machine learning: a survey. CoRR abs\/1502.05767 (2018)."},{"key":"e_1_2_2_3_1","volume-title":"DiffSharp: An AD Library for .NET Languages. CoRR abs\/1611.03423","author":"Baydin Atilim G\u00fcnes","year":"2016","unstructured":"Atilim G\u00fcnes Baydin , Barak A. Pearlmutter , and Jeffrey Mark Siskind . 2016. DiffSharp: An AD Library for .NET Languages. CoRR abs\/1611.03423 ( 2016 ). Atilim G\u00fcnes Baydin, Barak A. Pearlmutter, and Jeffrey Mark Siskind. 2016. DiffSharp: An AD Library for .NET Languages. CoRR abs\/1611.03423 (2016)."},{"key":"e_1_2_2_4_1","unstructured":"L. M. Beda L. N. Korolev N. V. Sukkikh and T. S. Frolova. 1959. Programs for automatic differentiation for the machine BESM. Technical Report. Institute for Precise Mechanics and Computation Techniques Academy of Science Moscow USSR. (In Russian).  L. M. Beda L. N. Korolev N. V. Sukkikh and T. S. Frolova. 1959. Programs for automatic differentiation for the machine BESM. Technical Report. Institute for Precise Mechanics and Computation Techniques Academy of Science Moscow USSR. (In Russian)."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/141471.141483"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2854038.2854042"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/PACT.2011.15"},{"key":"e_1_2_2_8_1","volume-title":"Applied optimal control: Optimization, estimation, and control (revised edition)","author":"Bryson A","year":"1975","unstructured":"A Bryson and Yu-Chi Ho. 1975. Applied optimal control: Optimization, estimation, and control (revised edition) . Levittown, Pennsylvania : Taylor &amp; Francis ( 1975 ). A Bryson and Yu-Chi Ho. 1975. Applied optimal control: Optimization, estimation, and control (revised edition). Levittown, Pennsylvania: Taylor &amp; Francis (1975)."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1115\/1.3640537"},{"key":"e_1_2_2_10_1","volume-title":"Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01274","author":"Chen Tianqi","year":"2015","unstructured":"Tianqi Chen , Mu Li , Yutian Li , Min Lin , Naiyan Wang , Minjie Wang , Tianjun Xiao , Bing Xu , Chiyuan Zhang , and Zheng Zhang . 2015 . Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01274 (2015). Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. 2015. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01274 (2015)."},{"key":"e_1_2_2_11_1","volume-title":"TVM: An Automated End-to-End Optimizing Compiler for Deep Learning","author":"Chen Tianqi","year":"2018","unstructured":"Tianqi Chen , Thierry Moreau , Ziheng Jiang , Lianmin Zheng , Eddie Q. Yan , Haichen Shen , Meghan Cowan , Leyuan Wang , Yuwei Hu , Luis Ceze , Carlos Guestrin , and Arvind Krishnamurthy . 2018 . TVM: An Automated End-to-End Optimizing Compiler for Deep Learning . In OSDI. USENIX Association , 578\u2013594. Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Q. Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning. In OSDI. USENIX Association, 578\u2013594."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1093\/imanum\/12.2.135"},{"key":"e_1_2_2_13_1","unstructured":"Jason Chuang. 2013. Stanford Sentiment Treebank. https:\/\/nlp.stanford.edu\/sentiment\/treebank.html  Jason Chuang. 2013. Stanford Sentiment Treebank. https:\/\/nlp.stanford.edu\/sentiment\/treebank.html"},{"key":"e_1_2_2_14_1","volume-title":"NIPS Workshop.","author":"Collobert Ronan","year":"2011","unstructured":"Ronan Collobert , Koray Kavukcuoglu , and Cl\u00e9ment Farabet . 2011 . Torch7: A Matlab-like Environment for Machine Learning. In BigLearn , NIPS Workshop. Ronan Collobert, Koray Kavukcuoglu, and Cl\u00e9ment Farabet. 2011. Torch7: A Matlab-like Environment for Machine Learning. In BigLearn, NIPS Workshop."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/91556.91622"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0960129500001535"},{"key":"e_1_2_2_17_1","first-page":"397","article-title":"There and back again","volume":"66","author":"Danvy Olivier","year":"2005","unstructured":"Olivier Danvy and Mayer Goldberg . 2005 . There and back again . Fundamenta Informaticae 66 , 4 (2005), 397 \u2013 413 . Olivier Danvy and Mayer Goldberg. 2005. There and back again. Fundamenta Informaticae 66, 4 (2005), 397\u2013413.","journal-title":"Fundamenta Informaticae"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2007.10.007"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/773184.773202"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0304-3975(02)00733-8"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2021068"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1596550.1596579"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3110271"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236765"},{"key":"e_1_2_2_25_1","volume-title":"Finding structure in time. Cognitive science 14, 2","author":"Elman Jeffrey L","year":"1990","unstructured":"Jeffrey L Elman . 1990. Finding structure in time. Cognitive science 14, 2 ( 1990 ), 179\u2013211. Jeffrey L Elman. 1990. Finding structure in time. Cognitive science 14, 2 (1990), 179\u2013211."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/73560.73576"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/155090.155113"},{"key":"e_1_2_2_28_1","volume-title":"Backprop as Functor: A compositional perspective on supervised learning. arXiv preprint arXiv:1711.10455","author":"Fong Brendan","year":"2017","unstructured":"Brendan Fong , David I Spivak , and R\u00e9my Tuy\u00e9ras . 2017. Backprop as Functor: A compositional perspective on supervised learning. arXiv preprint arXiv:1711.10455 ( 2017 ). Brendan Fong, David I Spivak, and R\u00e9my Tuy\u00e9ras. 2017. Backprop as Functor: A compositional perspective on supervised learning. arXiv preprint arXiv:1711.10455 (2017)."},{"key":"e_1_2_2_29_1","volume-title":"Neural Turing Machines. CoRR abs\/1410.5401","author":"Graves Alex","year":"2014","unstructured":"Alex Graves , Greg Wayne , and Ivo Danihelka . 2014. Neural Turing Machines. CoRR abs\/1410.5401 ( 2014 ). arXiv: 1410.5401 http:\/\/arxiv.org\/abs\/1410.5401 Alex Graves, Greg Wayne, and Ivo Danihelka. 2014. Neural Turing Machines. CoRR abs\/1410.5401 (2014). arXiv: 1410.5401 http:\/\/arxiv.org\/abs\/1410.5401"},{"key":"e_1_2_2_30_1","volume-title":"Mustafa Suleyman, and Phil Blunsom.","author":"Grefenstette Edward","year":"2015","unstructured":"Edward Grefenstette , Karl Moritz Hermann , Mustafa Suleyman, and Phil Blunsom. 2015 . Learning to Transduce with Unbounded Memory. In Advances in Neural Information Processing Systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (Eds.). Curran Associates, Inc ., 1828\u20131836. http:\/\/papers.nips.cc\/paper\/5648-learning-totransduce-with-unbounded-memory.pdf Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, and Phil Blunsom. 2015. Learning to Transduce with Unbounded Memory. In Advances in Neural Information Processing Systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (Eds.). Curran Associates, Inc., 1828\u20131836. http:\/\/papers.nips.cc\/paper\/5648-learning-totransduce-with-unbounded-memory.pdf"},{"key":"e_1_2_2_31_1","volume-title":"Deep Residual Learning for Image Recognition","author":"He Kaiming","unstructured":"Kaiming He , Xiangyu Zhang , Shaoqing Ren , and Jian Sun . 2016. Deep Residual Learning for Image Recognition . In CVPR. IEEE Computer Society , 770\u2013778. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. IEEE Computer Society, 770\u2013778."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(88)90469-8"},{"key":"e_1_2_2_33_1","volume-title":"SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and &lt;1MB model size. CoRR abs\/1602.07360","author":"Iandola Forrest N.","year":"2016","unstructured":"Forrest N. Iandola , Matthew W. Moskewicz , Khalid Ashraf , Song Han , William J. Dally , and Kurt Keutzer . 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and &lt;1MB model size. CoRR abs\/1602.07360 ( 2016 ). Forrest N. Iandola, Matthew W. Moskewicz, Khalid Ashraf, Song Han, William J. Dally, and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and &lt;1MB model size. CoRR abs\/1602.07360 (2016)."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/512644.512652"},{"key":"e_1_2_2_36_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba . 2014 . Adam : A Method for Stochastic Optimization. CoRR abs\/1412.6980 (2014). arXiv: 1412.6980 http:\/\/arxiv.org\/abs\/1412.6980 Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs\/1412.6980 (2014). arXiv: 1412.6980 http:\/\/arxiv.org\/abs\/1412.6980"},{"key":"e_1_2_2_37_1","unstructured":"Alex Krizhevsky. 2012. Learning Multiple Layers of Features from Tiny Images. University of Toronto (05 2012).  Alex Krizhevsky. 2012. Learning Multiple Layers of Features from Tiny Images. University of Toronto (05 2012)."},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/178243.178246"},{"key":"e_1_2_2_39_1","unstructured":"Yann LeCun. 2018. Deep Learning est mort. Vive Differentiable Programming! https:\/\/www.facebook.com\/yann.lecun\/ posts\/10155003011462143 .  Yann LeCun. 2018. Deep Learning est mort. Vive Differentiable Programming! https:\/\/www.facebook.com\/yann.lecun\/ posts\/10155003011462143 ."},{"key":"e_1_2_2_40_1","unstructured":"Yann LeCun Bernhard E Boser John S Denker Donnie Henderson Richard E Howard Wayne E Hubbard and Lawrence D Jackel. 1990. Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems. 396\u2013404.   Yann LeCun Bernhard E Boser John S Denker Donnie Henderson Richard E Howard Wayne E Hubbard and Lawrence D Jackel. 1990. Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems. 396\u2013404."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01931367"},{"key":"e_1_2_2_42_1","volume-title":"Deep Learning with Dynamic Computation Graphs. ICLR","author":"Looks Moshe","year":"2017","unstructured":"Moshe Looks , Marcello Herreshoff , DeLesley Hutchins , and Peter Norvig . 2017. Deep Learning with Dynamic Computation Graphs. ICLR ( 2017 ). Moshe Looks, Marcello Herreshoff, DeLesley Hutchins, and Peter Norvig. 2017. Deep Learning with Dynamic Computation Graphs. ICLR (2017)."},{"key":"e_1_2_2_44_1","unstructured":"Graham Neubig Chris Dyer Yoav Goldberg Austin Matthews Waleed Ammar Antonios Anastasopoulos Miguel Ballesteros David Chiang Daniel Clothiaux Trevor Cohn Kevin Duh Manaal Faruqui Cynthia Gan Dan Garrette Yangfeng Ji Lingpeng Kong Adhiguna Kuncoro Gaurav Kumar Chaitanya Malaviya Paul Michel Yusuke Oda Matthew Richardson Naomi Saphra Swabha Swayamdipta and Pengcheng Yin. 2017a. DyNet: The Dynamic Neural Network Toolkit. CoRR abs\/1701.03980 (2017).  Graham Neubig Chris Dyer Yoav Goldberg Austin Matthews Waleed Ammar Antonios Anastasopoulos Miguel Ballesteros David Chiang Daniel Clothiaux Trevor Cohn Kevin Duh Manaal Faruqui Cynthia Gan Dan Garrette Yangfeng Ji Lingpeng Kong Adhiguna Kuncoro Gaurav Kumar Chaitanya Malaviya Paul Michel Yusuke Oda Matthew Richardson Naomi Saphra Swabha Swayamdipta and Pengcheng Yin. 2017a. DyNet: The Dynamic Neural Network Toolkit. CoRR abs\/1701.03980 (2017)."},{"key":"e_1_2_2_45_1","unstructured":"Graham Neubig Yoav Goldberg and Chris Dyer. 2017b. On-the-fly Operation Batching in Dynamic Computation Graphs. In NIPS. 3974\u20133984.   Graham Neubig Yoav Goldberg and Chris Dyer. 2017b. On-the-fly Operation Batching in Dynamic Computation Graphs. In NIPS. 3974\u20133984."},{"key":"e_1_2_2_46_1","unstructured":"John F Nolan. 1953. Analytical differentiation on a digital computer.  John F Nolan. 1953. Analytical differentiation on a digital computer."},{"key":"e_1_2_2_47_1","unstructured":"Christopher Olah. 2015. Neural Networks Types and Functional Programming. http:\/\/colah.github.io\/posts\/2015-09-NN-Types-FP\/ .  Christopher Olah. 2015. Neural Networks Types and Functional Programming. http:\/\/colah.github.io\/posts\/2015-09-NN-Types-FP\/ ."},{"key":"e_1_2_2_48_1","volume-title":"ONNX: Open Neural Network Exchange format. https:\/\/onnx.ai\/","author":"ONNX","year":"2017","unstructured":"ONNX working groups. 2017 . ONNX: Open Neural Network Exchange format. https:\/\/onnx.ai\/ ONNX working groups. 2017. ONNX: Open Neural Network Exchange format. https:\/\/onnx.ai\/"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2015.7178964"},{"key":"e_1_2_2_50_1","volume-title":"Massachusetts Institute of Technology, and Sloan School of Management","author":"Parker D.B.","year":"1985","unstructured":"D.B. Parker , Massachusetts Institute of Technology, and Sloan School of Management . 1985 . Learning Logic : Casting the Cortex of the Human Brain in Silicon. Massachusetts Institute of Technology, Center for Computational Research in Economics and Management Science. https:\/\/books.google.com\/books?id=2kS9GwAACAAJ D.B. Parker, Massachusetts Institute of Technology, and Sloan School of Management. 1985. Learning Logic: Casting the Cortex of the Human Brain in Silicon. Massachusetts Institute of Technology, Center for Computational Research in Economics and Management Science. https:\/\/books.google.com\/books?id=2kS9GwAACAAJ"},{"key":"e_1_2_2_51_1","unstructured":"Adam Paszke Sam Gross Soumith Chintala and Gregory Chanan. 2017a. PyTorch: Tensors and dynamic neural networks in Python with strong GPU acceleration. www.pytorch.org  Adam Paszke Sam Gross Soumith Chintala and Gregory Chanan. 2017a. PyTorch: Tensors and dynamic neural networks in Python with strong GPU acceleration. www.pytorch.org"},{"key":"e_1_2_2_52_1","unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017b. Automatic differentiation in PyTorch. (2017).  Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017b. Automatic differentiation in PyTorch. (2017)."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1330017.1330018"},{"key":"e_1_2_2_54_1","volume-title":"https:\/\/pytorch.org\/docs\/master\/jit.html {Online","author":"Script Torch","year":"2019","unstructured":"PyTorch. 2019. Torch Script . https:\/\/pytorch.org\/docs\/master\/jit.html {Online ; accessed 1- June - 2019 }. PyTorch. 2019. Torch Script. https:\/\/pytorch.org\/docs\/master\/jit.html {Online; accessed 1-June-2019}."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(98)00116-6"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010027404223"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3211346.3211348"},{"key":"e_1_2_2_58_1","volume-title":"A List of Successes That Can Change the World (Lecture Notes in Computer Science)","author":"Rompf Tiark","unstructured":"Tiark Rompf . 2016. The Essence of Multi-stage Evaluation in LMS . In A List of Successes That Can Change the World (Lecture Notes in Computer Science) , Vol. 9600 . Springer , 318\u2013335. Tiark Rompf. 2016. The Essence of Multi-stage Evaluation in LMS. In A List of Successes That Can Change the World (Lecture Notes in Computer Science), Vol. 9600. Springer, 318\u2013335."},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2784731.2784760"},{"key":"e_1_2_2_60_1","volume-title":"SNAPL (LIPIcs)","volume":"32","author":"Rompf Tiark","year":"2015","unstructured":"Tiark Rompf , Kevin J. Brown , HyoukJoong Lee , Arvind K. Sujeeth , Manohar Jonnalagedda , Nada Amin , Georg Ofenbeck , Alen Stojanov , Yannis Klonatos , Mohammad Dashti , Christoph Koch , Markus P\u00fcschel , and Kunle Olukotun . 2015 . Go Meta! A Case for Generative Programming and DSLs in Performance Critical Systems . In SNAPL (LIPIcs) , Vol. 32 . Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 238\u2013261. Tiark Rompf, Kevin J. Brown, HyoukJoong Lee, Arvind K. Sujeeth, Manohar Jonnalagedda, Nada Amin, Georg Ofenbeck, Alen Stojanov, Yannis Klonatos, Mohammad Dashti, Christoph Koch, Markus P\u00fcschel, and Kunle Olukotun. 2015. Go Meta! A Case for Generative Programming and DSLs in Performance Critical Systems. In SNAPL (LIPIcs), Vol. 32. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 238\u2013261."},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/1596550.1596596"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/1868294.1868314"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2429069.2429128"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.4204\/EPTCS.66.5"},{"key":"e_1_2_2_65_1","volume-title":"Glow: Graph Lowering Compiler Techniques for Neural Networks. CoRR abs\/1805.00907","author":"Rotem Nadav","year":"2018","unstructured":"Nadav Rotem , Jordan Fix , Saleem Abdulrasool , Summer Deng , Roman Dzhabarov , James Hegeman , Roman Levenstein , Bert Maher , Nadathur Satish , Jakob Olesen , Jongsoo Park , Artem Rakhov , and Misha Smelyanskiy . 2018 . Glow: Graph Lowering Compiler Techniques for Neural Networks. CoRR abs\/1805.00907 (2018). Nadav Rotem, Jordan Fix, Saleem Abdulrasool, Summer Deng, Roman Dzhabarov, James Hegeman, Roman Levenstein, Bert Maher, Nadathur Satish, Jakob Olesen, Jongsoo Park, Artem Rakhov, and Misha Smelyanskiy. 2018. Glow: Graph Lowering Compiler Techniques for Neural Networks. CoRR abs\/1805.00907 (2018)."},{"key":"e_1_2_2_66_1","volume-title":"Learning representations by back-propagating errors. nature 323, 6088","author":"Rumelhart David E","year":"1986","unstructured":"David E Rumelhart , Geoffrey E Hinton , and Ronald J Williams . 1986. Learning representations by back-propagating errors. nature 323, 6088 ( 1986 ), 533. David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning representations by back-propagating errors. nature 323, 6088 (1986), 533."},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2945397"},{"key":"e_1_2_2_68_1","volume-title":"Simon Peyton Jones, and Christoph Koch","author":"Shaikhha Amir","year":"2018","unstructured":"Amir Shaikhha , Andrew Fitzgibbon , Dimitrios Vytiniotis , Simon Peyton Jones, and Christoph Koch . 2018 . Efficient Differentiable Programming in a Functional Array-Processing Language. CoRR abs\/1806.02136 (2018). Amir Shaikhha, Andrew Fitzgibbon, Dimitrios Vytiniotis, Simon Peyton Jones, and Christoph Koch. 2018. Efficient Differentiable Programming in a Functional Array-Processing Language. CoRR abs\/1806.02136 (2018)."},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10990-008-9037-1"},{"key":"e_1_2_2_70_1","volume-title":"Pearlmutter","author":"Siskind Jeffrey Mark","year":"2016","unstructured":"Jeffrey Mark Siskind and Barak A . Pearlmutter . 2016 . Efficient Implementation of a Higher-Order Language with Built-In AD. CoRR abs\/1611.03416 (2016). Jeffrey Mark Siskind and Barak A. Pearlmutter. 2016. Efficient Implementation of a Higher-Order Language with Built-In AD. CoRR abs\/1611.03416 (2016)."},{"key":"e_1_2_2_72_1","volume-title":"OptiML: An Implicitly Parallel Domain-Specific Language for Machine Learning","author":"Sujeeth Arvind K.","unstructured":"Arvind K. Sujeeth , HyoukJoong Lee , Kevin J. Brown , Tiark Rompf , Hassan Chafi , Michael Wu , Anand R. Atreya , Martin Odersky , and Kunle Olukotun . 2011. OptiML: An Implicitly Parallel Domain-Specific Language for Machine Learning . In ICML. Omnipress , 609\u2013616. Arvind K. Sujeeth, HyoukJoong Lee, Kevin J. Brown, Tiark Rompf, Hassan Chafi, Michael Wu, Anand R. Atreya, Martin Odersky, and Kunle Olukotun. 2011. OptiML: An Implicitly Parallel Domain-Specific Language for Machine Learning. In ICML. Omnipress, 609\u2013616."},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0304-3975(00)00053-0"},{"key":"e_1_2_2_74_1","volume-title":"Manning","author":"Tai Kai Sheng","year":"2015","unstructured":"Kai Sheng Tai , Richard Socher , and Christopher D . Manning . 2015 . Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks. CoRR abs\/1503.00075 (2015). arXiv: 1503.00075 http:\/\/arxiv.org\/abs\/1503.00075 Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015. Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks. CoRR abs\/1503.00075 (2015). arXiv: 1503.00075 http:\/\/arxiv.org\/abs\/1503.00075"},{"key":"e_1_2_2_75_1","volume-title":"Swift For TensorFlow. https:\/\/www.tensorflow.org\/swift {Online","year":"2019","unstructured":"TensorFlow. 2019. Swift For TensorFlow. https:\/\/www.tensorflow.org\/swift {Online ; accessed 1- June - 2019 }. TensorFlow. 2019. Swift For TensorFlow. https:\/\/www.tensorflow.org\/swift {Online; accessed 1-June-2019}."},{"key":"e_1_2_2_76_1","volume-title":"Proceedings of workshop on machine learning systems (LearningSys) in the twenty-ninth annual conference on neural information processing systems (NIPS)","volume":"5","author":"Tokui Seiya","year":"2015","unstructured":"Seiya Tokui , Kenta Oono , Shohei Hido , and Justin Clayton . 2015 . Chainer: a next-generation open source framework for deep learning . In Proceedings of workshop on machine learning systems (LearningSys) in the twenty-ninth annual conference on neural information processing systems (NIPS) , Vol. 5 . Seiya Tokui, Kenta Oono, Shohei Hido, and Justin Clayton. 2015. Chainer: a next-generation open source framework for deep learning. In Proceedings of workshop on machine learning systems (LearningSys) in the twenty-ninth annual conference on neural information processing systems (NIPS), Vol. 5."},{"key":"e_1_2_2_77_1","volume-title":"Tangent: Automatic Differentiation Using Source Code Transformation in Python. ArXiv e-prints (Nov.","author":"van Merri\u00ebnboer B.","year":"2017","unstructured":"B. van Merri\u00ebnboer , A. B. Wiltschko , and D. Moldovan . 2017 . Tangent: Automatic Differentiation Using Source Code Transformation in Python. ArXiv e-prints (Nov. 2017). arXiv: cs.MS\/1711.02712 B. van Merri\u00ebnboer, A. B. Wiltschko, and D. Moldovan. 2017. Tangent: Automatic Differentiation Using Source Code Transformation in Python. ArXiv e-prints (Nov. 2017). arXiv: cs.MS\/1711.02712"},{"key":"e_1_2_2_78_1","unstructured":"Fei Wang James Decker Xilun Wu Gregory Essertel and Tiark Rompf. 2018a. Backpropagation with Callbacks: Foundations for Efficient and Expressive Differentiable Programming. In NeurIPS.   Fei Wang James Decker Xilun Wu Gregory Essertel and Tiark Rompf. 2018a. Backpropagation with Callbacks: Foundations for Efficient and Expressive Differentiable Programming. In NeurIPS."},{"key":"e_1_2_2_79_1","volume-title":"ICLR Workshop Track","author":"Wang Fei","year":"2018","unstructured":"Fei Wang and Tiark Rompf . 2018 . A Language and Compiler View on Differentiable Programming . ICLR Workshop Track (2018). https:\/\/openreview.net\/forum?id=SJxJtYkPG Fei Wang and Tiark Rompf. 2018. A Language and Compiler View on Differentiable Programming. ICLR Workshop Track (2018). https:\/\/openreview.net\/forum?id=SJxJtYkPG"},{"key":"e_1_2_2_80_1","volume-title":"Demystifying Differentiable Programming: Shift\/Reset the Penultimate Backpropagator. CoRR abs\/1803.10228","author":"Wang Fei","year":"2018","unstructured":"Fei Wang , Xilun Wu , Gr\u00e9gory M. Essertel , James M. Decker , and Tiark Rompf . 2018b. Demystifying Differentiable Programming: Shift\/Reset the Penultimate Backpropagator. CoRR abs\/1803.10228 ( 2018 ). Fei Wang, Xilun Wu, Gr\u00e9gory M. Essertel, James M. Decker, and Tiark Rompf. 2018b. Demystifying Differentiable Programming: Shift\/Reset the Penultimate Backpropagator. CoRR abs\/1803.10228 (2018)."},{"key":"e_1_2_2_81_1","volume-title":"DLVM: A modern compiler infrastructure for deep learning systems. CoRR abs\/1711.03016","author":"Wei Richard","year":"2017","unstructured":"Richard Wei , Vikram S. Adve , and Lane Schwartz . 2017 a. DLVM: A modern compiler infrastructure for deep learning systems. CoRR abs\/1711.03016 (2017). Richard Wei, Vikram S. Adve, and Lane Schwartz. 2017a. DLVM: A modern compiler infrastructure for deep learning systems. CoRR abs\/1711.03016 (2017)."},{"key":"e_1_2_2_82_1","volume-title":"NIPS AutoDiff Workshop.","author":"Wei Richard","year":"2017","unstructured":"Richard Wei , Lane Schwartz , and Vikram Adve . 2017 b. A modern compiler infrastructure for deep learning systems with adjoint code generation in a domain-specific IR . In NIPS AutoDiff Workshop. Richard Wei, Lane Schwartz, and Vikram Adve. 2017b. A modern compiler infrastructure for deep learning systems with adjoint code generation in a domain-specific IR. In NIPS AutoDiff Workshop."},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/355586.364791"},{"key":"e_1_2_2_85_1","volume-title":"Tangent: Source-to-Source Debuggable Derivatives. https:\/\/research.googleblog.com\/2017\/11\/tangentsource-to-source-debuggable.html","author":"Wiltschko Alex","year":"2017","unstructured":"Alex Wiltschko . 2017 . Tangent: Source-to-Source Debuggable Derivatives. https:\/\/research.googleblog.com\/2017\/11\/tangentsource-to-source-debuggable.html Alex Wiltschko. 2017. Tangent: Source-to-Source Debuggable Derivatives. https:\/\/research.googleblog.com\/2017\/11\/tangentsource-to-source-debuggable.html"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3341700","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3341700","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:43:23Z","timestamp":1750193003000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3341700"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,26]]},"references-count":82,"journal-issue":{"issue":"ICFP","published-print":{"date-parts":[[2019,7,26]]}},"alternative-id":["10.1145\/3341700"],"URL":"https:\/\/doi.org\/10.1145\/3341700","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,26]]},"assertion":[{"value":"2019-07-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}