{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T11:10:03Z","timestamp":1770289803757,"version":"3.49.0"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"ICFP","license":[{"start":{"date-parts":[[2021,8,19]],"date-time":"2021-08-19T00:00:00Z","timestamp":1629331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2021,8,22]]},"abstract":"<jats:p>We present a novel programming language design that attempts to combine the clarity and safety of high-level functional languages with the efficiency and parallelism of low-level numerical languages. We treat arrays as eagerly-memoized functions on typed index sets, allowing abstract function manipulations, such as currying, to work on arrays. In contrast to composing primitive bulk-array operations, we argue for an explicit nested indexing style that mirrors application of functions to arguments. We also introduce a fine-grained typed effects system which affords concise and automatically-parallelized in-place updates. Specifically, an associative accumulation effect allows reverse-mode automatic differentiation of in-place updates in a way that preserves parallelism. Empirically, we benchmark against the Futhark array programming language, and demonstrate that aggressive inlining and type-driven compilation allows array programs to be written in an expressive, \"pointful\" style with little performance penalty.<\/jats:p>","DOI":"10.1145\/3473593","type":"journal-article","created":{"date-parts":[[2021,8,19]],"date-time":"2021-08-19T10:44:29Z","timestamp":1629369869000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["Getting to the point: index sets and parallelism-preserving autodiff for pointful array programming"],"prefix":"10.1145","volume":"5","author":[{"given":"Adam","family":"Paszke","sequence":"first","affiliation":[{"name":"Google Research, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel D.","family":"Johnson","sequence":"additional","affiliation":[{"name":"Google Research, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Duvenaud","sequence":"additional","affiliation":[{"name":"University of Toronto, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dimitrios","family":"Vytiniotis","sequence":"additional","affiliation":[{"name":"DeepMind, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexey","family":"Radul","sequence":"additional","affiliation":[{"name":"Google Research, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthew J.","family":"Johnson","sequence":"additional","affiliation":[{"name":"Google Research, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Ragan-Kelley","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dougal","family":"Maclaurin","sequence":"additional","affiliation":[{"name":"Google Research, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,8,19]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201916)","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi , Paul Barham , Jianmin Chen , Zhifeng Chen , Andy Davis , Jeffrey Dean , Matthieu Devin , Sanjay Ghemawat , Geoffrey Irving , Michael Isard , Manjunath Kudlur , Josh Levenberg , Rajat Monga , Sherry Moore , Derek G. Murray , Benoit Steiner , Paul Tucker , Vijay Vasudevan , Pete Warden , Martin Wicke , Yuan Yu , and Xiaoqiang Zheng . 2016 . TensorFlow: A System for Large-Scale Machine Learning . In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201916) . USENIX Association, USA. 265\u2013283. isbn:978 1931971331 Mart\u00edn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201916). USENIX Association, USA. 265\u2013283. isbn:9781931971331"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/3122009.3242010"},{"key":"e_1_2_2_3_1","volume-title":"Proceedings of the Python for scientific computing conference (SciPy). 4, 1\u20137.","author":"Bergstra James","year":"2010","unstructured":"James Bergstra , Olivier Breuleux , Fr\u00e9d\u00e9ric Bastien , Pascal Lamblin , Razvan Pascanu , Guillaume Desjardins , Joseph Turian , David Warde-Farley , and Yoshua Bengio . 2010 . Theano: a CPU and GPU math expression compiler . In Proceedings of the Python for scientific computing conference (SciPy). 4, 1\u20137. James Bergstra, Olivier Breuleux, Fr\u00e9d\u00e9ric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. 2010. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for scientific computing conference (SciPy). 4, 1\u20137."},{"key":"e_1_2_2_4_1","unstructured":"Gilbert Bernstein Michael Mara Tzu-Mao Li Dougal Maclaurin and Jonathan Ragan-Kelley. 2020. Differentiating a Tensor Language. arxiv:2008.11256.  Gilbert Bernstein Michael Mara Tzu-Mao Li Dougal Maclaurin and Jonathan Ragan-Kelley. 2020. Differentiating a Tensor Language. arxiv:2008.11256."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1137\/141000671"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1155\/1992\/717832"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/865114"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1375581.1375595"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3428194"},{"key":"e_1_2_2_10_1","volume-title":"Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang.","author":"Bradbury James","year":"2018","unstructured":"James Bradbury , Roy Frostig , Peter Hawkins , Matthew James Johnson , Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. 2018 . JAX: composable transformations of Python +NumPy programs. http:\/\/github.com\/google\/jax James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. 2018. JAX: composable transformations of Python+NumPy programs. http:\/\/github.com\/google\/jax"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1926354.1926358"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2009.5306797"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236765"},{"key":"e_1_2_2_14_1","volume-title":"POPL 2021 workshop on Languages for Inference.","author":"Frostig Roy","year":"2021","unstructured":"Roy Frostig , Matthew Johnson , Dougal Maclaurin , Adam Paszke , and Alexey Radul . 2021 . Decomposing reverse-mode automatic differentiation. In LAFI \u201921 : POPL 2021 workshop on Languages for Inference. Roy Frostig, Matthew Johnson, Dougal Maclaurin, Adam Paszke, and Alexey Radul. 2021. Decomposing reverse-mode automatic differentiation. In LAFI \u201921: POPL 2021 workshop on Languages for Inference."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898717761"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0129626412500107"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-020-2649-2"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2450153.2450158"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC41405.2020.00101"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2935323.2935326"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3062341.3062354"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-18506-0_7"},{"key":"e_1_2_2_23_1","volume-title":"DiffTaichi: Differentiable Programming for Physical Simulation. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=B1eB5xSFvr","author":"Hu Yuanming","year":"2020","unstructured":"Yuanming Hu , Luke Anderson , Tzu-Mao Li , Qi Sun , Nathan Carr , Jonathan Ragan-Kelley , and Fredo Durand . 2020 . DiffTaichi: Differentiable Programming for Physical Simulation. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=B1eB5xSFvr Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, and Fredo Durand. 2020. DiffTaichi: Differentiable Programming for Physical Simulation. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=B1eB5xSFvr"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356506"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3337821.3337906"},{"key":"e_1_2_2_26_1","volume-title":"Don\u2019t Unroll Adjoint: Differentiating SSA-Form Programs. CoRR, abs\/1810.07951","author":"Innes Michael","year":"2018","unstructured":"Michael Innes . 2018. Don\u2019t Unroll Adjoint: Differentiating SSA-Form Programs. CoRR, abs\/1810.07951 ( 2018 ), arxiv:1810.07951. Michael Innes. 2018. Don\u2019t Unroll Adjoint: Differentiating SSA-Form Programs. CoRR, abs\/1810.07951 (2018), arxiv:1810.07951."},{"key":"e_1_2_2_27_1","volume-title":"A Programming Language","author":"Iverson Kenneth E.","unstructured":"Kenneth E. Iverson . 1962. A Programming Language . John Wiley & Sons, Inc. , USA. isbn:978-0-471-43014-8 Kenneth E. Iverson. 1962. A Programming Language. John Wiley & Sons, Inc., USA. isbn:978-0-471-43014-8"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3122948.3122952"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/178243.178246"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.4204\/eptcs.153.8"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201383"},{"key":"e_1_2_2_32_1","volume-title":"AutoML workshop.","author":"Maclaurin Dougal","year":"2014","unstructured":"Dougal Maclaurin , David Duvenaud , and Ryan P Adams . 2014 . Autograd: Effortless gradients in numpy. ICML \u201915 AutoML workshop. Dougal Maclaurin, David Duvenaud, and Ryan P Adams. 2014. Autograd: Effortless gradients in numpy. ICML \u201915 AutoML workshop."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1017\/S095679681900008X"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-16478-1_5"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2500365.2500595"},{"key":"e_1_2_2_36_1","volume-title":"The Definition of Standard ML","author":"Milner Robin","year":"1814","unstructured":"Robin Milner , Mads Tofte , and David Macqueen . 1997. The Definition of Standard ML . MIT Press , Cambridge, MA, USA . isbn:026263 1814 Robin Milner, Mads Tofte, and David Macqueen. 1997. The Definition of Standard ML. MIT Press, Cambridge, MA, USA. isbn:0262631814"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1863543.1863588"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2847538.2847541"},{"key":"e_1_2_2_39_1","volume-title":"High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , Alban Desmaison , Andreas K\u00f6pf , Edward Yang , Zachary DeVito , Martin Raison , Alykhan Tejani , Sasank Chilamkurthy , Benoit Steiner , Lu Fang , Junjie Bai , and Soumith Chintala . 2019 . PyTorch: An Imperative Style , High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 , NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 8024\u20138035. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K\u00f6pf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 8024\u20138035."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1330017.1330018"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-89330-1_10"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0956796802004331"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0956796806006034"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499370.2462176"},{"key":"e_1_2_2_45_1","unstructured":"Sam Ritchie and Gerald Jay Sussman. 2021. AD on Higher Order Functions. Unpublished note.  Sam Ritchie and Gerald Jay Sussman. 2021. AD on Higher Order Functions. Unpublished note."},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3211346.3211348"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341701"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-54833-8_3"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-09766-4_190"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/CGO.2017.7863730"},{"key":"e_1_2_2_51_1","volume-title":"Parboil: A Revised Benchmark Suite for Scientific and Commercial Throughput Computing.","author":"Stratton J. A.","year":"2012","unstructured":"J. A. Stratton , Christopher I. Rodrigues , I- Jui Sung , Nady Obeid , Li-Wen Chang , N. Anssari , G. Liu , and W. Hwu . 2012 . Parboil: A Revised Benchmark Suite for Scientific and Commercial Throughput Computing. J. A. Stratton, Christopher I. Rodrigues, I-Jui Sung, Nady Obeid, Li-Wen Chang, N. Anssari, G. Liu, and W. Hwu. 2012. Parboil: A Revised Benchmark Suite for Scientific and Commercial Throughput Computing."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2034773.2034811"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330756"},{"key":"e_1_2_2_54_1","volume-title":"Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions. arxiv:1802.04730.","author":"Vasilache Nicolas","year":"2018","unstructured":"Nicolas Vasilache , Oleksandr Zinenko , Theodoros Theodoridis , Priya Goyal , Zachary DeVito , William S. Moses , Sven Verdoolaege , Andrew Adams , and Albert Cohen . 2018 . Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions. arxiv:1802.04730. Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary DeVito, William S. Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. 2018. Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions. arxiv:1802.04730."}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3473593","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3473593","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:16Z","timestamp":1750195696000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3473593"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,19]]},"references-count":54,"journal-issue":{"issue":"ICFP","published-print":{"date-parts":[[2021,8,22]]}},"alternative-id":["10.1145\/3473593"],"URL":"https:\/\/doi.org\/10.1145\/3473593","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,19]]},"assertion":[{"value":"2021-08-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}