{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T02:48:36Z","timestamp":1787453316371,"version":"build-2736575974"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA","license":[{"start":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T00:00:00Z","timestamp":1570665600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2019,10,10]]},"abstract":"<jats:p>Developers nowadays have to contend with a growing number of APIs. While in the long-term they are very useful to developers, many modern APIs have an incredibly steep learning curve, due to their hundreds of functions handling many arguments, obscure documentation, and frequently changing semantics. For APIs that perform data transformations, novices can often provide an I\/O example demonstrating the desired transformation, but may be stuck on how to translate it to the API. A programming-by-example synthesis engine that takes such I\/O examples and directly produces programs in the target API could help such novices. Such an engine presents unique challenges due to the breadth of real-world APIs, and the often-complex constraints over function arguments. We present a generator-based synthesis approach to contend with these problems. This approach uses a program candidate generator, which encodes basic constraints on the space of programs. We introduce neural-backed operators which can be seamlessly integrated into the program generator. To improve the efficiency of the search, we simply use these operators at non-deterministic decision points, instead of relying on domain-specific heuristics. We implement this technique for the Python pandas library in AutoPandas. AutoPandas supports 119 pandas dataframe transformation functions. We evaluate AutoPandas on 26 real-world benchmarks and find it solves 17 of them.<\/jats:p>","DOI":"10.1145\/3360594","type":"journal-article","created":{"date-parts":[[2019,10,11]],"date-time":"2019-10-11T14:53:33Z","timestamp":1570805613000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":62,"title":["AutoPandas: neural-backed generators for program synthesis"],"prefix":"10.1145","volume":"3","author":[{"given":"Rohan","family":"Bavishi","sequence":"first","affiliation":[{"name":"University of California at Berkeley, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Caroline","family":"Lemieux","sequence":"additional","affiliation":[{"name":"University of California at Berkeley, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roy","family":"Fox","sequence":"additional","affiliation":[{"name":"University of California at Irvine, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Koushik","family":"Sen","sequence":"additional","affiliation":[{"name":"University of California at Berkeley, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ion","family":"Stoica","sequence":"additional","affiliation":[{"name":"University of California at Berkeley, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,10,10]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Accessed October 11th","year":"2018","unstructured":"2014. The pandas project. https:\/\/pandas.pydata.org . Accessed October 11th , 2018 . 2014. The pandas project. https:\/\/pandas.pydata.org . Accessed October 11th, 2018."},{"key":"e_1_2_2_2_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=BJOFETxR-","author":"Allamanis Miltiadis","year":"2018","unstructured":"Miltiadis Allamanis , Marc Brockschmidt , and Mahmoud Khademi . 2018 . Learning to Represent Programs with Graphs . In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=BJOFETxR- Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi. 2018. Learning to Represent Programs with Graphs. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=BJOFETxR-"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/FMCAD.2013.6679385"},{"key":"e_1_2_2_4_1","volume-title":"DeepCoder: Learning to Write Programs. CoRR abs\/1611.01989","author":"Balog Matej","year":"2016","unstructured":"Matej Balog , Alexander L. Gaunt , Marc Brockschmidt , Sebastian Nowozin , and Daniel Tarlow . 2016. DeepCoder: Learning to Write Programs. CoRR abs\/1611.01989 ( 2016 ). arXiv: 1611.01989 http:\/\/arxiv.org\/abs\/1611.01989 Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2016. DeepCoder: Learning to Write Programs. CoRR abs\/1611.01989 (2016). arXiv: 1611.01989 http:\/\/arxiv.org\/abs\/1611.01989"},{"key":"e_1_2_2_5_1","volume-title":"Deep Reinforcement Fuzzing. CoRR abs\/1801.04589","author":"B\u00f6ttinger Konstantin","year":"2018","unstructured":"Konstantin B\u00f6ttinger , Patrice Godefroid , and Rishabh Singh . 2018. Deep Reinforcement Fuzzing. CoRR abs\/1801.04589 ( 2018 ). arXiv: 1801.04589 http:\/\/arxiv.org\/abs\/1801.04589 Konstantin B\u00f6ttinger, Patrice Godefroid, and Rishabh Singh. 2018. Deep Reinforcement Fuzzing. CoRR abs\/1801.04589 (2018). arXiv: 1801.04589 http:\/\/arxiv.org\/abs\/1801.04589"},{"key":"e_1_2_2_6_1","volume-title":"Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis. CoRR abs\/1805.04276","author":"Bunel Rudy","year":"2018","unstructured":"Rudy Bunel , Matthew J. Hausknecht , Jacob Devlin , Rishabh Singh , and Pushmeet Kohli . 2018. Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis. CoRR abs\/1805.04276 ( 2018 ). arXiv: 1805.04276 http: \/\/arxiv.org\/abs\/1805.04276 Rudy Bunel, Matthew J. Hausknecht, Jacob Devlin, Rishabh Singh, and Pushmeet Kohli. 2018. Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis. CoRR abs\/1805.04276 (2018). arXiv: 1805.04276 http: \/\/arxiv.org\/abs\/1805.04276"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1179"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/351240.351266"},{"key":"e_1_2_2_9_1","volume-title":"Learning Combinatorial Optimization Algorithms over Graphs. CoRR abs\/1704.01665","author":"Dai Hanjun","year":"2017","unstructured":"Hanjun Dai , Elias B. Khalil , Yuyu Zhang , Bistra Dilkina , and Le Song . 2017. Learning Combinatorial Optimization Algorithms over Graphs. CoRR abs\/1704.01665 ( 2017 ). arXiv: 1704.01665 http:\/\/arxiv.org\/abs\/1704.01665 Hanjun Dai, Elias B. Khalil, Yuyu Zhang, Bistra Dilkina, and Le Song. 2017. Learning Combinatorial Optimization Algorithms over Graphs. CoRR abs\/1704.01665 (2017). arXiv: 1704.01665 http:\/\/arxiv.org\/abs\/1704.01665"},{"key":"e_1_2_2_10_1","volume-title":"ICML","author":"Devlin Jacob","year":"2017","unstructured":"Jacob Devlin , Jonathan Uesato , Surya Bhupatiraju , Rishabh Singh , Abdelrahman Mohamed , and Pushmeet Kohli . 2017 . RobustFill: Neural Program Learning under Noisy I\/O . In ICML 2017. https:\/\/www.microsoft.com\/en-us\/research\/ publication\/robustfill-neural-program-learning-noisy-io\/ Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdelrahman Mohamed, and Pushmeet Kohli. 2017. RobustFill: Neural Program Learning under Noisy I\/O. In ICML 2017. https:\/\/www.microsoft.com\/en-us\/research\/ publication\/robustfill-neural-program-learning-noisy-io\/"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3192366.3192382"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3140587.3062351"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2593882.2593900"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1926385.1926423"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.14778\/3231751.3231766"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806833"},{"key":"e_1_2_2_17_1","unstructured":"A. Kalyan A. Mohta O. Polozov D. Batra P. Jain and S. Gulwani. 2018. Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples. ArXiv e-prints (April 2018). arXiv: cs.AI\/1804.01186  A. Kalyan A. Mohta O. Polozov D. Batra P. Jain and S. Gulwani. 2018. Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples. ArXiv e-prints (April 2018). arXiv: cs.AI\/1804.01186"},{"key":"e_1_2_2_18_1","volume-title":"Adam: A Method for Stochastic Optimization. ArXiv e-prints (Dec.","author":"Kingma D. P.","year":"2014","unstructured":"D. P. Kingma and J. Ba . 2014 . Adam: A Method for Stochastic Optimization. ArXiv e-prints (Dec. 2014). arXiv: 1412.6980 D. P. Kingma and J. Ba. 2014. Adam: A Method for Stochastic Optimization. ArXiv e-prints (Dec. 2014). arXiv: 1412.6980"},{"key":"e_1_2_2_19_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=ByxBFsRqYm","author":"Kool Wouter","year":"2019","unstructured":"Wouter Kool , Herke van Hoof , and Max Welling . 2019 . Attention, Learn to Solve Routing Problems! . In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=ByxBFsRqYm Wouter Kool, Herke van Hoof, and Max Welling. 2019. Attention, Learn to Solve Routing Problems!. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=ByxBFsRqYm"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2594291.2594333"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3192366.3192410"},{"key":"e_1_2_2_22_1","volume-title":"Zemel","author":"Li Yujia","year":"2015","unstructured":"Yujia Li , Daniel Tarlow , Marc Brockschmidt , and Richard S . Zemel . 2015 . Gated Graph Sequence Neural Networks. CoRR abs\/1511.05493 (2015). arXiv: 1511.05493 http:\/\/arxiv.org\/abs\/1511.05493 Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. 2015. Gated Graph Sequence Neural Networks. CoRR abs\/1511.05493 (2015). arXiv: 1511.05493 http:\/\/arxiv.org\/abs\/1511.05493"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3092703.3092711"},{"key":"e_1_2_2_24_1","volume-title":"Accessed October 17th","year":"2017","unstructured":"Microsoft. 2017 . Gated Graph Neural Network Samples. https:\/\/github.com\/Microsoft\/gated-graph-neural-network-samples . Accessed October 17th , 2018. Microsoft. 2017. Gated Graph Neural Network Samples. https:\/\/github.com\/Microsoft\/gated-graph-neural-network-samples. Accessed October 17th, 2018."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330576"},{"key":"e_1_2_2_26_1","volume-title":"NeuroSymbolic Program Synthesis. In ICLR","author":"Parisotto Emilio","year":"2017","unstructured":"Emilio Parisotto , Abdelrahman Mohamed , Rishabh Singh , Lihong Li , Denny Zhou , and Pushmeet Kohli . 2017 . NeuroSymbolic Program Synthesis. In ICLR 2017. https:\/\/www.microsoft.com\/en-us\/research\/publication\/neuro-symbolicprogram-synthesis-2\/ Emilio Parisotto, Abdelrahman Mohamed, Rishabh Singh, Lihong Li, Denny Zhou, and Pushmeet Kohli. 2017. NeuroSymbolic Program Synthesis. In ICLR 2017. https:\/\/www.microsoft.com\/en-us\/research\/publication\/neuro-symbolicprogram-synthesis-2\/"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980983.2908093"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2814270.2814310"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2594291.2594321"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.44"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908080.2908102"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1168857.1168907"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3158151"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3187009.3177735"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133887"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3360594","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3360594","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:22:59Z","timestamp":1750202579000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3360594"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,10]]},"references-count":35,"journal-issue":{"issue":"OOPSLA","published-print":{"date-parts":[[2019,10,10]]}},"alternative-id":["10.1145\/3360594"],"URL":"https:\/\/doi.org\/10.1145\/3360594","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,10]]},"assertion":[{"value":"2019-10-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}