{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T02:13:47Z","timestamp":1775873627603,"version":"3.50.1"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"PLDI","license":[{"start":{"date-parts":[[2023,6,6]],"date-time":"2023-06-06T00:00:00Z","timestamp":1686009600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["CCF-1811865,CCF-1918889"],"award-info":[{"award-number":["CCF-1811865,CCF-1918889"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2023,6,6]]},"abstract":"<jats:p>This paper presents a new synthesis-based approach for batch image processing. Unlike existing tools that can only apply global edits to the entire image, our method can apply fine-grained edits to individual objects within the image. For example, our method can selectively blur or crop specific objects that have a certain property. To facilitate such fine-grained image editing tasks, we propose a neuro-symbolic domain-specific language (DSL) that combines pre-trained neural networks for image classification with other language constructs that enable symbolic reasoning. Our method can automatically learn programs in this DSL from user demonstrations by utilizing a novel synthesis algorithm. We have implemented the proposed technique in a tool called ImageEye and evaluated it on 50 image editing tasks. Our evaluation shows that ImageEye is able to automate 96% of these tasks.<\/jats:p>","DOI":"10.1145\/3591248","type":"journal-article","created":{"date-parts":[[2023,6,6]],"date-time":"2023-06-06T20:06:24Z","timestamp":1686081984000},"page":"686-711","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["ImageEye: Batch Image Processing using Program Synthesis"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7688-6133","authenticated-orcid":false,"given":"Celeste","family":"Barnaby","sequence":"first","affiliation":[{"name":"University of Texas at Austin, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4680-5157","authenticated-orcid":false,"given":"Qiaochu","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Texas at Austin, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2456-217X","authenticated-orcid":false,"given":"Roopsha","family":"Samanta","sequence":"additional","affiliation":[{"name":"Purdue University, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8006-1230","authenticated-orcid":false,"given":"I\u015f\u0131l","family":"Dillig","sequence":"additional","affiliation":[{"name":"University of Texas at Austin, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,6,6]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Recursive Program Synthesis","author":"Albarghouthi Aws","unstructured":"Aws Albarghouthi , Sumit Gulwani , and Zachary Kincaid . 2013. Recursive Program Synthesis . In Computer Aided Verification, Natasha Sharygina and Helmut Veith (Eds.). Springer Berlin Heidelberg , Berlin, Heidelberg . 934\u2013950. isbn:978-3-642-39799-8 Aws Albarghouthi, Sumit Gulwani, and Zachary Kincaid. 2013. Recursive Program Synthesis. In Computer Aided Verification, Natasha Sharygina and Helmut Veith (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 934\u2013950. isbn:978-3-642-39799-8"},{"key":"e_1_2_1_2_1","volume-title":"Tools and Algorithms for the Construction and Analysis of Systems. 10205","author":"Alur Rajeev","unstructured":"Rajeev Alur , Arjun Radhakrishna , and Abhishek Udupa . 2017. Scaling Enumerative Program Synthesis via Divide and Conquer . In Tools and Algorithms for the Construction and Analysis of Systems. 10205 , Springer Berlin Heidelberg , Berlin, Heidelberg . 319\u2013336. isbn:978-3-662-54576-8 Rajeev Alur, Arjun Radhakrishna, and Abhishek Udupa. 2017. Scaling Enumerative Program Synthesis via Divide and Conquer. In Tools and Algorithms for the Construction and Analysis of Systems. 10205, Springer Berlin Heidelberg, Berlin, Heidelberg. 319\u2013336. isbn:978-3-662-54576-8"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2008.10.024"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-99524-9_24"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983990.2984020"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.7810841"},{"key":"#cr-split#-e_1_2_1_7_1.1","unstructured":"Celeste Barnaby Qiaochu Chen Roopsha Samanta and Isil Dillig. 2023. ImageEye: Batch Image Processing Using Program Synthesis. https:\/\/doi.org\/10.48550\/arXiv.2304.03253 10.48550\/arXiv.2304.03253"},{"key":"#cr-split#-e_1_2_1_7_1.2","unstructured":"Celeste Barnaby Qiaochu Chen Roopsha Samanta and Isil Dillig. 2023. ImageEye: Batch Image Processing Using Program Synthesis. https:\/\/doi.org\/10.48550\/arXiv.2304.03253"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22110-1_14"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30301-5_60"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-1986-6_17"},{"key":"e_1_2_1_11_1","volume-title":"Neural Photo Editing with Introspective Adversarial Networks. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=HkNKFiGex","author":"Brock Andrew","year":"2017","unstructured":"Andrew Brock , Theodore Lim , J.M. Ritchie , and Nick Weston . 2017 . Neural Photo Editing with Introspective Adversarial Networks. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=HkNKFiGex Andrew Brock, Theodore Lim, J.M. Ritchie, and Nick Weston. 2017. Neural Photo Editing with Introspective Adversarial Networks. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=HkNKFiGex"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133875"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3385412.3385988"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00916"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-444-88074-1.50011-1"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0747-7171(06)80002-7"},{"key":"e_1_2_1_17_1","unstructured":"R. Dillmann and H. Friedrich. 1996. Programming by demonstration: A machine learning approach to support skill acquision for robots. In Artificial Intelligence and Symbolic Mathematical Computation Jacques Calmet John A. Campbell and Jochen Pfalzgraf (Eds.). Springer Berlin Heidelberg Berlin Heidelberg. 87\u2013108. isbn:978-3-540-70740-0 \t\t\t\t  R. Dillmann and H. Friedrich. 1996. Programming by demonstration: A machine learning approach to support skill acquision for robots. In Artificial Intelligence and Symbolic Mathematical Computation Jacques Calmet John A. Campbell and Jochen Pfalzgraf (Eds.). Springer Berlin Heidelberg Berlin Heidelberg. 87\u2013108. isbn:978-3-540-70740-0"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3519939.3523711"},{"key":"e_1_2_1_19_1","volume-title":"Advances in Neural Information Processing Systems","author":"Ellis Kevin","year":"2018","unstructured":"Kevin Ellis , Daniel Ritchie , Armando Solar-Lezama , and Josh Tenenbaum . 2018. Learning to Infer Graphics Programs from Hand-Drawn Images . In Advances in Neural Information Processing Systems , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). 31, Curran Associates, Inc. . https:\/\/proceedings.neurips.cc\/paper\/ 2018 \/file\/6788076842014c83cedadbe6b0ba0314-Paper.pdf Kevin Ellis, Daniel Ritchie, Armando Solar-Lezama, and Josh Tenenbaum. 2018. Learning to Infer Graphics Programs from Hand-Drawn Images. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). 31, Curran Associates, Inc.. https:\/\/proceedings.neurips.cc\/paper\/2018\/file\/6788076842014c83cedadbe6b0ba0314-Paper.pdf"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3296979.3192382"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3062341.3062351"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2737924.2737977"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS51168.2021.9636540"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 37th International Conference on Machine Learning, Hal Daum\u00e9 III and Aarti Singh (Eds.) (Proceedings of Machine Learning Research","volume":"4506","author":"Huang Jiani","year":"2020","unstructured":"Jiani Huang , Calvin Smith , Osbert Bastani , Rishabh Singh , Aws Albarghouthi , and Mayur Naik . 2020 . Generating Programmatic Referring Expressions via Program Synthesis . In Proceedings of the 37th International Conference on Machine Learning, Hal Daum\u00e9 III and Aarti Singh (Eds.) (Proceedings of Machine Learning Research , Vol. 119). PMLR, 4495\u2013 4506 . https:\/\/proceedings.mlr.press\/v119\/huang20h.html Jiani Huang, Calvin Smith, Osbert Bastani, Rishabh Singh, Aws Albarghouthi, and Mayur Naik. 2020. Generating Programmatic Referring Expressions via Program Synthesis. In Proceedings of the 37th International Conference on Machine Learning, Hal Daum\u00e9 III and Aarti Singh (Eds.) (Proceedings of Machine Learning Research, Vol. 119). PMLR, 4495\u20134506. https:\/\/proceedings.mlr.press\/v119\/huang20h.html"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the IEEE International Conference on Computer Vision (ICCV).","author":"Johnson Justin","unstructured":"Justin Johnson , Bharath Hariharan , Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick. 2017. Inferring and Executing Programs for Visual Reasoning . In Proceedings of the IEEE International Conference on Computer Vision (ICCV). Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C. Lawrence Zitnick, and Ross Girshick. 2017. Inferring and Executing Programs for Visual Reasoning. In Proceedings of the IEEE International Conference on Computer Vision (ICCV)."},{"key":"e_1_2_1_26_1","volume-title":"Partial Evaluation and Automatic Program Generation","author":"Jones Neil D.","year":"2024","unstructured":"Neil D. Jones , Carsten K. Gomard , and Peter Sestoft . 1993. Partial Evaluation and Automatic Program Generation . Prentice-Hall, Inc. , USA. isbn:0130 2024 95 Neil D. Jones, Carsten K. Gomard, and Peter Sestoft. 1993. Partial Evaluation and Automatic Program Generation. Prentice-Hall, Inc., USA. isbn:0130202495"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917)","author":"Lample Guillaume","year":"2017","unstructured":"Guillaume Lample , Neil Zeghidour , Nicolas Usunier , Antoine Bordes , Ludovic Denoyer , and Marc\u2019Aurelio Ranzato . 2017 . Fader Networks: Manipulating Images by Sliding Attributes . In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917) . Curran Associates Inc., Red Hook, NY, USA. 5969\u20135978. isbn:9781510860964 Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, and Marc\u2019Aurelio Ranzato. 2017. Fader Networks: Manipulating Images by Sliding Attributes. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917). Curran Associates Inc., Red Hook, NY, USA. 5969\u20135978. isbn:9781510860964"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1025671410623"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/291080.291104"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2594291.2594333"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2993236.2993244"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1502650.1502667"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3408991"},{"key":"e_1_2_1_34_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rJgMlhRctm","author":"Mao Jiayuan","year":"2019","unstructured":"Jiayuan Mao , Chuang Gan , Pushmeet Kohli , Joshua B. Tenenbaum , and Jiajun Wu . 2019 . The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision . In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rJgMlhRctm Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, and Jiajun Wu. 2019. The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rJgMlhRctm"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3385412.3386012"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV) Workshops.","author":"Nazeri Kamyar","year":"2019","unstructured":"Kamyar Nazeri , Eric Ng , Tony Joseph , Faisal Qureshi , and Mehran Ebrahimi . 2019 . EdgeConnect: Structure Guided Image Inpainting using Edge Prediction . In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV) Workshops. Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Qureshi, and Mehran Ebrahimi. 2019. EdgeConnect: Structure Guided Image Inpainting using Edge Prediction. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV) Workshops."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2737924.2738007"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2814270.2814310"},{"key":"e_1_2_1_39_1","volume-title":"Rewriting techniques for program synthesis","author":"Reddy Uday S.","unstructured":"Uday S. Reddy . 1989. Rewriting techniques for program synthesis . In Rewriting Techniques and Applications, Nachum Dershowitz (Ed.). Springer Berlin Heidelberg , Berlin, Heidelberg . 388\u2013403. isbn:978-3-540-46149-4 Uday S. Reddy. 1989. Rewriting techniques for program synthesis. In Rewriting Techniques and Applications, Nachum Dershowitz (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg. 388\u2013403. isbn:978-3-540-46149-4"},{"key":"e_1_2_1_40_1","volume-title":"Reed and Nando de Freitas","author":"Scott","year":"2016","unstructured":"Scott E. Reed and Nando de Freitas . 2016 . Neural Programmer-Interpreters. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds .). arxiv:1511.06279 Scott E. Reed and Nando de Freitas. 2016. Neural Programmer-Interpreters. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). arxiv:1511.06279"},{"key":"e_1_2_1_41_1","volume-title":"Program Synthesis with Equivalence Reduction","author":"Smith Calvin","unstructured":"Calvin Smith and Aws Albarghouthi . 2019. Program Synthesis with Equivalence Reduction . In Verification, Model Checking, and Abstract Interpretation, Constantin Enea and Ruzica Piskac (Eds.). Springer International Publishing , Cham . 24\u201347. isbn:978-3-030-11245-5 Calvin Smith and Aws Albarghouthi. 2019. Program Synthesis with Equivalence Reduction. In Verification, Model Checking, and Abstract Interpretation, Constantin Enea and Ruzica Piskac (Eds.). Springer International Publishing, Cham. 24\u201347. isbn:978-3-030-11245-5"},{"key":"e_1_2_1_43_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rylNH20qFQ","author":"Tian Yonglong","year":"2019","unstructured":"Yonglong Tian , Andrew Luo , Xingyuan Sun , Kevin Ellis , William T. Freeman , Joshua B. Tenenbaum , and Jiajun Wu . 2019 . Learning to Infer and Execute 3D Shape Programs . In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rylNH20qFQ Yonglong Tian, Andrew Luo, Xingyuan Sun, Kevin Ellis, William T. Freeman, Joshua B. Tenenbaum, and Jiajun Wu. 2019. Learning to Infer and Execute 3D Shape Programs. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rylNH20qFQ"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2509578.2509586"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/289423.289425"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3062341.3062365"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00149"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00599"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3187009.3177735"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-emnlp.146"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning, Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.) (Proceedings of Machine Learning Research","volume":"7153","author":"Young Halley","year":"2019","unstructured":"Halley Young , Osbert Bastani , and Mayur Naik . 2019 . Learning Neurosymbolic Generative Models via Program Synthesis . In Proceedings of the 36th International Conference on Machine Learning, Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.) (Proceedings of Machine Learning Research , Vol. 97). PMLR, 7144\u2013 7153 . https:\/\/proceedings.mlr.press\/v97\/young19a.html Halley Young, Osbert Bastani, and Mayur Naik. 2019. Learning Neurosymbolic Generative Models via Program Synthesis. In Proceedings of the 36th International Conference on Machine Learning, Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.) (Proceedings of Machine Learning Research, Vol. 97). PMLR, 7144\u20137153. https:\/\/proceedings.mlr.press\/v97\/young19a.html"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00457"},{"key":"e_1_2_1_53_1","volume-title":"Computer Vision \u2013 ECCV","author":"Zhao Bo","year":"2018","unstructured":"Bo Zhao , Bo Chang , Zequn Jie , and Leonid Sigal . 2018. Modular Generative Adversarial Networks . In Computer Vision \u2013 ECCV 2018 , Vittorio Ferrari, Martial Hebert , Cristian Sminchisescu, and Yair Weiss (Eds.). Springer International Publishing , Cham. 157\u2013173. isbn:978-3-030-01264-9 Bo Zhao, Bo Chang, Zequn Jie, and Leonid Sigal. 2018. Modular Generative Adversarial Networks. In Computer Vision \u2013 ECCV 2018, Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). Springer International Publishing, Cham. 157\u2013173. isbn:978-3-030-01264-9"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3519939.3523712"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201285"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3591248","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3591248","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:47:47Z","timestamp":1750178867000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3591248"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,6]]},"references-count":55,"journal-issue":{"issue":"PLDI","published-print":{"date-parts":[[2023,6,6]]}},"alternative-id":["10.1145\/3591248"],"URL":"https:\/\/doi.org\/10.1145\/3591248","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,6]]},"assertion":[{"value":"2023-06-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}