{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T08:44:13Z","timestamp":1780994653059,"version":"3.54.1"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"POPL","license":[{"start":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T00:00:00Z","timestamp":1673222400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2023,1,9]]},"abstract":"<jats:p>In probabilistic programming languages (PPLs), a critical step in optimization-based inference methods is constructing, for a given model program, a trainable guide program.  \nSoundness and effectiveness of inference rely on constructing good guides, but the expressive power of a universal PPL poses challenges.  \nThis paper introduces an approach to automatically generating guides for deep amortized inference in a universal PPL.  \nGuides are generated using a type-directed translation per a novel behavioral type system.  \nGuide generation extracts and exploits independence structures using a syntactic approach to conditional independence, with a semantic account left to further work.  \nDespite the control-flow expressiveness allowed by the universal PPL, generated guides are guaranteed to satisfy a critical soundness condition and moreover, consistently improve training and inference over state-of-the-art baselines for a suite of benchmarks.<\/jats:p>","DOI":"10.1145\/3571243","type":"journal-article","created":{"date-parts":[[2023,1,11]],"date-time":"2023-01-11T21:58:14Z","timestamp":1673474294000},"page":"1454-1482","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Type-Preserving, Dependence-Aware Guide Generation for Sound, Effective Amortized Probabilistic Inference"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7371-3034","authenticated-orcid":false,"given":"Jianlin","family":"Li","sequence":"first","affiliation":[{"name":"University of Waterloo, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6033-9140","authenticated-orcid":false,"given":"Leni","family":"Aniva","sequence":"additional","affiliation":[{"name":"University of Waterloo, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7949-0406","authenticated-orcid":false,"given":"Pengyuan","family":"Shi","sequence":"additional","affiliation":[{"name":"University of Waterloo, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8206-4694","authenticated-orcid":false,"given":"Yizhou","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Waterloo, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,1,11]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2022. https:\/\/docs.pyro.ai\/en\/1.8.0\/infer.autoguide.html"},{"key":"e_1_2_1_2_1","volume-title":"Int\u2019l Conf. on Probabilistic Programming (PROBPROG). arxiv:2110","author":"Baudart Guillaume","year":"2021","unstructured":"Guillaume Baudart and Louis Mandel. 2021. Automatic guide generation for Stan via NumPyro. In Int\u2019l Conf. on Probabilistic Programming (PROBPROG). arxiv:2110.11790."},{"key":"e_1_2_1_3_1","article-title":"Pyro: Deep universal probabilistic programming","volume":"20","author":"Bingham Eli","year":"2019","unstructured":"Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D. Goodman. 2019. Pyro: Deep universal probabilistic programming. Journal of Machine Learning Research (JMLR), 20, 1 (2019), arxiv:1810.09538.","journal-title":"Journal of Machine Learning Research (JMLR)"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2951913.2951942"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3314221.3314642"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/24039.24041"},{"key":"e_1_2_1_7_1","volume-title":"Int\u2019l Conf. on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v37\/germain15","author":"Germain Mathieu","year":"2015","unstructured":"Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. 2015. MADE: Masked autoencoder for distribution estimation. In Int\u2019l Conf. on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v37\/germain15.pdf"},{"key":"e_1_2_1_8_1","volume-title":"Conf. on Uncertainty in Artificial Intelligence (UAI). arxiv:1206","author":"Goodman Noah","unstructured":"Noah Goodman, Vikash K. Mansinghka, Daniel M Roy, Keith Bonawitz, and Joshua B. Tenenbaum. 2008. Church: A language for generative models. In Conf. on Uncertainty in Artificial Intelligence (UAI). arxiv:1206.3255."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3490421"},{"key":"e_1_2_1_10_1","volume-title":"Alexander Bergholm, and Frank Wood.","author":"Harvey William","year":"2019","unstructured":"William Harvey, Andreas Munk, At\u0131l\u0131m G\u00fcne\u015f Baydin, Alexander Bergholm, and Frank Wood. 2019. Attention for inference compilation. arxiv:1910.11961."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.7761831"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0053567"},{"key":"e_1_2_1_14_1","volume-title":"Int\u2019l Conf. on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v80\/huang18d\/huang18d.pdf","author":"Huang Chin-Wei","year":"2018","unstructured":"Chin-Wei Huang, David Krueger, Alexandre Lacoste, and Aaron Courville. 2018. Neural autoregressive flows. In Int\u2019l Conf. on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v80\/huang18d\/huang18d.pdf"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2594291.2594303"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007665907178"},{"key":"e_1_2_1_17_1","volume-title":"Int\u2019l Conf. on Learning Representations (ICLR). arxiv:1312","author":"Diederik","unstructured":"Diederik P. Kingma and Max Welling. 2014. Auto-encoding variational Bayes. In Int\u2019l Conf. on Learning Representations (ICLR). arxiv:1312.6114."},{"key":"e_1_2_1_18_1","volume-title":"A Note on Importance Sampling Using Standardized Weights. Department of Statistics","author":"Kong Augustine","unstructured":"Augustine Kong. 1992. A Note on Importance Sampling Using Standardized Weights. Department of Statistics, University of Chicago. https:\/\/d3qi0qp55mx5f5.cloudfront.net\/stat\/docs\/tech-rpts\/tr348.pdf"},{"key":"e_1_2_1_19_1","volume-title":"Int\u2019l Conf. on Artificial Intelligence and Statistics (AISTATS). arxiv:1610","author":"Le Tuan Anh","year":"2017","unstructured":"Tuan Anh Le, At\u0131l\u0131m G\u00fcne\u015f Baydin, and Frank Wood. 2017. Inference compilation and universal probabilistic programming. In Int\u2019l Conf. on Artificial Intelligence and Statistics (AISTATS). arxiv:1610.09900."},{"key":"e_1_2_1_20_1","volume-title":"Conf. on Uncertainty in Artificial Intelligence (UAI). arxiv:1805","author":"Le Tuan Anh","year":"2019","unstructured":"Tuan Anh Le, Adam R. Kosiorek, N. Siddharth, Yee Whye Teh, and Frank Wood. 2019. Revisiting reweighted wake-sleep for models with stochastic control flow. In Conf. on Uncertainty in Artificial Intelligence (UAI). arxiv:1805.10469."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3371084"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3371087"},{"key":"e_1_2_1_23_1","volume-title":"Effective Deep Amortized Inference. School of Computer Science","author":"Li Jianlin","unstructured":"Jianlin Li, Leni Aniva, Pengyuan Shi, and Yizhou Zhang. 2022. Synthesizing Guide Programs for Sound, Effective Deep Amortized Inference. School of Computer Science, University of Waterloo."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72019-3_16"},{"key":"e_1_2_1_25_1","volume-title":"Foundations of Statistical Natural Language Processing","author":"Manning Christopher","unstructured":"Christopher Manning and Hinrich Sch\u00fctze. 1999. Foundations of Statistical Natural Language Processing. MIT Press."},{"key":"e_1_2_1_26_1","volume-title":"Conf. on Neural Information Processing Systems (NIPS). arxiv:1307","author":"Mansinghka Vikash K.","unstructured":"Vikash K. Mansinghka, Tejas D. Kulkarni, Yura N. Perov, and Joshua B. Tenenbaum. 2013. Approximate Bayesian image interpretation using generative probabilistic graphics programs. In Conf. on Neural Information Processing Systems (NIPS). arxiv:1307.0060."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3192366.3192409"},{"key":"e_1_2_1_28_1","volume-title":"Conf. on Uncertainty in Artificial Intelligence (UAI). arxiv:1302","author":"Meek Christopher","year":"1995","unstructured":"Christopher Meek. 1995. Strong completeness and faithfulness in Bayesian networks. In Conf. on Uncertainty in Artificial Intelligence (UAI). arxiv:1302.4973."},{"key":"e_1_2_1_29_1","volume-title":"Int\u2019l Conf. on Machine Learning (ICML). arxiv:1602","author":"Paige Brooks","year":"2016","unstructured":"Brooks Paige and Frank Wood. 2016. Inference networks for sequential Monte Carlo in graphical models. In Int\u2019l Conf. on Machine Learning (ICML). arxiv:1602.06701."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","unstructured":"Judea Pearl. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann. isbn:978-0-08-051489-5 https:\/\/doi.org\/10.1016\/B978-0-08-051489-5.50001-1 10.1016\/B978-0-08-051489-5.50001-1","DOI":"10.1016\/B978-0-08-051489-5.50001-1"},{"key":"e_1_2_1_31_1","volume-title":"Int\u2019l Conf. on Machine Learning (ICML). arxiv:1505","author":"Rezende Danilo","year":"2015","unstructured":"Danilo Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. In Int\u2019l Conf. on Machine Learning (ICML). arxiv:1505.05770."},{"key":"e_1_2_1_32_1","volume-title":"Goodman","author":"Ritchie Daniel","year":"2016","unstructured":"Daniel Ritchie, Paul Horsfall, and Noah D. Goodman. 2016. arxiv:1610.05735."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290350"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.2202\/1544-6115.1175"},{"key":"e_1_2_1_35_1","volume-title":"Conf. on Neural Information Processing Systems (NIPS). arxiv:1706","author":"Siddharth N.","unstructured":"N. Siddharth, Brooks Paige, Jan-Willem van de Meent, Alban Desmaison, Noah D. Goodman, Pushmeet Kohli, Frank Wood, and Philip Torr. 2017. Learning disentangled representations with semi-supervised deep generative models. In Conf. on Neural Information Processing Systems (NIPS). arxiv:1706.00400."},{"key":"e_1_2_1_36_1","volume-title":"Conf. on Neural Information Processing Systems (NIPS). https:\/\/proceedings.nips.cc\/paper\/2013\/file\/7f53f8c6c730af6aeb52e66eb74d8507-Paper.pdf","author":"Stuhlm\u00fcller Andreas","unstructured":"Andreas Stuhlm\u00fcller, Jessica Taylor, and Noah D. Goodman. 2013. Learning stochastic inverses. In Conf. on Neural Information Processing Systems (NIPS). https:\/\/proceedings.nips.cc\/paper\/2013\/file\/7f53f8c6c730af6aeb52e66eb74d8507-Paper.pdf"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","unstructured":"Marcin Szymczak and Joost-Pieter Katoen. 2019. Weakest preexpectation semantics for Bayesian inference. In Int\u2019l School on Engineering Trustworthy Software Systems (SETSS). https:\/\/doi.org\/10.1007\/978-3-030-55089-9_3 10.1007\/978-3-030-55089-9_3","DOI":"10.1007\/978-3-030-55089-9_3"},{"key":"e_1_2_1_38_1","volume-title":"Conf. on Neural Information Processing Systems (NeurIPS). arxiv:1811","author":"Tran Dustin","year":"2091","unstructured":"Dustin Tran, Matthew D. Hoffman, Dave Moore, Christopher Suter, Srinivas Vasudevan, Alexey Radul, Matthew Johnson, and Rif A. Saurous. 2018. Simple, distributed, and accelerated probabilistic programming. In Conf. on Neural Information Processing Systems (NeurIPS). arxiv:1811.02091."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1112\/plms"},{"key":"e_1_2_1_40_1","unstructured":"Jan-Willem van de Meent Brooks Paige Hongseok Yang and Frank Wood. 2021. An Introduction to Probabilistic Programming. arxiv:1809.10756."},{"key":"e_1_2_1_41_1","volume-title":"Conf. on Uncertainty in Artificial Intelligence (UAI). arxiv:1304","author":"Verma Thomas","year":"1988","unstructured":"Thomas Verma and Judea Pearl. 1988. Causal networks: Semantics and expressiveness. In Conf. on Uncertainty in Artificial Intelligence (UAI). arxiv:1304.2379."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3453483.3454077"},{"key":"e_1_2_1_43_1","unstructured":"Stefan Webb Jonathan P. Chen Martin Jankowiak and Noah Goodman. 2019. Improving automated variational inference with normalizing flows. In 6 ICML Workshop on Automated Machine Learning. https:\/\/www.automl.org\/wp-content\/uploads\/2019\/06\/automlws2019_Paper23.pdf"},{"key":"e_1_2_1_44_1","volume-title":"Conf. on Neural Information Processing Systems (NIPS). arxiv:1712","author":"Webb Stefan","year":"2018","unstructured":"Stefan Webb, Adam Goli\u0144ski, Robert Zinkov, N. Siddharth, Tom Rainforth, Yee Whye Teh, and Frank Wood. 2018. Faithful inversion of generative models for effective amortized inference. In Conf. on Neural Information Processing Systems (NIPS). arxiv:1712.00287."},{"key":"e_1_2_1_45_1","volume-title":"Int\u2019l Conf. on Artificial Intelligence and Statistics (AISTATS). http:\/\/proceedings.mlr.press\/v108\/weilbach20a\/weilbach20a.pdf","author":"Weilbach Christian","year":"2020","unstructured":"Christian Weilbach, Boyan Beronov, William Harvey, and Frank Wood. 2020. Structured conditional continuous normalizing flows for efficient amortized inference in graphical models. In Int\u2019l Conf. on Artificial Intelligence and Statistics (AISTATS). http:\/\/proceedings.mlr.press\/v108\/weilbach20a\/weilbach20a.pdf"},{"key":"e_1_2_1_46_1","volume-title":"Int\u2019l Conf. on Artificial Intelligence and Statistics (AISTATS). arxiv:1507","author":"Wood Frank D.","unstructured":"Frank D. Wood, Jan-Willem van de Meent, and Vikash Mansinghka. 2014. A new approach to probabilistic programming inference. In Int\u2019l Conf. on Artificial Intelligence and Statistics (AISTATS). arxiv:1507.00996."},{"key":"e_1_2_1_47_1","volume-title":"Int\u2019l Conf. on Learning Representations (ICLR). arxiv:1611","author":"Zhang Chiyuan","year":"2017","unstructured":"Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2017. Understanding deep learning requires rethinking generalization. In Int\u2019l Conf. on Learning Representations (ICLR). arxiv:1611.03530."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2889774"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3571243","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3571243","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:08:22Z","timestamp":1750183702000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3571243"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,9]]},"references-count":48,"journal-issue":{"issue":"POPL","published-print":{"date-parts":[[2023,1,9]]}},"alternative-id":["10.1145\/3571243"],"URL":"https:\/\/doi.org\/10.1145\/3571243","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,9]]},"assertion":[{"value":"2023-01-11","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}