{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T11:10:28Z","timestamp":1784200228378,"version":"3.55.0"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA2","license":[{"start":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T00:00:00Z","timestamp":1759968000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2025,10,9]]},"abstract":"<jats:p>\n                    Property-based testing validates software against an executable specification by evaluating it on randomly generated inputs. The standard way that PBT users generate test inputs is via\n                    <jats:italic toggle=\"yes\">generators<\/jats:italic>\n                    that describe how to sample test inputs through random choices. To achieve a good distribution over test inputs, users must\n                    <jats:italic toggle=\"yes\">tune<\/jats:italic>\n                    their generators, i.e., decide on the weights of these individual random choices. Unfortunately, it is very difficult to understand how to choose individual generator weights in order to achieve a desired distribution, so today this process is tedious and limits the distributions that can be practically achieved.\n                  <\/jats:p>\n                  <jats:p>\n                    In this paper, we develop techniques for the automatic and offline tuning of generators. Given a generator with undetermined\n                    <jats:italic toggle=\"yes\">symbolic weights<\/jats:italic>\n                    and an\n                    <jats:italic toggle=\"yes\">objective function,<\/jats:italic>\n                    our approach automatically learns values for these weights that optimize for the objective. We describe useful objective functions that allow users to (1) target desired distributions and (2) improve the diversity and validity of their test cases. We have implemented our approach in a novel discrete probabilistic programming system,\n                    <jats:sc>Loaded Dice<\/jats:sc>\n                    , that supports differentiation and parameter learning, and use it as a language for generators. We empirically demonstrate that our approach is effective at optimizing generator distributions according to the specified objective functions. We also perform a thorough evaluation on PBT benchmarks, demonstrating that, when automatically tuned for diversity and validity, the generators exhibit a 3.1\u20137.4\u00d7 speedup in bug finding.\n                  <\/jats:p>","DOI":"10.1145\/3763082","type":"journal-article","created":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T08:49:50Z","timestamp":1759999790000},"page":"894-921","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Tuning Random Generators: Property-Based Testing as Probabilistic Programming"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-0731-5398","authenticated-orcid":false,"given":"Ryan","family":"Tjoa","sequence":"first","affiliation":[{"name":"University of Washington, Seattle, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4753-3974","authenticated-orcid":false,"given":"Poorva","family":"Garg","sequence":"additional","affiliation":[{"name":"University of California, Los Angeles, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9631-1169","authenticated-orcid":false,"given":"Harrison","family":"Goldstein","sequence":"additional","affiliation":[{"name":"University of Maryland, College Park, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2031-1514","authenticated-orcid":false,"given":"Todd","family":"Millstein","sequence":"additional","affiliation":[{"name":"University of California, Los Angeles, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7839-1636","authenticated-orcid":false,"given":"Benjamin C.","family":"Pierce","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, Philadelphia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3434-2503","authenticated-orcid":false,"given":"Guy","family":"Van den Broeck","sequence":"additional","affiliation":[{"name":"University of California, Los Angeles, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,10,9]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.16852354"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/1159789.1159792"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2015.7107466"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.52202\/068431-0758"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0377-0427(00)00422-2"},{"key":"e_1_3_2_7_2","first-page":"4","volume-title":"Proceedings of the 1999 ACM SIGPLAN Workshop on Partial Evaluation and Semantics-Based Program Manipulation, San Antonio, Texas, USA, January 22-23, 1999. Technical report BRICS-NS-99-1","author":"Bawden Alan","year":"1999","unstructured":"Alan Bawden. 1999. Quasiquotation in Lisp. In Proceedings of the 1999 ACM SIGPLAN Workshop on Partial Evaluation and Semantics-Based Program Manipulation, San Antonio, Texas, USA, January 22-23, 1999. Technical report BRICS-NS-99-1, Olivier Danvy (Ed.). University of Aarhus, 4\u201312."},{"key":"e_1_3_2_8_2","unstructured":"Michael Betancourt. 2018. A Conceptual Introduction to Hamiltonian Monte Carlo. arXiv:1701.02434 [stat.ME] https:\/\/arxiv.org\/abs\/1701.02434"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1137\/141000671"},{"key":"e_1_3_2_10_2","first-page":"28:1","article-title":"Pyro: Deep Universal Probabilistic Programming","volume":"20","author":"Bingham Eli","year":"2019","unstructured":"Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul A. Szerlip, Paul Horsfall, and Noah D. Goodman. 2019. Pyro: Deep Universal Probabilistic Programming. J. Mach. Learn. Res. 20 (2019), 28:1\u201328:6. http:\/\/jmlr.org\/papers\/v20\/18-403.html","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2017.1285773"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1017\/9781108755528"},{"key":"e_1_3_2_13_2","unstructured":"James Bradbury Roy Frostig Peter Hawkins Matthew James Johnson Chris Leary Dougal Maclaurin George Necula Adam Paszke Jake VanderPlas Skye Wanderman-Milne and Qiao Zhang. 2018. JAX: composable transformations of Python+NumPy programs. http:\/\/github.com\/jax-ml\/jax"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/1687399.1687403"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.1986.1676819"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.5555\/3625834.3625859"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2007.11.002"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0956796815000143"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/351240.351266"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1017\/S1471068414000076"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3656412"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639581"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3563291"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3654777.3676407"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/SFCS.1978.3"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-87479-9_49"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3428208"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-30936-1_9"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-47147-7_4"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2016.37"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729694"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3009837.3009868"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3158133"},{"key":"e_1_3_2_34_2","unstructured":"Leonidas Lampropoulos and Benjamin C. Pierce. 2018. QuickChick: Property-Based Testing in Coq. Electronic textbook. http:\/\/www.cis.upenn.edu\/~bcpierce\/sf"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571198"},{"key":"e_1_3_2_36_2","unstructured":"Ziyang Li Jiani Huang and Mayur Naik. 2023. Scallop: A Language for Neurosymbolic Programming. arXiv:2304.04812 [cs.PL] https:\/\/arxiv.org\/abs\/2304.04812"},{"key":"e_1_3_2_37_2","unstructured":"Li-yao Xia. 2018. A quick tour of generic-random. https:\/\/hackage.haskell.org\/package\/generic-random-1.5.0.0\/docs\/Generic-Random.html"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3092703.3092711"},{"key":"e_1_3_2_39_2","unstructured":"Robin Manhaeve Sebastijan Duman\u010di\u0107 Angelika Kimmig Thomas Demeester and Luc De Raedt. 2018. DeepProbLog: Neural Probabilistic Logic Programming. arXiv:1805.10872 [cs.AI] https:\/\/arxiv.org\/abs\/1805.10872"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3412932.3412943"},{"key":"e_1_3_2_41_2","volume-title":"Probabilistic Machine Learning: An introduction","author":"Murphy Kevin P.","year":"2022","unstructured":"Kevin P. Murphy. 2022. Probabilistic Machine Learning: An introduction. MIT Press. http:\/\/probml.github.io\/book1"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3519939.3523707"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-22102-1_22"},{"key":"e_1_3_2_44_2","unstructured":"Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury Gregory Chanan Trevor Killeen Zeming Lin Natalia Gimelshein Luca Antiga Alban Desmaison Andreas K\u00f6pf Edward Yang Zach DeVito Martin Raison Alykhan Tejani Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai and Soumith Chintala. 2019. PyTorch: An Imperative Style High-Performance Deep Learning Library. arXiv:1912.01703 [cs.LG] https:\/\/arxiv.org\/abs\/1912.01703"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380399"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-4145-2"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3607860"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549139"},{"key":"e_1_3_2_50_2","unstructured":"Donald Stewart Koen Claessen Nick Smallbone and Simon Marlow. 2024. Test.QuickCheck \u2014 Hackage.Haskell.Org."},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.16868014"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992696"},{"key":"e_1_3_2_53_2","unstructured":"Jingyi Xu Zilu Zhang Tal Friedman Yitao Liang and Guy Van den Broeck. 2018. A Semantic Loss Function for Deep Learning with Symbolic Knowledge. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research Vol. 80) Jennifer Dy and Andreas Krause (Eds.). PMLR 5502\u20135511. https:\/\/proceedings.mlr.press\/v80\/xu18h.html"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3763082","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3763082","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T10:11:27Z","timestamp":1784196687000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3763082"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,9]]},"references-count":52,"journal-issue":{"issue":"OOPSLA2","published-print":{"date-parts":[[2025,10,9]]}},"alternative-id":["10.1145\/3763082"],"URL":"https:\/\/doi.org\/10.1145\/3763082","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,9]]},"assertion":[{"value":"2025-03-25","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-12","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}