{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,25]],"date-time":"2025-03-25T14:51:24Z","timestamp":1742914284604,"version":"3.40.3"},"publisher-location":"Cham","reference-count":58,"publisher":"Springer International Publishing","isbn-type":[{"type":"print","value":"9783030994280"},{"type":"electronic","value":"9783030994297"}],"license":[{"start":{"date-parts":[[2022,1,1]],"date-time":"2022-01-01T00:00:00Z","timestamp":1640995200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T00:00:00Z","timestamp":1648512000000},"content-version":"vor","delay-in-days":87,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Probabilistic programming aims to open the power of Bayesian reasoning to software developers and scientists, but identification of problems during inference and debugging are left entirely to the developers and typically require significant statistical expertise. A common class of problems when writing probabilistic programs is the lack of convergence of the probabilistic programs to their posterior distributions.<\/jats:p><jats:p>We present SixthSense, a novel approach for predicting probabilistic program convergence ahead of run and its application to debugging convergence problems in probabilistic programs. SixthSense\u2019s training algorithm learns a classifier that can predict whether a previously unseen probabilistic program will converge. It encodes the syntax of a probabilistic program as<jats:italic>motifs<\/jats:italic>\u2013 fragments of the syntactic program paths. The decisions of the classifier are interpretable and can be used to suggest the program features that contributed significantly to program convergence or non-convergence. We also present an algorithm for augmenting a set of training probabilistic programs that uses guided mutation.<\/jats:p><jats:p>We evaluated SixthSense on a broad range of widely used probabilistic programs. Our results show that SixthSense features are effective in predicting convergence of programs for given inference algorithms. SixthSense obtained Accuracy of over 78% for predicting convergence, substantially above the state-of-the-art techniques for predicting program properties Code2Vec and Code2Seq. We show the ability of SixthSense to guide the debugging of convergence problems, which pinpoints the causes of non-convergence significantly better by Stan\u2019s built-in warnings.<\/jats:p>","DOI":"10.1007\/978-3-030-99429-7_7","type":"book-chapter","created":{"date-parts":[[2022,3,28]],"date-time":"2022-03-28T20:02:48Z","timestamp":1648497768000},"page":"123-144","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["SixthSense: Debugging Convergence Problems in Probabilistic Programs via Program Representation Learning"],"prefix":"10.1007","author":[{"given":"Saikat","family":"Dutta","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zixin","family":"Huang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sasa","family":"Misailovic","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,3,29]]},"reference":[{"key":"7_CR1","unstructured":"Nearpy (2011), https:\/\/github.com\/pixelogik\/NearPy"},{"key":"7_CR2","unstructured":"Prior choice recommendations in stan (2011), https:\/\/github.com\/stan-dev\/stan\/wiki\/Prior-Choice-Recommendations"},{"key":"7_CR3","unstructured":"Allamanis, M., Peng, H., Sutton, C.: A convolutional attention network for extreme summarization of source code. In: International Conference on Machine Learning. pp. 2091\u20132100 (2016)"},{"key":"7_CR4","doi-asserted-by":"crossref","unstructured":"Alon, U., Brody, S., Levy, O., Yahav, E.: code2seq: Generating sequences from structured representations of code. In: International Conference on Learning Representations (2019), https:\/\/openreview.net\/forum?id=H1gKYo09tX","DOI":"10.1145\/3290353"},{"key":"7_CR5","doi-asserted-by":"crossref","unstructured":"Alon, U., Zilberstein, M., Levy, O., Yahav, E.: code2vec: Learning distributed representations of code. Proceedings of the ACM on Programming Languages 3(POPL), \u00a040 (2019)","DOI":"10.1145\/3290353"},{"key":"7_CR6","doi-asserted-by":"crossref","unstructured":"Andoni, A., Indyk, P.: Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Communications of the ACM 51(1), \u00a0117 (2008)","DOI":"10.1145\/1327452.1327494"},{"key":"7_CR7","unstructured":"Balunovic, M., Bielik, P., Vechev, M.: Learning to solve smt formulas. In: Advances in Neural Information Processing Systems. pp. 10338\u201310349 (2018)"},{"key":"7_CR8","doi-asserted-by":"crossref","unstructured":"Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: Proceedings of the international conference on Knowledge discovery and data mining (KDD). ACM (2001)","DOI":"10.1145\/502512.502546"},{"key":"7_CR9","doi-asserted-by":"crossref","unstructured":"Carpenter, B., Gelman, A., Hoffman, M., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M.A., Guo, J., Li, P., Riddell, A.: Stan: A probabilistic programming language. JSTATSOFT 20(2) (2016)","DOI":"10.18637\/jss.v076.i01"},{"key":"7_CR10","doi-asserted-by":"crossref","unstructured":"Claret, G., Rajamani, S.K., Nori, A.V., Gordon, A.D., Borgstr\u00f6m, J.: Bayesian inference using data flow analysis. In: FSE (2013)","DOI":"10.1145\/2491411.2491423"},{"key":"7_CR11","doi-asserted-by":"crossref","unstructured":"Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501 (2018)","DOI":"10.1109\/CVPR.2019.00020"},{"key":"7_CR12","doi-asserted-by":"crossref","unstructured":"Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: Proceedings of the twentieth annual symposium on Computational geometry. pp. 253\u2013262. ACM (2004)","DOI":"10.1145\/997817.997857"},{"key":"7_CR13","unstructured":"Deng, B., Yan, J., Lin, D.: Peephole: Predicting network performance before training. arXiv preprint arXiv:1712.03351 (2017)"},{"key":"7_CR14","doi-asserted-by":"crossref","unstructured":"Dutta, S., Arunachalam, A., Misailovic, S.: To seed or not to seed? an empirical analysis of usage of seeds for testing in machine learning projects. In: ICST (2022)","DOI":"10.1109\/ICST53961.2022.00026"},{"key":"7_CR15","doi-asserted-by":"crossref","unstructured":"Dutta, S., Legunsen, O., Huang, Z., Misailovic, S.: Testing probabilistic programming systems. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. pp. 574\u2013586. ACM (2018)","DOI":"10.1145\/3236024.3236057"},{"key":"7_CR16","doi-asserted-by":"crossref","unstructured":"Dutta, S., Selvam, J., Jain, A., Misailovic, S.: Tera: Optimizing stochastic regression tests in machine learning projects. In: ISSTA (2021)","DOI":"10.1145\/3460319.3464844"},{"key":"7_CR17","doi-asserted-by":"crossref","unstructured":"Dutta, S., Shi, A., Choudhary, R., Zhang, Z., Jain, A., Misailovic, S.: Detecting flaky tests in probabilistic and machine learning applications. In: ISSTA (2020)","DOI":"10.1145\/3395363.3397366"},{"key":"7_CR18","doi-asserted-by":"crossref","unstructured":"Dutta, S., Shi, A., Misailovic, S.: Flex: fixing flaky tests in machine learning projects by updating assertion bounds. In: FSE (2021)","DOI":"10.1145\/3468264.3468615"},{"key":"7_CR19","doi-asserted-by":"crossref","unstructured":"Dutta, S., Zhang, W., Huang, Z., Misailovic, S.: Storm: Program reduction for testing and debugging probabilistic programming systems. In: FSE (2019)","DOI":"10.1145\/3338906.3338972"},{"key":"7_CR20","unstructured":"Dutta, S., Joshi, G., Ghosh, S., Dube, P., Nagpurkar, P.: Slow and stale gradients can win the race: Error-runtime trade-offs in distributed sgd. arXiv preprint arXiv:1803.01113 (2018)"},{"key":"7_CR21","doi-asserted-by":"crossref","unstructured":"Fawcett, T.: An introduction to roc analysis. Pattern recognition letters 27(8), 861\u2013874 (2006)","DOI":"10.1016\/j.patrec.2005.10.010"},{"key":"7_CR22","unstructured":"Flaxman, S., Mishra, S., Gandy, A., Unwin, H.J.T., Mellan, T.A., Coupland, H., Whittaker, C., Zhu, H., Berah, T., Eaton, J.W., et\u00a0al.: Estimating the effects of non-pharmaceutical interventions on covid-19 in europe. Nature pp.\u00a01\u20135 (2020)"},{"key":"7_CR23","unstructured":"Gelman, A.: Stan being used to study and fight coronavirus (2020), https:\/\/discourse.mc-stan.org\/t\/stan-being-used-to-study-and-fight-coronavirus\/14296, Stan Forums"},{"key":"7_CR24","doi-asserted-by":"crossref","unstructured":"Gelman, A., Lee, D., Guo, J.: Stan a probabilistic programming language for bayesian inference and optimization. Journal of Educational and Behavioral Statistics (2015)","DOI":"10.3102\/1076998615606113"},{"key":"7_CR25","doi-asserted-by":"crossref","unstructured":"Gelman, A., Stern, H.S., Carlin, J.B., Dunson, D.B., Vehtari, A., Rubin, D.B.: Bayesian data analysis. Chapman and Hall\/CRC (2013)","DOI":"10.1201\/b16018"},{"key":"7_CR26","unstructured":"Goodman, N., Mansinghka, V., Roy, D.M., Bonawitz, K., Tenenbaum, J.B.: Church: a language for generative models. arXiv preprint arXiv:1206.3255 (2012)"},{"key":"7_CR27","unstructured":"Goodman, N.D., Stuhlm\u00fcller, A.: The design and implementation of probabilistic programming languages (2014)"},{"key":"7_CR28","unstructured":"Hoffman, M.D., Gelman, A.: The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. Journal of Machine Learning Research 15(1), 1593\u20131623 (2014)"},{"key":"7_CR29","doi-asserted-by":"crossref","unstructured":"Huang, Z., Dutta, S., Misailovic, S.: Aqua: Automated quantized inference for probabilistic programs. In: International Symposium on Automated Technology for Verification and Analysis. pp. 229\u2013246. Springer (2021)","DOI":"10.1007\/978-3-030-88885-5_16"},{"key":"7_CR30","doi-asserted-by":"crossref","unstructured":"Istrate, R., Scheidegger, F., Mariani, G., Nikolopoulos, D., Bekas, C., Malossi, A.C.I.: Tapas: Train-less accuracy predictor for architecture search. arXiv preprint arXiv:1806.00250 (2018)","DOI":"10.1609\/aaai.v33i01.33013927"},{"key":"7_CR31","doi-asserted-by":"crossref","unstructured":"Iyer, S., Konstas, I., Cheung, A., Zettlemoyer, L.: Summarizing source code using a neural attention model. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2073\u20132083 (2016)","DOI":"10.18653\/v1\/P16-1195"},{"key":"7_CR32","doi-asserted-by":"crossref","unstructured":"Khalil, E.B., Le\u00a0Bodic, P., Song, L., Nemhauser, G., Dilkina, B.: Learning to branch in mixed integer programming. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)","DOI":"10.1609\/aaai.v30i1.10080"},{"key":"7_CR33","unstructured":"Inference case studies in knitr (2019), https:\/\/github.com\/betanalpha\/knitr_case_studies"},{"key":"7_CR34","doi-asserted-by":"crossref","unstructured":"Leyton-Brown, K., Hoos, H.H., Hutter, F., Xu, L.: Understanding the empirical hardness of np-complete problems. Communications of the ACM 57(5), 98\u2013107 (2014)","DOI":"10.1145\/2594413.2594424"},{"key":"7_CR35","doi-asserted-by":"crossref","unstructured":"Long, F., Rinard, M.: Automatic patch generation by learning correct code. In: ACM SIGPLAN Notices. vol. 51, pp. 298\u2013312. ACM (2016).","DOI":"10.1145\/2914770.2837617"},{"key":"7_CR36","unstructured":"Mansinghka, V., Selsam, D., Perov, Y.: Venture: a higher-order probabilistic programming platform with programmable inference. arXiv preprint 1404.0099 (2014)"},{"key":"7_CR37","unstructured":"Mendis, C., Renda, A., Amarasinghe, S., Carbin, M.: Ithemal: Accurate, portable and fast basic block throughput estimation using deep neural networks. In: ICML (2019)"},{"key":"7_CR38","unstructured":"Minka, T., Winn, J., Guiver, J., Webster, S., Zaykov, Y., Yangel, B., Spengler, A., Bronskill, J.: Infer.NET 2.5 (2013), microsoft Research Cambridge. http:\/\/research.microsoft.com\/infernet"},{"key":"7_CR39","doi-asserted-by":"crossref","unstructured":"Nandi, C., Grossman, D., Sampson, A., Mytkowicz, T., McKinley, K.S.: Debugging probabilistic programs. In: MAPL (2017)","DOI":"10.1145\/3088525.3088564"},{"key":"7_CR40","doi-asserted-by":"crossref","unstructured":"Neal, R.M.: An improved acceptance procedure for the hybrid monte carlo algorithm. Journal of Computational Physics 111(1), 194\u2013203 (1994)","DOI":"10.1006\/jcph.1994.1054"},{"key":"7_CR41","unstructured":"Northcutt, C.G., Wu, T., Chuang, I.L.: Learning with confident examples: Rank pruning for robust classification with noisy labels. In: Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence. UAI\u201917, AUAI Press (2017), http:\/\/auai.org\/uai2017\/proceedings\/papers\/35.pdf"},{"key":"7_CR42","unstructured":"Obermeyer, F.: Deep probabilistic programming with pyro (2020), https:\/\/www.broadinstitute.org\/talks\/deep-probabilistic-programming-pyro, models, Inference, and Algorithms"},{"key":"7_CR43","doi-asserted-by":"crossref","unstructured":"Pu, Y., Narasimhan, K., Solar-Lezama, A., Barzilay, R.: sk_p: a neural program corrector for moocs. In: Companion Proceedings of the 2016 OOPSLA. pp. 39\u201340. ACM (2016)","DOI":"10.1145\/2984043.2989222"},{"key":"7_CR44","unstructured":"Modeling censored time-to-event data using pyro (2019), https:\/\/eng.uber.com\/modeling-censored-time-to-event-data-using-pyro\/"},{"key":"7_CR45","unstructured":"Pyro (2018), http:\/\/pyro.ai"},{"key":"7_CR46","unstructured":"Raiffa, H., Schlaifer, R.: Applied statistical decision theory (1961)"},{"key":"7_CR47","doi-asserted-by":"crossref","unstructured":"Raychev, V., Vechev, M., Krause, A.: Predicting program properties from big code. In: ACM SIGPLAN Notices. vol.\u00a050, pp. 111\u2013124. ACM (2015)","DOI":"10.1145\/2775051.2677009"},{"key":"7_CR48","unstructured":"Robert, C., Casella, G.: Monte Carlo statistical methods. Springer Science & Business Media (2013)"},{"key":"7_CR49","doi-asserted-by":"crossref","unstructured":"Sakia, R.: The box-cox transformation technique: a review. Journal of the Royal Statistical Society: Series D (The Statistician) 41(2), 169\u2013178 (1992)","DOI":"10.2307\/2348250"},{"key":"7_CR50","unstructured":"Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: Icdar. vol.\u00a03 (2003)"},{"key":"7_CR51","unstructured":"Stan. using target += syntax (2016), https:\/\/stackoverflow.com\/questions\/40289457\/stan-using-target-syntax"},{"key":"7_CR52","unstructured":"Stan Example Models (2018), https:\/\/github.com\/stan-dev\/example-models"},{"key":"7_CR53","doi-asserted-by":"crossref","unstructured":"Taylor, L., Nitschke, G.: Improving deep learning using generic data augmentation. arXiv preprint arXiv:1708.06020 (2017)","DOI":"10.1109\/SSCI.2018.8628742"},{"key":"7_CR54","unstructured":"Tehrani, N.K., Arora, N.S., Noursi, D., Tingley, M., Torabi, N., Lippert, E.: Bean machine: A declarative probabilistic programming language for efficient programmable inference. In: PGM (2020)"},{"key":"7_CR55","unstructured":"Tran, D., Kucukelbir, A., Dieng, A.B., Rudolph, M., Liang, D., Blei, D.M.: Edward: A library for probabilistic modeling, inference, and criticism. arXiv (2016)"},{"key":"7_CR56","unstructured":"Tree interpreter package (2020), https:\/\/github.com\/andosa\/treeinterpreter"},{"key":"7_CR57","doi-asserted-by":"crossref","unstructured":"Wang, K., Su, Z.: Learning blended, precise semantic program embeddings. ArXiv, vol. abs\/1907.02136 (2019)","DOI":"10.1145\/3385412.3385999"},{"key":"7_CR58","unstructured":"Wood, F., van\u00a0de Meent, J.W., Mansinghka, V.: A new approach to probabilistic programming inference. In: AISTATS (2014)"}],"container-title":["Lecture Notes in Computer Science","Fundamental Approaches to Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-030-99429-7_7","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,30]],"date-time":"2023-01-30T14:04:14Z","timestamp":1675087454000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-030-99429-7_7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"ISBN":["9783030994280","9783030994297"],"references-count":58,"URL":"https:\/\/doi.org\/10.1007\/978-3-030-99429-7_7","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"type":"print","value":"0302-9743"},{"type":"electronic","value":"1611-3349"}],"subject":[],"published":{"date-parts":[[2022]]},"assertion":[{"value":"29 March 2022","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"FASE","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Fundamental Approaches to Software Engineering","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Munich","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Germany","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2022","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"4 April 2022","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"5 April 2022","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"25","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"fase2022","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/etaps.org\/2022\/fase","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Double-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"EasyChair","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"61","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"17","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"28% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"7","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"Yes","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}