{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T02:39:59Z","timestamp":1775097599441,"version":"3.50.1"},"reference-count":43,"publisher":"Privacy Enhancing Technologies Symposium Advisory Board","issue":"2","license":[{"start":{"date-parts":[[2022,3,3]],"date-time":"2022-03-03T00:00:00Z","timestamp":1646265600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/3.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We present <jats:monospace>d3p<\/jats:monospace>, a software package designed to help fielding runtime efficient widely-applicable Bayesian inference under differential privacy guarantees. <jats:monospace>d3p <\/jats:monospace>achieves general applicability to a wide range of probabilistic modelling problems by implementing the differentially private variational inference algorithm, allowing users to fit any parametric probabilistic model with a differentiable density function. <jats:monospace>d3p <\/jats:monospace>adopts the probabilistic programming paradigm as a powerful way for the user to flexibly define such models. We demonstrate the use of our software on a hierarchical logistic regression example, showing the expressiveness of the modelling approach as well as the ease of running the parameter inference. We also perform an empirical evaluation of the runtime of the private inference on a complex model and find a ~10 fold speed-up compared to an implementation using TensorFlow Privacy.<\/jats:p>","DOI":"10.2478\/popets-2022-0052","type":"journal-article","created":{"date-parts":[[2022,3,5]],"date-time":"2022-03-05T04:25:16Z","timestamp":1646454316000},"page":"407-425","source":"Crossref","is-referenced-by-count":4,"title":["d3p - A Python Package for Differentially-Private Probabilistic Programming"],"prefix":"10.56553","volume":"2022","author":[{"given":"Lukas","family":"Prediger","sequence":"first","affiliation":[{"name":"Aalto University , Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Niki","family":"Loppi","sequence":"additional","affiliation":[{"name":"NVIDIA AI Technology Center , Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samuel","family":"Kaski","sequence":"additional","affiliation":[{"name":"Aalto University , Finland & University of Manchester , UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Antti","family":"Honkela","sequence":"additional","affiliation":[{"name":"University of Helsinki , Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"35752","published-online":{"date-parts":[[2022,3,3]]},"reference":[{"key":"2022060207220572303_j_popets-2022-0052_ref_001","unstructured":"[1] Mart\u00edn Abadi et al. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. URL https:\/\/www.tensorflow.org\/. Software available from tensor-flow.org."},{"key":"2022060207220572303_j_popets-2022-0052_ref_002","doi-asserted-by":"crossref","unstructured":"[2] Martin Abadi et al. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308\u2013318, 2016.10.1145\/2976749.2978318","DOI":"10.1145\/2976749.2978318"},{"key":"2022060207220572303_j_popets-2022-0052_ref_003","unstructured":"[3] Eli Bingham et al. Pyro: Deep Universal Probabilistic Programming. arXiv preprint arXiv:1810.09538, 2018."},{"key":"2022060207220572303_j_popets-2022-0052_ref_004","unstructured":"[4] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, and Skye Wanderman-Milne. JAX: composable transformations of Python+NumPy programs. https:\/\/github.com\/google\/jax, 2018."},{"key":"2022060207220572303_j_popets-2022-0052_ref_005","unstructured":"[5] Cl\u00e9ment L Canonne, Gautam Kamath, and Thomas Steinke. The discrete gaussian for differential privacy. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 15676\u201315688. Curran Associates, Inc., 2020."},{"key":"2022060207220572303_j_popets-2022-0052_ref_006","doi-asserted-by":"crossref","unstructured":"[6] Bob Carpenter et al. Stan: a probabilistic programming language. Journal of Statistical Software, 76(1), 2017.10.18637\/jss.v076.i01","DOI":"10.18637\/jss.v076.i01"},{"key":"2022060207220572303_j_popets-2022-0052_ref_007","unstructured":"[7] Joshua V. Dillon et al. Tensorflow distributions. arXiv preprint arXiv:1711.10604, 2017."},{"key":"2022060207220572303_j_popets-2022-0052_ref_008","doi-asserted-by":"crossref","unstructured":"[8] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3-4):211\u2013407, 2014.10.1561\/0400000042","DOI":"10.1561\/0400000042"},{"key":"2022060207220572303_j_popets-2022-0052_ref_009","doi-asserted-by":"crossref","unstructured":"[9] Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Annual International Conference on the Theory and Applications of Cryptographic Techniques, pages 486\u2013503. Springer, 2006.10.1007\/11761679_29","DOI":"10.1007\/11761679_29"},{"key":"2022060207220572303_j_popets-2022-0052_ref_010","doi-asserted-by":"crossref","unstructured":"[10] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pages 265\u2013284. Springer, 2006.10.1007\/11681878_14","DOI":"10.1007\/11681878_14"},{"key":"2022060207220572303_j_popets-2022-0052_ref_011","unstructured":"[11] \u00dalfar Erlingsson, Ilya Mironov, Ananth Raghunathan, and Shuang Song. That which we call private. arXiv preprint arXiv:1908.03566, 2019."},{"key":"2022060207220572303_j_popets-2022-0052_ref_012","unstructured":"[12] Facebook. Opacus. https:\/\/opacus.ai\/, 2020."},{"key":"2022060207220572303_j_popets-2022-0052_ref_013","doi-asserted-by":"crossref","unstructured":"[13] Horst Feistel. Cryptography and computer privacy. Scientific american, 228(5):15\u201323, 1973.10.1038\/scientificamerican0573-15","DOI":"10.1038\/scientificamerican0573-15"},{"key":"2022060207220572303_j_popets-2022-0052_ref_014","unstructured":"[14] Chris Fonnesbeck, Anand Patil, David Huard, and John Salvatier. PyMC: Bayesian stochastic modelling in python. Astrophysics Source Code Library, 2015."},{"key":"2022060207220572303_j_popets-2022-0052_ref_015","unstructured":"[15] Roy Frostig, Matthew James Johnson, and Chris Leary. Compiling machine learning programs via high-level tracing. Systems for Machine Learning, 2018."},{"key":"2022060207220572303_j_popets-2022-0052_ref_016","doi-asserted-by":"crossref","unstructured":"[16] Simson L. Garfinkel and Philip Leclerc. Randomness concerns when deploying differential privacy. In Proceedings of the 19th Workshop on Privacy in the Electronic Society, WPES\u201920, page 73\u201386, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450380867. 10.1145\/3411497.3420211.10.1145\/3411497.3420211","DOI":"10.1145\/3411497.3420211"},{"key":"2022060207220572303_j_popets-2022-0052_ref_017","doi-asserted-by":"crossref","unstructured":"[17] Charles R. Harris et al. Array programming with NumPy. Nature, 585(7825):357\u2013362, September 2020. 10.1038\/s41586-020-2649-2.10.1038\/s41586-020-2649-2775946132939066","DOI":"10.1038\/s41586-020-2649-2"},{"key":"2022060207220572303_j_popets-2022-0052_ref_018","unstructured":"[18] Joonas J\u00e4lk\u00f6, Onur Dikmen, and Antti Honkela. Differentially private variational inference for non-conjugate models. In Uncertainty in Artificial Intelligence 2017 Proceedings of the 33rd Conference, UAI 2017. The Association for Uncertainty in Artificial Intelligence, 2017."},{"key":"2022060207220572303_j_popets-2022-0052_ref_019","doi-asserted-by":"crossref","unstructured":"[19] Joonas J\u00e4lk\u00f6, Eemil Lagerspetz, Jari Haukka, Sasu Tarkoma, Antti Honkela, and Samuel Kaski. Privacy-preserving data sharing via probabilistic modeling. Patterns, 2(7):100271, 2021. ISSN 2666-3899. 10.1016\/j.patter.2021.100271.10.1016\/j.patter.2021.100271827601534286296","DOI":"10.1016\/j.patter.2021.100271"},{"key":"2022060207220572303_j_popets-2022-0052_ref_020","doi-asserted-by":"crossref","unstructured":"[20] Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola, and Lawrence K. Saul. An introduction to variational methods for graphical models. Machine learning, 37(2):183\u2013233, 1999.10.1023\/A:1007665907178","DOI":"10.1023\/A:1007665907178"},{"key":"2022060207220572303_j_popets-2022-0052_ref_021","unstructured":"[21] Peter Kairouz, Ziyu Liu, and Thomas Steinke. The distributed discrete gaussian mechanism for federated learning with secure aggregation. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 5201\u20135212. PMLR, 18\u201324 Jul 2021."},{"key":"2022060207220572303_j_popets-2022-0052_ref_022","unstructured":"[22] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR 2015), 2015."},{"key":"2022060207220572303_j_popets-2022-0052_ref_023","unstructured":"[23] Diederik P. Kingma and Max Welling. Auto-encoding variational Bayes. In 2nd International Conference on Learning Representations (ICLR 2014), 2014."},{"key":"2022060207220572303_j_popets-2022-0052_ref_024","unstructured":"[24] Antti Koskela, Joonas J\u00e4lk\u00f6, and Antti Honkela. Computing tight differential privacy guarantees using FFT. In International Conference on Artificial Intelligence and Statistics, pages 2560\u20132569. PMLR, 2020."},{"key":"2022060207220572303_j_popets-2022-0052_ref_025","unstructured":"[25] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009."},{"key":"2022060207220572303_j_popets-2022-0052_ref_026","doi-asserted-by":"crossref","unstructured":"[26] Yann LeCun, L\u00e9on Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278\u20132324, 1998.10.1109\/5.726791","DOI":"10.1109\/5.726791"},{"key":"2022060207220572303_j_popets-2022-0052_ref_027","doi-asserted-by":"crossref","unstructured":"[27] Michael Luby and Charles Rackoff. How to construct pseudorandom permutations from pseudorandom functions. SIAM Journal on Computing, 17(2):373\u2013386, 1988.10.1137\/0217022","DOI":"10.1137\/0217022"},{"key":"2022060207220572303_j_popets-2022-0052_ref_028","doi-asserted-by":"crossref","unstructured":"[28] Ilya Mironov, Omkant Pandey, Omer Reingold, and Salil Vadhan. Computational differential privacy. In Annual International Cryptology Conference, pages 126\u2013142. Springer, 2009.10.1007\/978-3-642-03356-8_8","DOI":"10.1007\/978-3-642-03356-8_8"},{"key":"2022060207220572303_j_popets-2022-0052_ref_029","unstructured":"[29] Rory Mitchell, Daniel Stokes, Eibe Frank, and Geoffrey Holmes. Bandwidth-optimal random shuffling for GPUs. arXiv preprint arXiv:2106.06161, abs\/2106.06161, 2021."},{"key":"2022060207220572303_j_popets-2022-0052_ref_030","unstructured":"[30] Adam Paszke et al. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024\u20138035. Curran Associates, Inc., 2019."},{"key":"2022060207220572303_j_popets-2022-0052_ref_031","unstructured":"[31] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825\u20132830, 2011."},{"key":"2022060207220572303_j_popets-2022-0052_ref_032","unstructured":"[32] Du Phan, Neeraj Pradhan, and Martin Jankowiak. Composable effects for flexible and accelerated probabilistic programming in NumPyro. arXiv preprint arXiv:1912.11554, 2019."},{"key":"2022060207220572303_j_popets-2022-0052_ref_033","unstructured":"[33] Carey Radebaugh and Ulfar Erlingsson. Introducing Tensor-Flow privacy: Learning with differential privacy for training data. TensorFlow Blog, https:\/\/blog.tensorflow.org\/2019\/03\/introducing-tensorflow-privacy-learning.html, 2019."},{"key":"2022060207220572303_j_popets-2022-0052_ref_034","doi-asserted-by":"crossref","unstructured":"[34] Shuang Song, Kamalika Chaudhuri, and Anand D. Sarwate. Stochastic gradient descent with differentially private updates. In 2013 IEEE Global Conference on Signal and Information Processing, pages 245\u2013248. IEEE, 2013.10.1109\/GlobalSIP.2013.6736861","DOI":"10.1109\/GlobalSIP.2013.6736861"},{"key":"2022060207220572303_j_popets-2022-0052_ref_035","unstructured":"[35] Daniel Stokes and Rory Mitchell. CUDA-Shuffle: GPU shuffle using bijective functions. https:\/\/github.com\/djns99\/CUDA-Shuffle, 2021."},{"key":"2022060207220572303_j_popets-2022-0052_ref_036","unstructured":"[36] Pranav Subramani, Nicholas Vadivelu, and Gautam Kamath. Enabling fast differentially private SGD via just-in-time compilation and vectorization. arXiv preprint arXiv:2010.09063, 2020."},{"key":"2022060207220572303_j_popets-2022-0052_ref_037","unstructured":"[37] Theano Development Team. Theano: A python framework for fast computation of mathematical expressions. arXiv preprint arXiv:1605.02688, 2016."},{"key":"2022060207220572303_j_popets-2022-0052_ref_038","unstructured":"[38] Michalis Titsias and Miguel L\u00e1zaro-Gredilla. Doubly stochastic variational Bayes for non-conjugate inference. In International conference on machine learning, pages 1971\u20131979, 2014."},{"key":"2022060207220572303_j_popets-2022-0052_ref_039","unstructured":"[39] Dustin Tran et al. Simple, distributed, and accelerated probabilistic programming. In Neural Information Processing Systems, 2018."},{"key":"2022060207220572303_j_popets-2022-0052_ref_040","doi-asserted-by":"crossref","unstructured":"[40] Martin J. Wainwright and Michael Irwin Jordan. Graphical models, exponential families, and variational inference. Now Publishers Inc, 2008.10.1561\/9781601981851","DOI":"10.1561\/9781601981851"},{"key":"2022060207220572303_j_popets-2022-0052_ref_041","unstructured":"[41] Chris Waites. PyVacy. https:\/\/github.com\/ChrisWaites\/pyvacy, 2019."},{"key":"2022060207220572303_j_popets-2022-0052_ref_042","doi-asserted-by":"crossref","unstructured":"[42] George Y. Wong and William M. Mason. The hierarchical logistic regression model for multilevel analysis. Journal of the American Statistical Association, 80(391):513\u2013524, 1985. ISSN 01621459.10.1080\/01621459.1985.10478148","DOI":"10.1080\/01621459.1985.10478148"},{"key":"2022060207220572303_j_popets-2022-0052_ref_043","unstructured":"[43] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017."}],"container-title":["Proceedings on Privacy Enhancing Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.sciendo.com\/pdf\/10.2478\/popets-2022-0052","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,20]],"date-time":"2022-07-20T16:32:06Z","timestamp":1658334726000},"score":1,"resource":{"primary":{"URL":"https:\/\/petsymposium.org\/popets\/2022\/popets-2022-0052.php"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,3]]},"references-count":43,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,3,3]]},"published-print":{"date-parts":[[2022,4,1]]}},"alternative-id":["10.2478\/popets-2022-0052"],"URL":"https:\/\/doi.org\/10.2478\/popets-2022-0052","relation":{},"ISSN":["2299-0984"],"issn-type":[{"value":"2299-0984","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,3]]}}}