{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,4,10]],"date-time":"2024-04-10T06:42:34Z","timestamp":1712731354135},"reference-count":69,"publisher":"Privacy Enhancing Technologies Symposium Advisory Board","issue":"4","license":[{"start":{"date-parts":[[2021,7,23]],"date-time":"2021-07-23T00:00:00Z","timestamp":1626998400000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/3.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Deep learning techniques have achieved remarkable performance in wide-ranging tasks. However, when trained on privacy-sensitive datasets, the model parameters may expose private information in training data. Prior attempts for differentially private training, although offering rigorous privacy guarantees, lead to much lower model performance than the non-private ones. Besides, different runs of the same training algorithm produce models with large performance variance. To address these issues, we propose DPlis\u2013 Differentially Private Learning wIth Smoothing. The core idea of DPlis is to construct a smooth loss function that favors noise-resilient models lying in large flat regions of the loss landscape. We provide theoretical justification for the utility improvements of DPlis. Extensive experiments also demonstrate that DPlis can effectively boost model quality and training stability under a given privacy budget.<\/jats:p>","DOI":"10.2478\/popets-2021-0065","type":"journal-article","created":{"date-parts":[[2021,7,24]],"date-time":"2021-07-24T23:16:56Z","timestamp":1627168616000},"page":"163-183","source":"Crossref","is-referenced-by-count":2,"title":["DPlis: Boosting Utility of Differentially Private Deep Learning via Randomized Smoothing"],"prefix":"10.56553","volume":"2021","author":[{"given":"Wenxiao","family":"Wang","sequence":"first","affiliation":[{"name":"Tsinghua University ."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tianhao","family":"Wang","sequence":"additional","affiliation":[{"name":"Harvard University ."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lun","family":"Wang","sequence":"additional","affiliation":[{"name":"University of California , Berkeley ."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nanqing","family":"Luo","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology ."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pan","family":"Zhou","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology ."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dawn","family":"Song","sequence":"additional","affiliation":[{"name":"University of California , Berkeley ."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ruoxi","family":"Jia","sequence":"additional","affiliation":[{"name":"Virginia Tech ."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"35752","published-online":{"date-parts":[[2021,7,23]]},"reference":[{"key":"2022060521174450984_j_popets-2021-0065_ref_001","doi-asserted-by":"crossref","unstructured":"[1] M. Abadi, A. Chu, I. J. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. In E. R. Weippl, S. Katzenbeisser, C. Kruegel, A. C. Myers, and S. Halevi, editors, Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016, pages 308\u2013318. ACM, 2016.10.1145\/2976749.2978318","DOI":"10.1145\/2976749.2978318"},{"key":"2022060521174450984_j_popets-2021-0065_ref_002","unstructured":"[2] R. Al-Rfou, M. Pickett, J. Snaider, Y.-h. Sung, B. Strope, and R. Kurzweil. Conversational contextual cues: The case of personalization and history for response ranking. arXiv preprint arXiv:1606.00372, 2016."},{"key":"2022060521174450984_j_popets-2021-0065_ref_003","doi-asserted-by":"crossref","unstructured":"[3] G. Ateniese, L. V. Mancini, A. Spognardi, A. Villani, D. Vitali, and G. Felici. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. International Journal of Security and Networks, 10(3):137\u2013150, 2015.","DOI":"10.1504\/IJSN.2015.071829"},{"key":"2022060521174450984_j_popets-2021-0065_ref_004","doi-asserted-by":"crossref","unstructured":"[4] M. Bun and T. Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In Theory of Cryptography Conference, pages 635\u2013658. Springer, 2016.10.1007\/978-3-662-53641-4_24","DOI":"10.1007\/978-3-662-53641-4_24"},{"key":"2022060521174450984_j_popets-2021-0065_ref_005","unstructured":"[5] N. Carlini, C. Liu, \u00da. Erlingsson, J. Kos, and D. Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th {USENIX} Security Symposium ({USENIX} Security 19), pages 267\u2013284, 2019."},{"key":"2022060521174450984_j_popets-2021-0065_ref_006","doi-asserted-by":"crossref","unstructured":"[6] P. Chaudhari, A. Choromanska, S. Soatto, Y. LeCun, C. Baldassi, C. Borgs, J. Chayes, L. Sagun, and R. Zecchina. Entropy-sgd: Biasing gradient descent into wide valleys. Journal of Statistical Mechanics: Theory and Experiment, 2019(12):124018, 2019.","DOI":"10.1088\/1742-5468\/ab39d9"},{"key":"2022060521174450984_j_popets-2021-0065_ref_007","unstructured":"[7] K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. Differentially private empirical risk minimization. Journal of Machine Learning Research, 12(3), 2011."},{"key":"2022060521174450984_j_popets-2021-0065_ref_008","doi-asserted-by":"crossref","unstructured":"[8] D. Chen, N. Yu, Y. Zhang, and M. Fritz. Gan-leaks: A taxonomy of membership inference attacks against gans. arXiv preprint arXiv:1909.03935, 2019.","DOI":"10.1145\/3372297.3417238"},{"key":"2022060521174450984_j_popets-2021-0065_ref_009","doi-asserted-by":"crossref","unstructured":"[9] G. Chiachia, A. X. Falc\u00e3o, N. Pinto, A. Rocha, and D. D. Cox. Learning person-specific representations from faces in the wild. IEEE Trans. Inf. Forensics Secur., 9(12):2089\u20132099, 2014.10.1109\/TIFS.2014.2359543","DOI":"10.1109\/TIFS.2014.2359543"},{"key":"2022060521174450984_j_popets-2021-0065_ref_010","doi-asserted-by":"crossref","unstructured":"[10] J. C. Duchi, P. L. Bartlett, and M. J. Wainwright. Randomized smoothing for (parallel) stochastic optimization. In Proceedings of the 51th IEEE Conference on Decision and Control, CDC 2012, December 10-13, 2012, Maui, HI, USA, pages 5442\u20135444. IEEE, 2012.10.1109\/CDC.2012.6426698","DOI":"10.1109\/CDC.2012.6426698"},{"key":"2022060521174450984_j_popets-2021-0065_ref_011","unstructured":"[11] C. Dwork. Differential privacy. In M. Bugliesi, B. Preneel, V. Sassone, and I. Wegener, editors, Automata, Languages and Programming, 33rd International Colloquium, ICALP 2006, Venice, Italy, July 10-14, 2006, Proceedings, Part II, volume 4052 of Lecture Notes in Computer Science, pages 1\u201312. Springer, 2006."},{"key":"2022060521174450984_j_popets-2021-0065_ref_012","doi-asserted-by":"crossref","unstructured":"[12] C. Dwork and J. Lei. Differential privacy and robust statistics. Proceedings of the Annual ACM Symposium on Theory of Computing, pages 371\u2013380, 2009.10.1145\/1536414.1536466","DOI":"10.1145\/1536414.1536466"},{"key":"2022060521174450984_j_popets-2021-0065_ref_013","doi-asserted-by":"crossref","unstructured":"[13] C. Dwork and A. Roth. The Algorithmic Foundations of Differential Privacy. Foundations and Trends\u00ae in Theoretical Computer Science, 9(3-4):211\u2013407, 2014.10.1561\/0400000042","DOI":"10.1561\/0400000042"},{"key":"2022060521174450984_j_popets-2021-0065_ref_014","doi-asserted-by":"crossref","unstructured":"[14] C. Dwork, A. Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3-4):211\u2013407, 2014.10.1561\/0400000042","DOI":"10.1561\/0400000042"},{"key":"2022060521174450984_j_popets-2021-0065_ref_015","unstructured":"[15] W. N. Francis and H. Kucera. Brown corpus manual. Letters to the Editor, 5(2):7, 1979."},{"key":"2022060521174450984_j_popets-2021-0065_ref_016","doi-asserted-by":"crossref","unstructured":"[16] M. Fredrikson, S. Jha, and T. Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322\u20131333, 2015.10.1145\/2810103.2813677","DOI":"10.1145\/2810103.2813677"},{"key":"2022060521174450984_j_popets-2021-0065_ref_017","unstructured":"[17] M. Fredrikson, E. Lantz, S. Jha, S. Lin, D. Page, and T. Ristenpart. Privacy in pharmacogenetics: An end-toend case study of personalized warfarin dosing. In 23rd {USENIX} Security Symposium ({USENIX} Security 14), pages 17\u201332, 2014."},{"key":"2022060521174450984_j_popets-2021-0065_ref_018","doi-asserted-by":"crossref","unstructured":"[18] K. Ganju, Q. Wang, W. Yang, C. A. Gunter, and N. Borisov. Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pages 619\u2013633, 2018.10.1145\/3243734.3243834","DOI":"10.1145\/3243734.3243834"},{"key":"2022060521174450984_j_popets-2021-0065_ref_019","doi-asserted-by":"crossref","unstructured":"[19] S. Ghadimi and G. Lan. Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM Journal on Optimization, 23(4):2341\u20132368, 2013.10.1137\/120880811","DOI":"10.1137\/120880811"},{"key":"2022060521174450984_j_popets-2021-0065_ref_020","unstructured":"[20] M. Hardt, B. Recht, and Y. Singer. Train faster, generalize better: Stability of stochastic gradient descent. In M. Balcan and K. Q. Weinberger, editors, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pages 1225\u20131234. JMLR.org, 2016."},{"key":"2022060521174450984_j_popets-2021-0065_ref_021","doi-asserted-by":"crossref","unstructured":"[21] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770\u2013778. IEEE Computer Society, 2016.10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"2022060521174450984_j_popets-2021-0065_ref_022","doi-asserted-by":"crossref","unstructured":"[22] J. Hong, H. Wang, Z. Wang, and J. Zhou. Learning model-based privacy protection under budget constraints. 2021.","DOI":"10.1609\/aaai.v35i9.16941"},{"key":"2022060521174450984_j_popets-2021-0065_ref_023","doi-asserted-by":"crossref","unstructured":"[23] R. Iyengar, J. P. Near, D. Song, O. Thakkar, A. Thakurta, and L. Wang. Towards practical differentially private convex optimization. In 2019 IEEE Symposium on Security and Privacy (SP), pages 299\u2013316. IEEE, 2019.10.1109\/SP.2019.00001","DOI":"10.1109\/SP.2019.00001"},{"key":"2022060521174450984_j_popets-2021-0065_ref_024","unstructured":"[24] P. Jain and A. G. Thakurta. (near) dimension independent risk bounds for differentially private learning. In International Conference on Machine Learning, pages 476\u2013484, 2014."},{"key":"2022060521174450984_j_popets-2021-0065_ref_025","doi-asserted-by":"crossref","unstructured":"[25] G. Kerrigan, D. Slack, and J. Tuyls. Differentially private language models benefit from public pre-training. arXiv preprint arXiv:2009.05886, 2020.","DOI":"10.18653\/v1\/2020.privatenlp-1.5"},{"key":"2022060521174450984_j_popets-2021-0065_ref_026","unstructured":"[26] N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836, 2016."},{"key":"2022060521174450984_j_popets-2021-0065_ref_027","unstructured":"[27] D. Kifer, A. Smith, and A. Thakurta. Private convex empirical risk minimization and high-dimensional regression. In Conference on Learning Theory, pages 25\u20131, 2012."},{"key":"2022060521174450984_j_popets-2021-0065_ref_028","unstructured":"[28] A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009."},{"key":"2022060521174450984_j_popets-2021-0065_ref_029","doi-asserted-by":"crossref","unstructured":"[29] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278\u20132324, November 1998.10.1109\/5.726791","DOI":"10.1109\/5.726791"},{"key":"2022060521174450984_j_popets-2021-0065_ref_030","doi-asserted-by":"crossref","unstructured":"[30] J. Lee and D. Kifer. Concentrated differentially private gradient descent with adaptive per-iteration privacy budget. In Y. Guo and F. Farooq, editors, Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, pages 1656\u20131665. ACM, 2018.10.1145\/3219819.3220076","DOI":"10.1145\/3219819.3220076"},{"key":"2022060521174450984_j_popets-2021-0065_ref_031","unstructured":"[31] J. Lee and D. Kifer. Differentially private deep learning with direct feedback alignment. arXiv preprint arXiv:2010.03701, 2020."},{"key":"2022060521174450984_j_popets-2021-0065_ref_032","unstructured":"[32] H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein. Visualizing the loss landscape of neural nets. In S. Bengio, H. M. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montr\u00e9al, Canada, pages 6391\u20136401, 2018."},{"key":"2022060521174450984_j_popets-2021-0065_ref_033","unstructured":"[33] Z. Li and Y. Zhang. Label-leaks: Membership inference attack with label. arXiv preprint arXiv:2007.15528, 2020."},{"key":"2022060521174450984_j_popets-2021-0065_ref_034","doi-asserted-by":"crossref","unstructured":"[34] J. Liu and K. Talwar. Private selection from private candidates. In M. Charikar and E. Cohen, editors, Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, STOC 2019, Phoenix, AZ, USA, June 23-26, 2019, pages 298\u2013309. ACM, 2019.10.1145\/3313276.3316377","DOI":"10.1145\/3313276.3316377"},{"key":"2022060521174450984_j_popets-2021-0065_ref_035","doi-asserted-by":"crossref","unstructured":"[35] J. Liu, C. Zhang, et al. Distributed learning systems with first-order methods. Foundations and Trends\u00ae in Databases, 9(1):1\u2013100, 2020.10.1561\/1900000062","DOI":"10.1561\/1900000062"},{"key":"2022060521174450984_j_popets-2021-0065_ref_036","doi-asserted-by":"crossref","unstructured":"[36] Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.10.1109\/ICCV.2015.425","DOI":"10.1109\/ICCV.2015.425"},{"key":"2022060521174450984_j_popets-2021-0065_ref_037","unstructured":"[37] Y. Long, V. Bindschaedler, L. Wang, D. Bu, X. Wang, H. Tang, C. A. Gunter, and K. Chen. Understanding membership inferences on well-generalized learning models. arXiv preprint arXiv:1802.04889, 2018."},{"key":"2022060521174450984_j_popets-2021-0065_ref_038","unstructured":"[38] H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang. Learning differentially private recurrent language models. arXiv preprint arXiv:1710.06963, 2017."},{"key":"2022060521174450984_j_popets-2021-0065_ref_039","doi-asserted-by":"crossref","unstructured":"[39] I. Mironov. R\u00e9nyi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pages 263\u2013275. IEEE, 2017.10.1109\/CSF.2017.11","DOI":"10.1109\/CSF.2017.11"},{"key":"2022060521174450984_j_popets-2021-0065_ref_040","unstructured":"[40] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Ng. Reading digits in natural images with unsupervised feature learning. NIPS, 01 2011."},{"key":"2022060521174450984_j_popets-2021-0065_ref_041","unstructured":"[41] N. Papernot, M. Abadi, U. Erlingsson, I. Goodfellow, and K. Talwar. Semi-supervised knowledge transfer for deep learning from private training data. arXiv preprint arXiv:1610.05755, 2016."},{"key":"2022060521174450984_j_popets-2021-0065_ref_042","unstructured":"[42] N. Papernot, S. Song, I. Mironov, A. Raghunathan, K. Talwar, and \u00da. Erlingsson. Scalable private learning with pate. arXiv preprint arXiv:1802.08908, 2018."},{"key":"2022060521174450984_j_popets-2021-0065_ref_043","unstructured":"[43] N. Papernot, A. Thakurta, S. Song, S. Chien, and \u00da. Erlingsson. Tempered sigmoid activations for deep learning with differential privacy. CoRR, abs\/2007.14191, 2020."},{"key":"2022060521174450984_j_popets-2021-0065_ref_044","doi-asserted-by":"crossref","unstructured":"[44] N. Pinto, Z. Stone, T. E. Zickler, and D. D. Cox. Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2011, Colorado Springs, CO, USA, 20-25 June, 2011, pages 35\u201342. IEEE Computer Society, 2011.10.1109\/CVPRW.2011.5981788","DOI":"10.1109\/CVPRW.2011.5981788"},{"key":"2022060521174450984_j_popets-2021-0065_ref_045","doi-asserted-by":"crossref","unstructured":"[45] Z. Qin, Y. Yang, T. Yu, I. Khalil, X. Xiao, and K. Ren. Heavy hitter estimation over set-valued data with local differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 192\u2013203, 2016.10.1145\/2976749.2978409","DOI":"10.1145\/2976749.2978409"},{"key":"2022060521174450984_j_popets-2021-0065_ref_046","unstructured":"[46] M. A. Rahman, T. Rahman, R. Lagani\u00e8re, N. Mohammed, and Y. Wang. Membership inference attack against differentially private deep learning model. Trans. Data Priv., 11(1):61\u201379, 2018."},{"key":"2022060521174450984_j_popets-2021-0065_ref_047","doi-asserted-by":"crossref","unstructured":"[47] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211\u2013252, 2015.10.1007\/s11263-015-0816-y","DOI":"10.1007\/s11263-015-0816-y"},{"key":"2022060521174450984_j_popets-2021-0065_ref_048","unstructured":"[48] A. Salem, A. Bhattacharya, M. Backes, M. Fritz, and Y. Zhang. Updates-leak: Data set inference and reconstruction attacks in online learning. In 29th {USENIX} Security Symposium ({USENIX} Security 20), pages 1291\u20131308, 2020."},{"key":"2022060521174450984_j_popets-2021-0065_ref_049","doi-asserted-by":"crossref","unstructured":"[49] A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, and M. Backes. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.01246, 2018.","DOI":"10.14722\/ndss.2019.23119"},{"key":"2022060521174450984_j_popets-2021-0065_ref_050","doi-asserted-by":"crossref","unstructured":"[50] R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pages 3\u201318. IEEE, 2017.10.1109\/SP.2017.41","DOI":"10.1109\/SP.2017.41"},{"key":"2022060521174450984_j_popets-2021-0065_ref_051","unstructured":"[51] M. T. Smith, M. A. \u00c1lvarez, and N. D. Lawrence. Differentially private regression and classification with sparse gaussian processes. CoRR, abs\/1909.09147, 2019."},{"key":"2022060521174450984_j_popets-2021-0065_ref_052","doi-asserted-by":"crossref","unstructured":"[52] S. Song, K. Chaudhuri, and A. D. Sarwate. Stochastic gradient descent with differentially private updates. In 2013 IEEE Global Conference on Signal and Information Processing, pages 245\u2013248. IEEE, 2013.10.1109\/GlobalSIP.2013.6736861","DOI":"10.1109\/GlobalSIP.2013.6736861"},{"key":"2022060521174450984_j_popets-2021-0065_ref_053","unstructured":"[53] K. Talwar, A. Thakurta, and L. Zhang. Private empirical risk minimization beyond the worst case: The effect of the constraint set geometry. arXiv preprint arXiv:1411.5417, 2014."},{"key":"2022060521174450984_j_popets-2021-0065_ref_054","unstructured":"[54] M. Tan and Q. V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 6105\u20136114. PMLR, 2019."},{"key":"2022060521174450984_j_popets-2021-0065_ref_055","unstructured":"[55] O. Thakkar, G. Andrew, and H. B. McMahan. Differentially private learning with adaptive clipping. arXiv preprint arXiv:1905.03871, 2019."},{"key":"2022060521174450984_j_popets-2021-0065_ref_056","unstructured":"[56] A. G. Thakurta and A. Smith. Differentially private feature selection via stability arguments, and the robustness of the lasso. In Conference on Learning Theory, pages 819\u2013850, 2013."},{"key":"2022060521174450984_j_popets-2021-0065_ref_057","unstructured":"[57] B. Wang, Q. Gu, M. Boedihardjo, F. Barekat, and S. J. Osher. DP-LSSGD: A stochastic optimization method to lift the utility in privacy-preserving ERM. CoRR, abs\/1906.12056, 2019."},{"key":"2022060521174450984_j_popets-2021-0065_ref_058","unstructured":"[58] L. Wang, B. Jayaraman, D. Evans, and Q. Gu. Efficient privacy-preserving nonconvex optimization. CoRR, abs\/1910.13659, 2019."},{"key":"2022060521174450984_j_popets-2021-0065_ref_059","unstructured":"[59] T. Wang and F. Kerschbaum. Robust and undetectable white-box watermarks for deep neural networks. arXiv preprint arXiv:1910.14268, 2019."},{"key":"2022060521174450984_j_popets-2021-0065_ref_060","unstructured":"[60] T. Wang, Y. Zhang, and R. Jia. Improving robustness to model inversion attacks via mutual information regularization. arXiv preprint arXiv:2009.05241, 2020."},{"key":"2022060521174450984_j_popets-2021-0065_ref_061","unstructured":"[61] Y.-X. Wang, B. Balle, and S. P. Kasiviswanathan. Sub-sampled r\u00e9nyi differential privacy and analytical moments accountant. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1226\u20131235. PMLR, 2019."},{"key":"2022060521174450984_j_popets-2021-0065_ref_062","unstructured":"[62] M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient langevin dynamics. In L. Getoor and T. Scheffer, editors, Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011, pages 681\u2013688. Omnipress, 2011."},{"key":"2022060521174450984_j_popets-2021-0065_ref_063","unstructured":"[63] W. Wen, Y. Wang, F. Yan, C. Xu, C. Wu, Y. Chen, and H. Li. Smoothout: Smoothing out sharp minima to improve generalization in deep learning. arXiv preprint arXiv:1805.07898, 2018."},{"key":"2022060521174450984_j_popets-2021-0065_ref_064","doi-asserted-by":"crossref","unstructured":"[64] X. Wu, F. Li, A. Kumar, K. Chaudhuri, S. Jha, and J. Naughton. Bolt-on differential privacy for scalable stochastic gradient descent-based analytics. In Proceedings of the 2017 ACM International Conference on Management of Data, pages 1307\u20131322, 2017.10.1145\/3035918.3064047","DOI":"10.1145\/3035918.3064047"},{"key":"2022060521174450984_j_popets-2021-0065_ref_065","unstructured":"[65] Z. Yang, E.-C. Chang, and Z. Liang. Adversarial neural network inversion via auxiliary knowledge alignment. arXiv preprint arXiv:1902.08552, 2019."},{"key":"2022060521174450984_j_popets-2021-0065_ref_066","doi-asserted-by":"crossref","unstructured":"[66] L. Yu, L. Liu, C. Pu, M. E. Gursoy, and S. Truex. Differentially private model publishing for deep learning. In 2019 IEEE Symposium on Security and Privacy, SP 2019, San Francisco, CA, USA, May 19-23, 2019, pages 332\u2013349. IEEE, 2019.10.1109\/SP.2019.00019","DOI":"10.1109\/SP.2019.00019"},{"key":"2022060521174450984_j_popets-2021-0065_ref_067","doi-asserted-by":"crossref","unstructured":"[67] J. Zhang, K. Zheng, W. Mou, and L. Wang. Efficient private erm for smooth objectives. arXiv preprint arXiv:1703.09947, 2017.","DOI":"10.24963\/ijcai.2017\/548"},{"key":"2022060521174450984_j_popets-2021-0065_ref_068","doi-asserted-by":"crossref","unstructured":"[68] Y. Zhang, R. Jia, H. Pei, W. Wang, B. Li, and D. Song. The secret revealer: generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pages 253\u2013261, 2020.10.1109\/CVPR42600.2020.00033","DOI":"10.1109\/CVPR42600.2020.00033"},{"key":"2022060521174450984_j_popets-2021-0065_ref_069","unstructured":"[69] Y. Zhu and Y.-X. Wang. Poission subsampled r\u00e9nyi differential privacy. volume 97 of Proceedings of Machine Learning Research, pages 7634\u20137642, Long Beach, California, USA, 09\u201315 Jun 2019. PMLR."}],"container-title":["Proceedings on Privacy Enhancing Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.sciendo.com\/pdf\/10.2478\/popets-2021-0065","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,5]],"date-time":"2023-01-05T12:02:32Z","timestamp":1672920152000},"score":1,"resource":{"primary":{"URL":"https:\/\/petsymposium.org\/popets\/2021\/popets-2021-0065.php"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,23]]},"references-count":69,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,7,23]]},"published-print":{"date-parts":[[2021,10,1]]}},"alternative-id":["10.2478\/popets-2021-0065"],"URL":"https:\/\/doi.org\/10.2478\/popets-2021-0065","relation":{},"ISSN":["2299-0984"],"issn-type":[{"value":"2299-0984","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,23]]}}}