{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T21:50:33Z","timestamp":1774475433228,"version":"3.50.1"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,4,1]],"date-time":"2021-04-01T00:00:00Z","timestamp":1617235200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,13]],"date-time":"2021-04-13T00:00:00Z","timestamp":1618272000000},"content-version":"vor","delay-in-days":12,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"National Science Foundation","award":["1801751"],"award-info":[{"award-number":["1801751"]}]},{"DOI":"10.13039\/100006754","name":"Army Research Laboratory","doi-asserted-by":"publisher","award":["W911NF-13-2-0045"],"award-info":[{"award-number":["W911NF-13-2-0045"]}],"id":[{"id":"10.13039\/100006754","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2021,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Research showed that deep learning models are vulnerable to membership inference attacks, which aim to determine if an example is in the training set of the model. We propose a new framework to defend against this sort of attack. Our key insight is that if we retrain the original classifier with a new dataset that is independent of the original training set while their elements are sampled from the same distribution, the retrained classifier will leak no information that cannot be inferred from the distribution about the original training set. Our framework consists of three phases. First, we transferred the original classifier to a Joint Energy-based Model (JEM) to exploit the model\u2019s implicit generative power. Then, we sampled from the JEM to create a new dataset. Finally, we used the new dataset to retrain or fine-tune the original classifier. We empirically studied different transfer learning schemes for the JEM and fine-tuning\/retraining strategies for the classifier against shadow-model attacks. Our evaluation shows that our framework can suppress the attacker\u2019s membership advantage to a negligible level while keeping the classifier\u2019s accuracy acceptable. We compared it with other state-of-the-art defenses considering adaptive attackers and showed our defense is effective even under the worst-case scenario. Besides, we also found that combining other defenses with our framework often achieves better robustness. Our code will be made available at <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/ChenJiyu\/meminf-defense.git\">https:\/\/github.com\/ChenJiyu\/meminf-defense.git<\/jats:ext-link>.<\/jats:p>","DOI":"10.1007\/s10994-021-05951-6","type":"journal-article","created":{"date-parts":[[2021,4,13]],"date-time":"2021-04-13T19:08:51Z","timestamp":1618340931000},"page":"651-674","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Protect privacy of deep classification networks by exploiting their generative power"],"prefix":"10.1007","volume":"110","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0144-6376","authenticated-orcid":false,"given":"Jiyu","family":"Chen","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiwen","family":"Guo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qianjun","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hao","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,4,13]]},"reference":[{"key":"5951_CR1","doi-asserted-by":"crossref","unstructured":"Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016). Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, (pp. 308\u2013318).","DOI":"10.1145\/2976749.2978318"},{"issue":"1","key":"5951_CR2","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1207\/s15516709cog0901_7","volume":"9","author":"DH Ackley","year":"1985","unstructured":"Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for boltzmann machines. Cognitive science, 9(1), 147\u2013169.","journal-title":"Cognitive science"},{"key":"5951_CR3","first-page":"1069","volume":"12","author":"K Chaudhuri","year":"2011","unstructured":"Chaudhuri, K., Monteleoni, C., & Sarwate, A. D. (2011). Differentially private empirical risk minimization. Journal of Machine Learning Research, 12, 1069\u20131109.","journal-title":"Journal of Machine Learning Research"},{"key":"5951_CR4","doi-asserted-by":"crossref","unstructured":"Chen, J., Jordan, M. I., & Wainwright, M. J. (2020). Hopskipjumpattack: A query-efficient decision-based attack. In 2020 IEEE symposium on security and privacy (sp) (pp. 1277\u20131294). IEEE","DOI":"10.1109\/SP40000.2020.00045"},{"key":"5951_CR5","unstructured":"Choo, C. A. C., Tramer, F., Carlini, N., & Papernot, N. (2020). Label-only membership inference attacks. arXiv:200714321."},{"key":"5951_CR6","unstructured":"Du, Y., & Mordatch, I. (2019). Implicit generation and generalization in energy-based models. arXiv: 190308689."},{"key":"5951_CR7","doi-asserted-by":"crossref","unstructured":"Dwork, C. (2008). Differential privacy: A survey of results. In International conference on theory and applications of models of computation (pp. 1\u201319). Springer","DOI":"10.1007\/978-3-540-79228-4_1"},{"key":"5951_CR8","doi-asserted-by":"crossref","unstructured":"Fredrikson, M., Jha, S., & Ristenpart, T. (2015). Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security (pp 1322\u20131333).","DOI":"10.1145\/2810103.2813677"},{"key":"5951_CR9","doi-asserted-by":"crossref","unstructured":"Ganju, K., Wang, Q., Yang, W., Gunter, C. A., & Borisov, N. (2018). Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security (pp. 619\u2013633).","DOI":"10.1145\/3243734.3243834"},{"key":"5951_CR10","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems (pp. 2672\u20132680)."},{"key":"5951_CR11","unstructured":"Grathwohl, W., Wang, K. C., Jacobsen, J. H., Duvenaud, D., Norouzi, M., & Swersky, K. (2019). Your classifier is secretly an energy based model and you should treat it like one. arXiv:191203263."},{"key":"5951_CR12","doi-asserted-by":"publisher","first-page":"133","DOI":"10.2478\/popets-2019-0008","volume":"1","author":"J Hayes","year":"2019","unstructured":"Hayes, J., Melis, L., Danezis, G., & De Cristofaro, E. (2019). Logan: Membership inference attacks against generative models. Proceedings on Privacy Enhancing Technologies, 1, 133\u2013152.","journal-title":"Proceedings on Privacy Enhancing Technologies"},{"key":"5951_CR13","unstructured":"Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in neural information processing systems (pp. 6626\u20136637)."},{"key":"5951_CR14","doi-asserted-by":"publisher","first-page":"232","DOI":"10.2478\/popets-2019-0067","volume":"4","author":"B Hilprecht","year":"2019","unstructured":"Hilprecht, B., & H\u00e4rterich, M. (2019). Monte carlo and reconstruction membership inference attacks against generative models. Proceedings on Privacy Enhancing Technologies, 4, 232\u2013249.","journal-title":"Proceedings on Privacy Enhancing Technologies"},{"issue":"4","key":"5951_CR15","doi-asserted-by":"publisher","first-page":"725","DOI":"10.1207\/s15516709cog0000_76","volume":"30","author":"G Hinton","year":"2006","unstructured":"Hinton, G., Osindero, S., Welling, M., & Teh, Y. W. (2006). Unsupervised discovery of nonlinear structure using contrastive backpropagation. Cognitive Science, 30(4), 725\u2013731.","journal-title":"Cognitive Science"},{"key":"5951_CR16","unstructured":"Jain, P., Kulkarni, V., Thakurta, A., & Williams, O. (2015). To drop or not to drop: Robustness, consistency and differential privacy properties of dropout. arXiv:150302031."},{"key":"5951_CR17","doi-asserted-by":"crossref","unstructured":"Jia, J., Salem, A., Backes, M., Zhang, Y., & Gong, N. Z. (2019). Memguard: Defending against black-box membership inference attacks via adversarial examples. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, (pp. 259\u2013274).","DOI":"10.1145\/3319535.3363201"},{"key":"5951_CR18","unstructured":"Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv:13126114."},{"key":"5951_CR19","unstructured":"Krizhevsky, A., & Hinton, G. et\u00a0al. (2009). Learning multiple layers of features from tiny images. Technical report, University of Toronto."},{"key":"5951_CR20","unstructured":"LeCun, Y., & Huang, F. J. (2005). Loss functions for discriminative training of energy-based models. In AIStats, Citeseer (Vol.\u00a06, p\u00a034)."},{"key":"5951_CR21","first-page":"2579","volume":"9","author":"LVD Maaten","year":"2008","unstructured":"Maaten, L. V. D., & Hinton, G. (2008). Visualizing data using t-sne. Journal of Machine Learning Research, 9, 2579\u20132605.","journal-title":"Journal of Machine Learning Research"},{"key":"5951_CR22","doi-asserted-by":"crossref","unstructured":"Melis, L., Song ,C,. De\u00a0Cristofaro, E., & Shmatikov, V. (2019). Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE symposium on security and privacy (SP) (pp. 691\u2013706). IEEE","DOI":"10.1109\/SP.2019.00029"},{"key":"5951_CR23","doi-asserted-by":"crossref","unstructured":"Nasr, M., Shokri, R., & Houmansadr, A. (2018). Machine learning with membership privacy using adversarial regularization. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security (pp. 634\u2013646).","DOI":"10.1145\/3243734.3243855"},{"key":"5951_CR24","doi-asserted-by":"crossref","unstructured":"Nasr, M., Shokri, R., & Houmansadr, A. (2019). Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP) (pp. 739\u2013753). IEEE","DOI":"10.1109\/SP.2019.00065"},{"key":"5951_CR25","unstructured":"Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., & Ng, A. Y. (2011). Reading digits in natural images with unsupervised feature learning.\u00a0In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011."},{"key":"5951_CR26","doi-asserted-by":"crossref","unstructured":"Salem, A., Zhang, Y., Humbert, M., Berrang, P., Fritz, M., & Backes, M. (2018). Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv: 180601246.","DOI":"10.14722\/ndss.2019.23119"},{"key":"5951_CR27","unstructured":"Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016). Improved techniques for training gans. In Advances in neural information processing systems (pp. 2234\u20132242)."},{"key":"5951_CR28","doi-asserted-by":"crossref","unstructured":"Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017). Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP) (pp. 3\u201318). IEEE","DOI":"10.1109\/SP.2017.41"},{"key":"5951_CR29","unstructured":"Shokri, R., Strobel, M., & Zick, Y. (2019). Privacy risks of explaining machine learning models. arXiv:190700164."},{"key":"5951_CR30","unstructured":"Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv:14091556."},{"issue":"1","key":"5951_CR31","first-page":"1929","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929\u20131958.","journal-title":"The Journal of Machine Learning Research"},{"key":"5951_CR32","unstructured":"Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013). Intriguing properties of neural networks. arXiv:13126199."},{"key":"5951_CR33","unstructured":"Tram\u00e8r, F., Zhang, F., Juels, A., Reiter, M. K., Ristenpart, T. (2016). Stealing machine learning models via prediction APIS. In 25th USENIX security symposium (USENIX security 16) (pp. 601\u2013618)."},{"key":"5951_CR34","doi-asserted-by":"crossref","unstructured":"Wang, B., & Gong, N. Z. (2018). Stealing hyperparameters in machine learning. In 2018 IEEE symposium on security and privacy (SP) (pp. 36\u201352). IEEE","DOI":"10.1109\/SP.2018.00038"},{"key":"5951_CR35","doi-asserted-by":"crossref","unstructured":"Wang, J., Zhang, J., Bao, W., Zhu, X., Cao, B., & Yu, P. S. (2018). Not just privacy: Improving performance of private deep learning in mobile cloud. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, (pp 2407\u20132416).","DOI":"10.1145\/3219819.3220106"},{"key":"5951_CR36","unstructured":"Welling, M., & Teh, Y. W. (2011). Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11) (pp. 681\u2013688)."},{"key":"5951_CR37","unstructured":"Xiang, L., Ma, H., Zhang, H., Zhang, Y., Ren, J., & Zhang, Q. (2019). Interpretable complex-valued neural networks for privacy protection. arXiv: 190109546."},{"key":"5951_CR38","doi-asserted-by":"crossref","unstructured":"Yeom, S., Giacomelli, I., Fredrikson, M., & Jha, S. (2018). Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF) (pp. 268\u2013282). IEEE","DOI":"10.1109\/CSF.2018.00027"},{"key":"5951_CR39","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., & Komodakis, N. (2016). Wide residual networks. arXiv:160507146.","DOI":"10.5244\/C.30.87"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-05951-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-021-05951-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-05951-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,12]],"date-time":"2021-05-12T15:14:30Z","timestamp":1620832470000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-021-05951-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4]]},"references-count":39,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,4]]}},"alternative-id":["5951"],"URL":"https:\/\/doi.org\/10.1007\/s10994-021-05951-6","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4]]},"assertion":[{"value":"20 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 December 2020","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 January 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 April 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}