{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:17:46Z","timestamp":1760059066406,"version":"build-2065373602"},"reference-count":58,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T00:00:00Z","timestamp":1747699200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Institute of Cybersecurity (INCIBE)","award":["IAFER-Cib (C074\/23)"],"award-info":[{"award-number":["IAFER-Cib (C074\/23)"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Deep learning models have an intrinsic privacy issue as they memorize parts of their training data, creating a privacy leakage. Membership inference attacks (MIAs) exploit this to obtain confidential information about the data used for training, aiming to steal information. They can be repurposed as a measurement of data integrity by inferring whether the data were used to train a machine learning model. While state-of-the-art attacks achieve significant privacy leakage, their requirements render them infeasible, hindering their use as practical tools to assess the magnitude of the privacy risk. Moreover, the most appropriate evaluation metric of MIA, the true positive rate at a low false positive rate, lacks interpretability. We claim that the incorporation of few-shot learning techniques into the MIA field and a suitable qualitative and quantitative privacy evaluation measure should resolve these issues. In this context, our proposal is twofold. We propose a few-shot learning-based MIA, termed the FeS-MIA model, which eases the evaluation of the privacy breach of a deep learning model by significantly reducing the number of resources required for this purpose. Furthermore, we propose an interpretable quantitative and qualitative measure of privacy, referred to as the Log-MIA measure. Jointly, these proposals provide new tools to assess privacy leakages and to ease the evaluation of the training data integrity of deep learning models, i.e., to analyze the privacy breach of a deep learning model. Experiments carried out with MIA over image classification and language modeling tasks, and a comparison to the state of the art, show that our proposals excel in identifying privacy leakages in a deep learning model with little extra information.<\/jats:p>","DOI":"10.3390\/make7020043","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T08:41:12Z","timestamp":1747730472000},"page":"43","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Membership Inference Attacks Fueled by Few-Shot Learning to Detect Privacy Leakage and Address Data Integrity"],"prefix":"10.3390","volume":"7","author":[{"given":"Daniel","family":"Jim\u00e9nez-L\u00f3pez","sequence":"first","affiliation":[{"name":"Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, 18071 Granada, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7172-9059","authenticated-orcid":false,"given":"Nuria","family":"Rodr\u00edguez-Barroso","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, 18071 Granada, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"M. Victoria","family":"Luz\u00f3n","sequence":"additional","affiliation":[{"name":"Department of Software Engineering, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, 18071 Granada, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1260-9775","authenticated-orcid":false,"given":"Javier","family":"Del Ser","sequence":"additional","affiliation":[{"name":"TECNALIA, Basque Research & Technology Alliance (BRTA), 20730 Azpeitia, Spain"},{"name":"Department of Mathematics, University of the Basque Country (UPV\/EHU), 48013 Bilbao, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7283-312X","authenticated-orcid":false,"given":"Francisco","family":"Herrera","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, 18071 Granada, Spain"},{"name":"ADIA Lab, AI Maryah Island, Abu Dhabi P.O. Box 111999, United Arab Emirates"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.inffus.2022.09.011","article-title":"Survey on federated learning threats: Concepts, taxonomy on attacks and defences, experimental study and challenges","volume":"90","author":"Herrera","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Long, T., Gao, Q., Xu, L., and Zhou, Z. (2022). A survey on adversarial attacks in computer vision: Taxonomy, visualization and future directions. Comput. Secur., 121.","DOI":"10.1016\/j.cose.2022.102847"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ganju, K., Wang, Q., Yang, W., Gunter, C.A., and Borisov, N. (2018, January 15\u201319). Property Inference Attacks on Fully Connected Neural Networks Using Permutation Invariant Representations. Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, Toronto, ON, Canada.","DOI":"10.1145\/3243734.3243834"},{"key":"ref_4","unstructured":"Salem, A., Bhattacharya, A., Backes, M., Fritz, M., and Zhang, Y. (2020, January 12\u201314). Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning. Proceedings of the 29th USENIX Security Symposium (USENIX Security 20), Berkeley, CA, USA."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wu, D., Qi, S., Qi, Y., Li, Q., Cai, B., Guo, Q., and Cheng, J. (2023). Understanding and defending against White-box membership inference attack in deep learning. Knowl.-Based Syst., 259.","DOI":"10.1016\/j.knosys.2022.110014"},{"key":"ref_6","unstructured":"Manzonelli, N., Zhang, W., and Vadhan, S. (2024). Membership Inference Attacks and Privacy in Topic Modeling. arXiv."},{"key":"ref_7","unstructured":"European Commission (2019). High-level expert group on artificial intelligence. Ethics Guidelines for Trustworthy AI, European Union."},{"key":"ref_8","unstructured":"Li, M., Ye, Z., Li, Y., Song, A., Zhang, G., and Liu, F. (2025). Membership Inference Attack Should Move On to Distributional Statistics for Distilled Generative Models. arXiv."},{"key":"ref_9","unstructured":"Zhu, G., Li, D., Gu, H., Yao, Y., Fan, L., and Han, Y. (2024). FedMIA: An Effective Membership Inference Attack Exploiting \u201cAll for One\u201d Principle in Federated Learning. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1109\/TDSC.2022.3141391","article-title":"Securely Outsourcing Neural Network Inference to the Cloud with Lightweight Techniques","volume":"20","author":"Liu","year":"2023","journal-title":"IEEE Trans. Dependable Secur. Comput."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ruan, W., Xu, M., Fang, W., Wang, L., Wang, L., and Han, W. (2023, January 21\u201325). Private, Efficient, and Accurate: Protecting Models Trained by Multi-party Learning with Differential Privacy. Proceedings of the 2023 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP46215.2023.10179422"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dealcala, D., Mancera, G., Morales, A., Fierrez, J., Tolosana, R., and Ortega-Garcia, J. (2024, January 17\u201321). A Comprehensive Analysis of Factors Impacting Membership Inference. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Seattle, WA, USA.","DOI":"10.1109\/CVPRW63382.2024.00362"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Landau, O., Cohen, A., Gordon, S., and Nissim, N. (2020). Mind your privacy: Privacy leakage through BCI applications using machine learning methods. Knowl.-Based Syst., 198.","DOI":"10.1016\/j.knosys.2020.105932"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., and Tram\u00e8r, F. (2022, January 23\u201325). Membership Inference Attacks From First Principles. Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP46214.2022.9833649"},{"key":"ref_15","unstructured":"Ho, G., Sharma, A., Javed, M., Paxson, V., and Wagner, D. (2017, January 16\u201318). Detecting Credential Spearphishing in Enterprise Settings. Proceedings of the 26th USENIX Security Symposium (USENIX Security 17), Vancouver, BC, Canada."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kantchelian, A., Tschantz, M.C., Afroz, S., Miller, B., Shankar, V., Bachwani, R., Joseph, A.D., and Tygar, J.D. (2015, January 16). Better Malware Ground Truth: Techniques for Weighting Anti-Virus Vendor Labels. Proceedings of the 8th ACM Workshop on Artificial Intelligence and Security. Association for Computing Machinery (ACM), Denver, CO, USA.","DOI":"10.1145\/2808769.2808780"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lazarevic, A., Ert\u00f6z, L., Kumar, V., Ozgur, A., and Srivastava, J. (2003, January 1\u20133). A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection. Proceedings of the SIAM International Conference on Data Mining (SDM), San Francisco, CA, USA.","DOI":"10.1137\/1.9781611972733.3"},{"key":"ref_18","first-page":"15479","article-title":"Differential Privacy has disparate impact on model accuracy","volume":"32","author":"Bagdasaryan","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_19","unstructured":"Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., and Nabeshima, N. (2020). The Pile: An 800GB Dataset of Diverse Text for Language Modelling. arXiv."},{"key":"ref_20","first-page":"1877","article-title":"Language Models are Few-Shot Learners","volume":"Volume 33","author":"Larochelle","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_21","unstructured":"Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. (2022). Hierarchical text-conditional image generation with clip latents. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shokri, R., Stronati, M., Song, C., and Shmatikov, V. (2017, January 22\u201326). Membership Inference Attacks against machine learning models. Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA.","DOI":"10.1109\/SP.2017.41"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. (2018, January 9\u201312). Privacy risk in Machine Learning: Analyzing the connection to overfitting. Proceedings of the 2018 IEEE 31st Computer Security Foundations Symposium (CSF), Oxford, UK.","DOI":"10.1109\/CSF.2018.00027"},{"key":"ref_24","first-page":"5558","article-title":"White-box vs Black-box: Bayes Optimal Strategies for Membership Inference","volume":"Volume 97","author":"Chaudhuri","year":"2019","journal-title":"Proceedings of the 36th International Conference on Machine Learning"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"348","DOI":"10.2478\/popets-2021-0031","article-title":"Revisiting Membership Inference Under Realistic Assumptions","volume":"2021","author":"Jayaraman","year":"2020","journal-title":"Priv. Enhancing Technol."},{"key":"ref_26","unstructured":"Song, L., and Mittal, P. (2021, January 11\u201313). Systematic Evaluation of Privacy Risks of Machine Learning Models. Proceedings of the 30th USENIX Security Symposium (USENIX Security 21), Vancouver, BC, Canada."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Long, Y., Wang, L., Bu, D., Bindschaedler, V., Wang, X., Tang, H., Gunter, C.A., and Chen, K. (2020, January 7\u201311). A Pragmatic Approach to Membership Inferences on Machine Learning Models. Proceedings of the 5th IEEE European Symposium on Security and Privacy, Euro S and P, Genoa, Italy.","DOI":"10.1109\/EuroSP48549.2020.00040"},{"key":"ref_28","unstructured":"Watson, L., Guo, C., Cormode, G., and Sablayrolles, A. (2022). On the Importance of Difficulty Calibration in Membership Inference Attacks. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ye, J., Maddi, A., Murakonda, S.K., Bindschaedler, V., and Shokri, R. (2022, January 7\u201311). Enhanced Membership Inference Attacks against Machine Learning Models. Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, Los Angeles, CA, USA.","DOI":"10.1145\/3548606.3560675"},{"key":"ref_30","unstructured":"Zarifzadeh, S., Liu, P., and Shokri, R. (2024). Low-Cost High-Power Membership Inference Attacks. Int. Conf. Mach. Learn. (ICML), 2403."},{"key":"ref_31","unstructured":"Bertran, M., Tang, S., Kearns, M., Morgenstern, J., Roth, A., and Wu, Z.S. (2024, January 10\u201315). Scalable membership inference attacks via quantile regression. Proceedings of the 37th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada. NIPS \u201923."},{"key":"ref_32","unstructured":"Merity, S., Xiong, C., Bradbury, J., and Socher, R. (2017). Pointer Sentinel Mixture Models. arXiv."},{"key":"ref_33","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Rezaei, S., and Liu, X. (2021, January 19\u201325). On the Difficulty of Membership Inference Attacks. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00780"},{"key":"ref_35","unstructured":"Dhillon, G.S., Chaudhari, P., Ravichandran, A., and Soatto, S. (2020). A Baseline for Few-Shot Image Classification. arXiv."},{"key":"ref_36","first-page":"1","article-title":"Generalizing from a few examples: A survey on few-shot learning","volume":"53","author":"Wang","year":"2020","journal-title":"ACM Comput. Surv."},{"key":"ref_37","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Delta-encoder: An effective sample synthesis method for few-shot object recognition. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Hariharan, B., and Girshick, R. (2017, January 22\u201329). Low-Shot Visual Recognition by Shrinking and Hallucinating Features. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.328"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ye, H.J., Hu, H., Zhan, D.C., and Sha, F. (2020, January 14\u201319). Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00883"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Liu, J., Song, L., and Qin, Y. (2020, January 23\u201328). Prototype Rectification for Few-Shot Learning. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.","DOI":"10.1007\/978-3-030-58452-8_43"},{"key":"ref_41","unstructured":"Ziko, I.M., Dolz, J., Granger, \u00c9., and Ayed, I.B. (2020). Laplacian Regularized Few-Shot Learning. arXiv."},{"key":"ref_42","first-page":"2445","article-title":"Information Maximization for Few-Shot Learning","volume":"Volume 33","author":"Larochelle","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Hu, H., Salcic, Z., Sun, L., Dobbie, G., Yu, P.S., and Zhang, X. (2022). Membership Inference Attacks on Machine Learning: A Survey. ACM Comput. Surv., 54.","DOI":"10.1145\/3523273"},{"key":"ref_44","unstructured":"Tang, J., Korolova, A., Bai, X., Wang, X., and Wang, X. (2017). Privacy Loss in Apple\u2019s Implementation of Differential Privacy on MacOS 10.12. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1145\/3287287","article-title":"Understanding database reconstruction attacks on public data","volume":"62","author":"Garfinkel","year":"2019","journal-title":"Commun. ACM"},{"key":"ref_46","first-page":"104995","article-title":"Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable","volume":"Volume 37","author":"Globerson","year":"2024","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_47","unstructured":"Panchendrarajan, R., and Bhoi, S. (2021, January 13\u201315). Dataset reconstruction attack against language models. Proceedings of the CEUR Workshop, Online."},{"key":"ref_48","unstructured":"Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., and Erlingsson, U. (2021, January 11\u201313). Extracting training data from large language models. Proceedings of the 30th USENIX Security Symposium (USENIX Security 21), Vancouver, BC, Canada."},{"key":"ref_49","unstructured":"Wang, Y., Chao, W.L., Weinberger, K.Q., and Van Der Maaten, L. (2019). Simpleshot: Revisiting nearest-neighbor classification for few-shot learning. arXiv."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Tram\u00e8r, F., Shokri, R., San Joaquin, A., Le, H., Jagielski, M., Hong, S., and Carlini, N. (2022, January 7\u201311). Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets. Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, Los Angeles, CA, USA.","DOI":"10.1145\/3548606.3560554"},{"key":"ref_51","unstructured":"Carlini, N., Tram\u00e8r, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T.B., Song, D.X., and Erlingsson, \u00da. (2020, January 12\u201314). Extracting Training Data from Large Language Models. Proceedings of the USENIX Security Symposium, Boston, MA, USA."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_53","unstructured":"Tan, M., and Le, Q. (2019, January 9\u201315). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., and Komodakis, N. (2016). Wide Residual Networks. arXiv.","DOI":"10.5244\/C.30.87"},{"key":"ref_55","unstructured":"Metsis, V., Androutsopoulos, I., and Paliouras, G. (2006, January 27\u201328). Spam filtering with naive bayes-which naive bayes?. Proceedings of the Conference on Email and Anti-Spam, Mountain View, CA, USA."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_57","unstructured":"Krizhevsky, A., and Hinton, G. (2009). Learning Multiple Layers of Features From Tiny Images, University of Toronto. Technical report."},{"key":"ref_58","unstructured":"Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J.W., and Kreps, S. (2019). Release strategies and the social impacts of language models. arXiv."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/43\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:35:47Z","timestamp":1760031347000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/43"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,20]]},"references-count":58,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["make7020043"],"URL":"https:\/\/doi.org\/10.3390\/make7020043","relation":{},"ISSN":["2504-4990"],"issn-type":[{"type":"electronic","value":"2504-4990"}],"subject":[],"published":{"date-parts":[[2025,5,20]]}}}