{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,17]],"date-time":"2025-09-17T03:16:25Z","timestamp":1758078985249,"version":"3.44.0"},"reference-count":26,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:p>Synthetic data generation (SDG) is the process of generating a new synthetic dataset based on the statistical properties of a confidential existing dataset. Differential privacy is the property of a SDG mechanism that establishes how protected individuals whose sensitive data is part of the confidential dataset are, when sharing such data. To ensure a SDG is differentially private, noise is injected into the statistics learned from the dataset. Depending on the amount of noise injected, we witness a trade-off between privacy and utility. Privacy is then measured via a set of privacy metrics that usually establish a lower bound on a few aspects of the privacy-utility trade-off. Therefore, it is not possible to assess privacy based only on one metric. To close this gap, we demonstrate PrivEval, a tool to assist users in evaluating the privacy properties of a synthetic dataset. PrivEval implements several privacy metrics and validates them on both a single user and the overall dataset. Besides, PrivEval checks assumptions behind each metric. Hence, PrivEval is a first step to bridge the gap between privacy experts and the general public to make privacy estimation more transparent.<\/jats:p>","DOI":"10.14778\/3750601.3750649","type":"journal-article","created":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T13:38:05Z","timestamp":1758029885000},"page":"5271-5274","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["PrivEval: A Tool for Interactive Evaluation of Privacy Metrics in Synthetic Data Generation"],"prefix":"10.14778","volume":"18","author":[{"given":"Frederik Marinus","family":"Trudslev","sequence":"first","affiliation":[{"name":"Aalborg University, Aalborg, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matteo","family":"Lissandrini","sequence":"additional","affiliation":[{"name":"University of Verona, Verona, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Juan Manuel","family":"Rodriguez","sequence":"additional","affiliation":[{"name":"Aalborg University, Aalborg, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martin","family":"B\u00f8gsted","sequence":"additional","affiliation":[{"name":"Aalborg University and Aalborg University Hospital, Aalborg, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniele","family":"Dell'Aglio","sequence":"additional","affiliation":[{"name":"Aalborg University, Aalborg, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,9,16]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models. CoRR abs\/2102.08921","author":"Alaa Ahmed M.","year":"2021","unstructured":"Ahmed M. Alaa, Boris van Breugel, Evgeny Saveliev, and Mihaela van der Schaar. 2021. How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models. CoRR abs\/2102.08921 (2021). arXiv:2102.08921"},{"key":"e_1_2_1_2_1","unstructured":"Edward Choi Siddharth Biswal Bradley Malin Jon Duke Walter F. Stewart and Jimeng Sun. 2018. Generating Multi-label Discrete Patient Records using Generative Adversarial Networks. arXiv:1703.06490 [cs.LG]"},{"key":"e_1_2_1_3_1","volume-title":"The Algorithmic Foundations of Differential Privacy","author":"Dwork Cynthia","unstructured":"Cynthia Dwork and Aaron Roth. 2014. The Algorithmic Foundations of Differential Privacy. Vol. 9."},{"key":"e_1_2_1_4_1","volume-title":"Annual Review of Statistics and Its Application","author":"Dwork Cynthia","year":"2017","unstructured":"Cynthia Dwork, Adam Smith, Thomas Steinke, and Jonathan Ullman. 2017. Exposed! A Survey of Attacks on Private Data. Annual Review of Statistics and Its Application (2017)."},{"key":"e_1_2_1_5_1","volume-title":"Patient-centric synthetic data generation, no reason to risk re-identification in biomedical data analysis. npj Digital Medicine 6, 1 (10","author":"Guillaudeux Morgan","year":"2023","unstructured":"Morgan Guillaudeux, Olivia Rousseau, Julien Petot, Zineb Bennis, Charles-Axel Dein, Thomas Goronflot, Nicolas Vince, Sophie Limou, Matilde Karakachoff, Matthieu Wargny, and Pierre-Antoine Gourraud. 2023. Patient-centric synthetic data generation, no reason to risk re-identification in biomedical data analysis. npj Digital Medicine 6, 1 (10 Mar 2023), 37."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2899387"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352099"},{"key":"e_1_2_1_8_1","unstructured":"James Jordon Lukasz Szpruch Florimond Houssiau Mirko Bottarelli Giovanni Cherubin Carsten Maple Samuel N. Cohen and Adrian Weller. 2022. Synthetic Data - what why and how? arXiv:2205.03257 [cs.LG]"},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Anton Danholt Lautrup Tobias Hyrup Arthur Zimek and Peter Schneider-Kamp. 2024. SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Data. arXiv:2404.15821 [cs.LG]","DOI":"10.1007\/s10618-024-01081-4"},{"key":"e_1_2_1_10_1","volume-title":"Systematic Review of Generative Modelling Tools and Utility Metrics for Fully Synthetic Tabular Data. ACM Comput. Surv. 57, 4","author":"Lautrup Anton Danholt","year":"2025","unstructured":"Anton Danholt Lautrup, Tobias Hyrup, Arthur Zimek, and Peter Schneider-Kamp. 2025. Systematic Review of Generative Modelling Tools and Utility Metrics for Fully Synthetic Tabular Data. ACM Comput. Surv. 57, 4 (2025), 90:1\u201390:38."},{"key":"e_1_2_1_11_1","volume-title":"Information Security.","author":"Lee Jaewoo","unstructured":"Jaewoo Lee and Chris Clifton. 2011. How Much Is Enough? Choosing \u03f5 for Differential Privacy. In Information Security. Berlin, Heidelberg, 325\u2013340."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.14778\/2733004.2733059"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.14778\/3231751.3231757"},{"key":"e_1_2_1_14_1","volume-title":"The Synthetic Data Vault. In 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA). 399\u2013410","author":"Patki Neha","year":"2016","unstructured":"Neha Patki, Roy Wedge, and Kalyan Veeramachaneni. 2016. The Synthetic Data Vault. In 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA). 399\u2013410."},{"key":"e_1_2_1_15_1","unstructured":"Bjarne Pfitzner and Bert Arnrich. 2022. DPD-fVAE: Synthetic Data Generation Using Federated Variational Autoencoders With Differentially-Private Decoder. arXiv:2211.11591 [cs.LG]"},{"key":"e_1_2_1_16_1","first-page":"4","article-title":"A Survey of Privacy Attacks in Machine","volume":"56","author":"Rigaki Maria","year":"2024","unstructured":"Maria Rigaki and Sebastian Garcia. 2024. A Survey of Privacy Attacks in Machine Learning. Comput. Surveys 56, 4 (April 2024), 1\u201334. arXiv:2007.07646 [cs].","journal-title":"Learning. Comput. Surveys"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","unstructured":"C. Sakar and Yomi Kastro. 2018. Online Shoppers Purchasing Intention Dataset. UCI Machine Learning Repository. 10.24432\/C5F88Q","DOI":"10.24432\/C5F88Q"},{"key":"e_1_2_1_18_1","volume-title":"Synthetic Data - Anonymisation Groundhog Day. In 31st USENIX Security Symposium. 1451\u20131468","author":"Stadler Theresa","year":"2022","unstructured":"Theresa Stadler, Bristena Oprisanu, and Carmela Troncoso. 2022. Synthetic Data - Anonymisation Groundhog Day. In 31st USENIX Security Symposium. 1451\u20131468."},{"key":"e_1_2_1_19_1","volume-title":"Martin B\u00f8gsted, and Daniele Dell'Aglio.","author":"Trudslev Frederik Marinus","year":"2025","unstructured":"Frederik Marinus Trudslev, Matteo Lissandrini, Juan Manuel Rodriguez, Martin B\u00f8gsted, and Daniele Dell'Aglio. 2025. A Review of Privacy Metrics for Privacy-Preserving Synthetic Data Generation. arXiv:2507.11324 [cs.CR] https:\/\/arxiv.org\/abs\/2507.11324"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512232"},{"key":"e_1_2_1_21_1","unstructured":"Liyang Xie Kaixiang Lin Shu Wang Fei Wang and Jiayu Zhou. 2018. Differentially Private Generative Adversarial Network. arXiv:1802.06739 [cs.LG]"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.12.136"},{"key":"e_1_2_1_23_1","volume-title":"Anonymization Through Data Synthesis Using Generative Adversarial Networks (ADSGAN)","author":"Yoon Jinsung","year":"2020","unstructured":"Jinsung Yoon, Lydia N. Drumright, and Mihaela van der Schaar. 2020. Anonymization Through Data Synthesis Using Generative Adversarial Networks (ADSGAN). IEEE journal of biomedical and health informatics 24, 8 (2020), 2378\u20132388."},{"key":"e_1_2_1_24_1","volume-title":"PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees. In International Conference on Learning Representations.","author":"Yoon Jinsung","unstructured":"Jinsung Yoon, James Jordon, and Mihaela van der Schaar. 2019. PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees. In International Conference on Learning Representations."},{"key":"e_1_2_1_25_1","unstructured":"Hengrui Zhang Jiani Zhang Balasubramaniam Srinivasan Zhengyuan Shen Xiao Qin Christos Faloutsos Huzefa Rangwala and George Karypis. 2024. Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space. In The twelfth International Conference on Learning Representations."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134428"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3750601.3750649","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T13:39:03Z","timestamp":1758029943000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3750601.3750649"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8]]},"references-count":26,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2025,8]]}},"alternative-id":["10.14778\/3750601.3750649"],"URL":"https:\/\/doi.org\/10.14778\/3750601.3750649","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2025,8]]},"assertion":[{"value":"2025-09-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}