{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T03:48:15Z","timestamp":1782445695168,"version":"3.54.5"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2023,7]]},"abstract":"<jats:p>Differential privacy (DP) data synthesizers are increasingly proposed to afford public release of sensitive information, offering theoretical guarantees for privacy (and, in some cases, utility), but limited empirical evidence of utility in practical settings. Utility is typically measured as the error on representative proxy tasks, such as descriptive statistics, multivariate correlations, the accuracy of trained classifiers, or performance over a query workload. The ability for these results to generalize to practitioners' experience has been questioned in a number of settings, including the U.S. Census. In this paper, we propose an evaluation methodology for synthetic data that avoids assumptions about the representativeness of proxy tasks, instead measuring the likelihood that published conclusions would change had the authors used synthetic data, a condition we call epistemic parity. Our methodology consists of reproducing empirical conclusions of peer-reviewed papers on real, publicly available data, then re-running these experiments a second time on DP synthetic data and comparing the results.<\/jats:p>\n          <jats:p>We instantiate our methodology over a benchmark of recent peer-reviewed papers that analyze public datasets in the ICPSR social science repository. We model quantitative claims computationally to automate the experimental workflow, and model qualitative claims by reproducing visualizations and comparing the results manually. We then generate DP synthetic datasets using multiple state-of-the-art mechanisms, and estimate the likelihood that these conclusions will hold. We find that, for reasonable privacy regimes, state-of-the-art DP synthesizers are able to achieve high epistemic parity for several papers in our benchmark. However, some papers, and particularly some specific findings, are difficult to reproduce for any of the synthesizers. Given these results, we advocate for a new class of mechanisms that can reorder the priorities for DP data synthesis: favor stronger guarantees for utility (as measured by epistemic parity) and offer privacy protection with a focus on application-specific threat models and risk-assessment.<\/jats:p>","DOI":"10.14778\/3611479.3611517","type":"journal-article","created":{"date-parts":[[2023,8,25]],"date-time":"2023-08-25T02:08:08Z","timestamp":1692929288000},"page":"3178-3191","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Epistemic Parity: Reproducibility as an Evaluation Metric for Differential Privacy"],"prefix":"10.14778","volume":"16","author":[{"given":"Lucas","family":"Rosenblatt","sequence":"first","affiliation":[{"name":"New York University, New York, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bernease","family":"Herman","sequence":"additional","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anastasia","family":"Holovenko","sequence":"additional","affiliation":[{"name":"Ukrainian Catholic University, Lviv, Ukraine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wonkwon","family":"Lee","sequence":"additional","affiliation":[{"name":"New York University, New York, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joshua","family":"Loftus","sequence":"additional","affiliation":[{"name":"London School of Economics, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elizabeth","family":"McKinnie","sequence":"additional","affiliation":[{"name":"Microsoft, Seattle, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taras","family":"Rumezhak","sequence":"additional","affiliation":[{"name":"Ukrainian Catholic University, Lviv, Ukraine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrii","family":"Stadnik","sequence":"additional","affiliation":[{"name":"Ukrainian Catholic University, Lviv, Ukraine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bill","family":"Howe","sequence":"additional","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julia","family":"Stoyanovich","sequence":"additional","affiliation":[{"name":"New York University, New York, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,8,24]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"1","article-title":"MFE: Towards reproducible meta-feature extraction","volume":"21","author":"Alcoba\u00e7a Edesio","year":"2020","unstructured":"Edesio Alcoba\u00e7a , Felipe Siqueira , Adriano Rivolli , Lu\u00eds P. F. Garcia , Jefferson T. Oliva , and Andr\u00e9 C. P. L. F. de Carvalho . 2020 . MFE: Towards reproducible meta-feature extraction . Journal of Machine Learning Research 21 , 111 (2020), 1 -- 5 . http:\/\/jmlr.org\/papers\/v21\/19-348.html Edesio Alcoba\u00e7a, Felipe Siqueira, Adriano Rivolli, Lu\u00eds P. F. Garcia, Jefferson T. Oliva, and Andr\u00e9 C. P. L. F. de Carvalho. 2020. MFE: Towards reproducible meta-feature extraction. Journal of Machine Learning Research 21, 111 (2020), 1--5. http:\/\/jmlr.org\/papers\/v21\/19-348.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.3390\/ijerph16193705"},{"key":"e_1_2_1_3_1","volume-title":"International Conference on Machine Learning. PMLR, 457--467","author":"Aydore Sergul","year":"2021","unstructured":"Sergul Aydore , William Brown , Michael Kearns , Krishnaram Kenthapadi , Luca Melis , Aaron Roth , and Ankit A Siva . 2021 . Differentially private query release through adaptive projection . In International Conference on Machine Learning. PMLR, 457--467 . Sergul Aydore, William Brown, Michael Kearns, Krishnaram Kenthapadi, Luca Melis, Aaron Roth, and Ankit A Siva. 2021. Differentially private query release through adaptive projection. In International Conference on Machine Learning. PMLR, 457--467."},{"key":"e_1_2_1_4_1","volume-title":"Differential privacy has disparate impact on model accuracy. Advances in neural information processing systems 32","author":"Bagdasaryan Eugene","year":"2019","unstructured":"Eugene Bagdasaryan , Omid Poursaeed , and Vitaly Shmatikov . 2019. Differential privacy has disparate impact on model accuracy. Advances in neural information processing systems 32 ( 2019 ). Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. 2019. Differential privacy has disparate impact on model accuracy. Advances in neural information processing systems 32 (2019)."},{"key":"e_1_2_1_5_1","volume-title":"Nature 533, 7604","author":"Baker Monya","year":"2016","unstructured":"Monya Baker . 2016. 1,500 scientists lift the lid on reproducibility. Nature 533, 7604 ( 2016 ). Monya Baker. 2016. 1,500 scientists lift the lid on reproducibility. Nature 533, 7604 (2016)."},{"key":"e_1_2_1_6_1","volume-title":"Terminologies for reproducible research. arXiv preprint arXiv:1802.03311","author":"Barba Lorena A","year":"2018","unstructured":"Lorena A Barba . 2018. Terminologies for reproducible research. arXiv preprint arXiv:1802.03311 ( 2018 ). Lorena A Barba. 2018. Terminologies for reproducible research. arXiv preprint arXiv:1802.03311 (2018)."},{"key":"e_1_2_1_7_1","first-page":"280","article-title":"Comparative study of differentially private data synthesis methods","volume":"35","author":"McKay Bowen Claire","year":"2020","unstructured":"Claire McKay Bowen and Fang Liu . 2020 . Comparative study of differentially private data synthesis methods . Statist. Sci. 35 , 2 (2020), 280 -- 307 . Claire McKay Bowen and Fang Liu. 2020. Comparative study of differentially private data synthesis methods. Statist. Sci. 35, 2 (2020), 280--307.","journal-title":"Statist. Sci."},{"key":"e_1_2_1_8_1","volume-title":"Differential Perspectives: Epistemic Disconnects Surrounding the US Census Bureau's Use of Differential Privacy. Harvard Data Science Review (Forthcoming)","author":"Boyd Danah","year":"2022","unstructured":"Danah Boyd and Jayshree Sarathy . 2022 . Differential Perspectives: Epistemic Disconnects Surrounding the US Census Bureau's Use of Differential Privacy. Harvard Data Science Review (Forthcoming) (2022). Danah Boyd and Jayshree Sarathy. 2022. Differential Perspectives: Epistemic Disconnects Surrounding the US Census Bureau's Use of Differential Privacy. Harvard Data Science Review (Forthcoming) (2022)."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-53641-4_24"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476249.3476272"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-018-0399-z"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP46214.2022.9833668"},{"key":"e_1_2_1_13_1","volume-title":"International Conference on Language Resources & Evalua-tion:[proceedings]. International Conference on Language Resources and Evaluation","volume":"2018","author":"Cohen K Bretonnel","year":"2018","unstructured":"K Bretonnel Cohen , Jingbo Xia , Pierre Zweigenbaum , Tiffany J Callahan , Orin Hargraves , Foster Goss , Nancy Ide , Aur\u00e9lie N\u00e9v\u00e9ol , Cyril Grouin , and Lawrence E Hunter . 2018 . Three dimensions of reproducibility in natural language processing. In LREC ... International Conference on Language Resources & Evalua-tion:[proceedings]. International Conference on Language Resources and Evaluation , Vol. 2018 . NIH Public Access, 156. K Bretonnel Cohen, Jingbo Xia, Pierre Zweigenbaum, Tiffany J Callahan, Orin Hargraves, Foster Goss, Nancy Ide, Aur\u00e9lie N\u00e9v\u00e9ol, Cyril Grouin, and Lawrence E Hunter. 2018. Three dimensions of reproducibility in natural language processing. In LREC... International Conference on Language Resources & Evalua-tion:[proceedings]. International Conference on Language Resources and Evaluation, Vol. 2018. NIH Public Access, 156."},{"key":"e_1_2_1_15_1","volume-title":"Retiring adult: New datasets for fair machine learning. NeurIPS","author":"Ding Frances","year":"2021","unstructured":"Frances Ding , Moritz Hardt , John Miller , and Ludwig Schmidt . 2021. Retiring adult: New datasets for fair machine learning. NeurIPS ( 2021 ). Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. 2021. Retiring adult: New datasets for fair machine learning. NeurIPS (2021)."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i01.5402"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/773153.773173"},{"key":"e_1_2_1_18_1","unstructured":"Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. http:\/\/archive.ics.uci.edu\/ml  Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. http:\/\/archive.ics.uci.edu\/ml"},{"key":"e_1_2_1_19_1","volume-title":"Theory of cryptography conference","author":"Dwork Cynthia","unstructured":"Cynthia Dwork , Frank McSherry , Kobbi Nissim , and Adam Smith . 2006. Calibrating noise to sensitivity in private data analysis . In Theory of cryptography conference . Springer , 265--284. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference. Springer, 265--284."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1536414.1536467"},{"key":"e_1_2_1_21_1","first-page":"3","article-title":"The algorithmic foundations of differential privacy","volume":"9","author":"Dwork Cynthia","year":"2014","unstructured":"Cynthia Dwork , Aaron Roth , 2014 . The algorithmic foundations of differential privacy . Foundations and Trends in Theoretical Computer Science 9 , 3 -- 4 (2014), 211--407. Cynthia Dwork, Aaron Roth, et al. 2014. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science 9, 3--4 (2014), 211--407.","journal-title":"Foundations and Trends in Theoretical Computer Science"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.67995"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11121-018-0908-3"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1002\/ajcp.12233"},{"key":"e_1_2_1_25_1","volume-title":"Robin Hood and Matthew Effects-Differential Privacy Has Disparate Impact on Synthetic Data. arXiv preprint arXiv:2109.11429","author":"Ganev Georgi","year":"2021","unstructured":"Georgi Ganev , Bristena Oprisanu , and Emiliano De Cristofaro . 2021. Robin Hood and Matthew Effects-Differential Privacy Has Disparate Impact on Synthetic Data. arXiv preprint arXiv:2109.11429 ( 2021 ). Georgi Ganev, Bristena Oprisanu, and Emiliano De Cristofaro. 2021. Robin Hood and Matthew Effects-Differential Privacy Has Disparate Impact on Synthetic Data. arXiv preprint arXiv:2109.11429 (2021)."},{"key":"e_1_2_1_26_1","volume-title":"A simple and practical algorithm for differentially private data release. arXiv preprint arXiv:1012.4763","author":"Hardt Moritz","year":"2010","unstructured":"Moritz Hardt , Katrina Ligett , and Frank McSherry . 2010. A simple and practical algorithm for differentially private data release. arXiv preprint arXiv:1012.4763 ( 2010 ). Moritz Hardt, Katrina Ligett, and Frank McSherry. 2010. A simple and practical algorithm for differentially private data release. arXiv preprint arXiv:1012.4763 (2010)."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.3886\/ICPSR21600.v25"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2882931"},{"key":"e_1_2_1_29_1","volume-title":"Medical Data Privacy Handbook","author":"Hill Raquel","unstructured":"Raquel Hill . 2015. Evaluating the Utility of Differential Privacy: A Use Case Study of a Behavioral Science Dataset . In Medical Data Privacy Handbook . Springer , 59--82. Raquel Hill. 2015. Evaluating the Utility of Differential Privacy: A Use Case Study of a Behavioral Science Dataset. In Medical Data Privacy Handbook. Springer, 59--82."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.3886\/ICPSR04690.v9"},{"key":"e_1_2_1_31_1","volume-title":"High school football and risk for depression and suicidality in adulthood: findings from a national longitudinal study. Frontiers in neurology 12","author":"Iverson Grant L","year":"2021","unstructured":"Grant L Iverson and Douglas P Terry . 2021. High school football and risk for depression and suicidality in adulthood: findings from a national longitudinal study. Frontiers in neurology 12 ( 2021 ). Grant L Iverson and Douglas P Terry. 2021. High school football and risk for depression and suicidality in adulthood: findings from a national longitudinal study. Frontiers in neurology 12 (2021)."},{"key":"e_1_2_1_32_1","unstructured":"Satish Iyengar and Joel B Greenhouse. 1988. Selection models and the file drawer problem. Statist. Sci. (1988) 109--117.  Satish Iyengar and Joel B Greenhouse. 1988. Selection models and the file drawer problem. Statist. Sci. (1988) 109--117."},{"key":"e_1_2_1_33_1","unstructured":"Matthew Jagielski Michael Kearns Jieming Mao Alina Oprea Aaron Roth Saeed Sharifi-Malvajerdi and Jonathan Ullman. 2019. Differentially private fair learning. In ICML. 3000--3008.  Matthew Jagielski Michael Kearns Jieming Mao Alina Oprea Aaron Roth Saeed Sharifi-Malvajerdi and Jonathan Ullman. 2019. Differentially private fair learning. In ICML. 3000--3008."},{"key":"e_1_2_1_34_1","volume-title":"28th USENIX Security Symposium (USENIX Security 19)","author":"Jayaraman Bargav","year":"2019","unstructured":"Bargav Jayaraman and David Evans . 2019 . Evaluating differentially private machine learning in practice . In 28th USENIX Security Symposium (USENIX Security 19) . 1895--1912. Bargav Jayaraman and David Evans. 2019. Evaluating differentially private machine learning in practice. In 28th USENIX Security Symposium (USENIX Security 19). 1895--1912."},{"key":"e_1_2_1_35_1","volume-title":"35th Conference on Neural Information Processing Systems (NeurIPS 2021) Workshop on Math AI for Education (MATHAI4ED).","author":"Jeong Haewon","unstructured":"Haewon Jeong , Michael D. Wu , Nilanjana Dasgupta , Muriel M\u00e9dard , and Flavio P. Calmon . 2021. WhoGets the Benefit of the Doubt? Racial Bias in Machine Learning Algorithms Applied to Secondary School Math Education . In 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Workshop on Math AI for Education (MATHAI4ED). Haewon Jeong, Michael D. Wu, Nilanjana Dasgupta, Muriel M\u00e9dard, and Flavio P. Calmon. 2021. WhoGets the Benefit of the Doubt? Racial Bias in Machine Learning Algorithms Applied to Secondary School Math Education. In 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Workshop on Math AI for Education (MATHAI4ED)."},{"key":"e_1_2_1_36_1","volume-title":"Tyler Simko, and Kosuke Imai.","author":"Kenny Christopher T","year":"2021","unstructured":"Christopher T Kenny , Shiro Kuriwaki , Cory McCartan , Evan TR Rosenman , Tyler Simko, and Kosuke Imai. 2021 . The use of differential privacy for census data and its impact on redistricting: The case of the 2020 US Census. Science advances 7, 41 (2021), eabk3283. Christopher T Kenny, Shiro Kuriwaki, Cory McCartan, Evan TR Rosenman, Tyler Simko, and Kosuke Imai. 2021. The use of differential privacy for census data and its impact on redistricting: The case of the 2020 US Census. Science advances 7, 41 (2021), eabk3283."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.adolescence.2021.01.008"},{"key":"e_1_2_1_39_1","first-page":"690","article-title":"Iterative methods for private synthetic data: Unifying framework and new methods","volume":"34","author":"Liu Terrance","year":"2021","unstructured":"Terrance Liu , Giuseppe Vietri , and Steven Z Wu . 2021 . Iterative methods for private synthetic data: Unifying framework and new methods . Advances in Neural Information Processing Systems 34 (2021), 690 -- 702 . Terrance Liu, Giuseppe Vietri, and Steven Z Wu. 2021. Iterative methods for private synthetic data: Unifying framework and new methods. Advances in Neural Information Processing Systems 34 (2021), 690--702.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_40_1","volume-title":"Optimizing error of high-dimensional statistical queries under differential privacy. arXiv preprint arXiv:1808.03537","author":"McKenna Ryan","year":"2018","unstructured":"Ryan McKenna , Gerome Miklau , Michael Hay , and Ashwin Machanavajjhala . 2018. Optimizing error of high-dimensional statistical queries under differential privacy. arXiv preprint arXiv:1808.03537 ( 2018 ). Ryan McKenna, Gerome Miklau, Michael Hay, and Ashwin Machanavajjhala. 2018. Optimizing error of high-dimensional statistical queries under differential privacy. arXiv preprint arXiv:1808.03537 (2018)."},{"key":"e_1_2_1_41_1","volume-title":"Winning the NIST Contest: A scalable and general approach to differentially private synthetic data. arXiv preprint arXiv:2108.04978","author":"McKenna Ryan","year":"2021","unstructured":"Ryan McKenna , Gerome Miklau , and Daniel Sheldon . 2021. Winning the NIST Contest: A scalable and general approach to differentially private synthetic data. arXiv preprint arXiv:2108.04978 ( 2021 ). Ryan McKenna, Gerome Miklau, and Daniel Sheldon. 2021. Winning the NIST Contest: A scalable and general approach to differentially private synthetic data. arXiv preprint arXiv:2108.04978 (2021)."},{"key":"e_1_2_1_42_1","volume-title":"AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Data. arXiv preprint arXiv:2201.12677","author":"McKenna Ryan","year":"2022","unstructured":"Ryan McKenna , Brett Mullins , Daniel Sheldon , and Gerome Miklau . 2022 . AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Data. arXiv preprint arXiv:2201.12677 (2022). Ryan McKenna, Brett Mullins, Daniel Sheldon, and Gerome Miklau. 2022. AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Data. arXiv preprint arXiv:2201.12677 (2022)."},{"key":"e_1_2_1_43_1","volume-title":"Katherine S Button, Christopher D Chambers, Nathalie Percie du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J Ware, and John Ioannidis.","author":"Munaf\u00f2 Marcus R","year":"2017","unstructured":"Marcus R Munaf\u00f2 , Brian A Nosek , Dorothy VM Bishop , Katherine S Button, Christopher D Chambers, Nathalie Percie du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J Ware, and John Ioannidis. 2017 . A manifesto for reproducible science. Nature human behaviour 1, 1 (2017), 1--9. Marcus R Munaf\u00f2, Brian A Nosek, Dorothy VM Bishop, Katherine S Button, Christopher D Chambers, Nathalie Percie du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J Ware, and John Ioannidis. 2017. A manifesto for reproducible science. Nature human behaviour 1, 1 (2017), 1--9."},{"key":"e_1_2_1_44_1","volume-title":"Engineering, and Medicine and others","author":"National Academies of Sciences","year":"2019","unstructured":"National Academies of Sciences , Engineering, and Medicine and others . 2019 . Reproducibility and replicability in science. (2019). National Academies of Sciences, Engineering, and Medicine and others. 2019. Reproducibility and replicability in science. (2019)."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1708274114"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1177\/0265407518817400"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3085504.3091117"},{"key":"e_1_2_1_48_1","volume-title":"Advances in Knowledge Discovery and Data Mining","author":"Pinto F\u00e1bio","unstructured":"F\u00e1bio Pinto , Carlos Soares , and Jo\u00e3o Mendes-Moreira . 2016. Towards Automatic Generation of Metafeatures . In Advances in Knowledge Discovery and Data Mining , James Bailey, Latifur Khan, Takashi Washio, Gill Dobbie, Joshua Zhexue Huang, and Ruili Wang (Eds.). Springer International Publishing , Cham , 215--226. F\u00e1bio Pinto, Carlos Soares, and Jo\u00e3o Mendes-Moreira. 2016. Towards Automatic Generation of Metafeatures. In Advances in Knowledge Discovery and Data Mining, James Bailey, Latifur Khan, Takashi Washio, Gill Dobbie, Joshua Zhexue Huang, and Ruili Wang (Eds.). Springer International Publishing, Cham, 215--226."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.3758\/BRM.40.3.879"},{"key":"e_1_2_1_50_1","first-page":"1","article-title":"Multiple imputation for statistical disclosure limitation","volume":"19","author":"Raghunathan Trivellore E","year":"2003","unstructured":"Trivellore E Raghunathan , Jerome P Reiter , and Donald B Rubin . 2003 . Multiple imputation for statistical disclosure limitation . Journal of official statistics 19 , 1 (2003), 1 . Trivellore E Raghunathan, Jerome P Reiter, and Donald B Rubin. 2003. Multiple imputation for statistical disclosure limitation. Journal of official statistics 19, 1 (2003), 1.","journal-title":"Journal of official statistics"},{"key":"e_1_2_1_51_1","volume-title":"Noise-Aware Statistical Inference with Differentially Private Synthetic Data. arXiv preprint arXiv:2205.14485","author":"R\u00e4is\u00e4 Ossi","year":"2022","unstructured":"Ossi R\u00e4is\u00e4 , Joonas J\u00e4lk\u00f6 , Samuel Kaski , and Antti Honkela . 2022. Noise-Aware Statistical Inference with Differentially Private Synthetic Data. arXiv preprint arXiv:2205.14485 ( 2022 ). Ossi R\u00e4is\u00e4, Joonas J\u00e4lk\u00f6, Samuel Kaski, and Antti Honkela. 2022. Noise-Aware Statistical Inference with Differentially Private Synthetic Data. arXiv preprint arXiv:2205.14485 (2022)."},{"key":"e_1_2_1_52_1","volume-title":"Characterizing classification datasets: a study of meta-features for meta-learning. arXiv: Learning","author":"Rivolli Adriano","year":"2018","unstructured":"Adriano Rivolli , Lu\u00eds Paulo F. Garcia , Carlos Soares , Joaquin Vanschoren , and Andr\u00e9 Carlos Ponce de Leon Ferreira de Carvalho . 2018. Characterizing classification datasets: a study of meta-features for meta-learning. arXiv: Learning ( 2018 ). Adriano Rivolli, Lu\u00eds Paulo F. Garcia, Carlos Soares, Joaquin Vanschoren, and Andr\u00e9 Carlos Ponce de Leon Ferreira de Carvalho. 2018. Characterizing classification datasets: a study of meta-features for meta-learning. arXiv: Learning (2018)."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2204.12903"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2208.12700"},{"key":"e_1_2_1_55_1","volume-title":"Differentially private synthetic data: Applied evaluations and enhancements. arXiv preprint arXiv:2011.05537","author":"Rosenblatt Lucas","year":"2020","unstructured":"Lucas Rosenblatt , Xiaoyan Liu , Samira Pouyanfar , Eduardo de Leon , Anuj Desai , and Joshua Allen . 2020. Differentially private synthetic data: Applied evaluations and enhancements. arXiv preprint arXiv:2011.05537 ( 2020 ). Lucas Rosenblatt, Xiaoyan Liu, Samira Pouyanfar, Eduardo de Leon, Anuj Desai, and Joshua Allen. 2020. Differentially private synthetic data: Applied evaluations and enhancements. arXiv preprint arXiv:2011.05537 (2020)."},{"key":"e_1_2_1_56_1","volume-title":"The file drawer problem and tolerance for null results. Psychological bulletin 86, 3","author":"Rosenthal Robert","year":"1979","unstructured":"Robert Rosenthal . 1979. The file drawer problem and tolerance for null results. Psychological bulletin 86, 3 ( 1979 ), 638. Robert Rosenthal. 1979. The file drawer problem and tolerance for null results. Psychological bulletin 86, 3 (1979), 638."},{"key":"e_1_2_1_57_1","volume-title":"The bayesian bootstrap. The annals of statistics","author":"Rubin Donald B","year":"1981","unstructured":"Donald B Rubin . 1981. The bayesian bootstrap. The annals of statistics ( 1981 ), 130--134. Donald B Rubin. 1981. The bayesian bootstrap. The annals of statistics (1981), 130--134."},{"key":"e_1_2_1_58_1","volume-title":"AEA papers and proceedings","author":"Ruggles Steven","unstructured":"Steven Ruggles , Catherine Fitch , Diana Magnuson , and Jonathan Schroeder . 2019. Differential privacy and census data: Implications for social and economic research . In AEA papers and proceedings , Vol. 109 . 403--08. Steven Ruggles, Catherine Fitch, Diana Magnuson, and Jonathan Schroeder. 2019. Differential privacy and census data: Implications for social and economic research. In AEA papers and proceedings, Vol. 109. 403--08."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.3102\/0013189X18787818"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE51399.2021.00022"},{"key":"e_1_2_1_62_1","volume-title":"Benchmarking Differentially Private Synthetic Data Generation Algorithms. arXiv preprint arXiv:2112.09238","author":"Tao Yuchao","year":"2021","unstructured":"Yuchao Tao , Ryan McKenna , Michael Hay , Ashwin Machanavajjhala , and Gerome Miklau . 2021. Benchmarking Differentially Private Synthetic Data Generation Algorithms. arXiv preprint arXiv:2112.09238 ( 2021 ). Yuchao Tao, Ryan McKenna, Michael Hay, Ashwin Machanavajjhala, and Gerome Miklau. 2021. Benchmarking Differentially Private Synthetic Data Generation Algorithms. arXiv preprint arXiv:2112.09238 (2021)."},{"key":"e_1_2_1_63_1","volume-title":"DP-CGAN: Differentially Private Synthetic Data and Label Generation. CoRR abs\/2001.09700","author":"Torkzadehmahani Reihaneh","year":"2020","unstructured":"Reihaneh Torkzadehmahani , Peter Kairouz , and Benedict Paten . 2020. DP-CGAN: Differentially Private Synthetic Data and Label Generation. CoRR abs\/2001.09700 ( 2020 ). arXiv:2001.09700 https:\/\/arxiv.org\/abs\/2001.09700 Reihaneh Torkzadehmahani, Peter Kairouz, and Benedict Paten. 2020. DP-CGAN: Differentially Private Synthetic Data and Label Generation. CoRR abs\/2001.09700 (2020). arXiv:2001.09700 https:\/\/arxiv.org\/abs\/2001.09700"},{"key":"e_1_2_1_65_1","unstructured":"Giuseppe Vietri Grace Tian Mark Bun Thomas Steinke and Steven Wu. 2020. New oracle-efficient algorithms for private synthetic data release. In ICML. 9765--9774.  Giuseppe Vietri Grace Tian Mark Bun Thomas Steinke and Steven Wu. 2020. New oracle-efficient algorithms for private synthetic data release. In ICML. 9765--9774."},{"key":"e_1_2_1_66_1","volume-title":"Modeling tabular data using conditional gan. Advances in Neural Information Processing Systems 32","author":"Xu Lei","year":"2019","unstructured":"Lei Xu , Maria Skoularidou , Alfredo Cuesta-Infante , and Kalyan Veeramachaneni . 2019. Modeling tabular data using conditional gan. Advances in Neural Information Processing Systems 32 ( 2019 ). Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. 2019. Modeling tabular data using conditional gan. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_2_1_67_1","volume-title":"Xiaokui Xiao. SIGMOD","author":"Zhang Jun","year":"2014","unstructured":"Jun Zhang , Graham Cormode , Cecilia M Procopiuc , Divesh Srivastava , and Xiaokui Xiao. SIGMOD 2014 . PrivBayes : Private Data Release via Bayesian Networks. (SIGMOD 2014). Jun Zhang, Graham Cormode, Cecilia M Procopiuc, Divesh Srivastava, and Xiaokui Xiao. SIGMOD 2014. PrivBayes: Private Data Release via Bayesian Networks. (SIGMOD 2014)."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3611479.3611517","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,23]],"date-time":"2023-09-23T22:22:04Z","timestamp":1695507724000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3611479.3611517"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7]]},"references-count":63,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2023,7]]}},"alternative-id":["10.14778\/3611479.3611517"],"URL":"https:\/\/doi.org\/10.14778\/3611479.3611517","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2023,7]]},"assertion":[{"value":"2023-08-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}