{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T00:45:04Z","timestamp":1781743504652,"version":"3.54.5"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2025,6,4]],"date-time":"2025-06-04T00:00:00Z","timestamp":1748995200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,6,4]],"date-time":"2025-06-04T00:00:00Z","timestamp":1748995200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Digit. Soc."],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Synthetic data - artificially produced data used for various data science tasks - have become the subject of intense scholarly interest, engendering both hope and hype in fields like machine learning (ML) and data privacy. In this commentary, we shed light on a little-studied facet of the emerging synthetic data landscape: their evaluation through the use of different quality measures, such as privacy, utility, and fidelity metrics. While these may seem highly technical, this commentary argues that evaluation metrics are inextricably linked to the expectations, ethics and politics of synthetic data. Situating synthetic data metrics within longer histories of data measurement in big data and ML discourses, we unfold a conceptualization of synthetic data metrics as <jats:italic>metrological regimes<\/jats:italic> which highlights the multifaceted ways in which they are implicitly and explicitly political. We put this concept to use by providing a three-fold preliminary analysis of metrics for the evaluation of synthetic tabular data: first, we outline the current <jats:italic>constitution<\/jats:italic> of synthetic data\u2019s metrological regimes around utility, privacy, and fidelity metrics; second, we highlight the <jats:italic>performativity<\/jats:italic> of these metrological regimes; that is, how they overshadow other crucial measures and enact quantifications of essentially contested concepts; and third, we emphasize the <jats:italic>fragility<\/jats:italic> of synthetic data\u2019s metrological regimes by pointing to the eruption of specific negotiations regarding which privacy metrics (not) to use for synthetic data evaluation. By foregrounding how metrics shape the expectations, ethics, and politics of synthetic data, this commentary underlines the need for their critical study.<\/jats:p>","DOI":"10.1007\/s44206-025-00200-y","type":"journal-article","created":{"date-parts":[[2025,6,4]],"date-time":"2025-06-04T08:22:05Z","timestamp":1749025325000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Unraveling the Regimes of Synthetic Data Metrics: Expectations, Ethics, and Politics"],"prefix":"10.1007","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-4303-6905","authenticated-orcid":false,"given":"Louis","family":"Ravn","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8363-4855","authenticated-orcid":false,"given":"Vassilis","family":"Galanos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1510-414X","authenticated-orcid":false,"given":"Matthew","family":"Archer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4019-8958","authenticated-orcid":false,"given":"Danielle","family":"Shanley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,6,4]]},"reference":[{"key":"200_CR1","doi-asserted-by":"publisher","unstructured":"Amoore, L., Campolo, A., Jacobsen, B., & Rella, L. (2023). Machine learning, meaning making: On reading computer science texts. Big Data & Society, 10(1). https:\/\/doi.org\/10.1177\/20539517231166887","DOI":"10.1177\/20539517231166887"},{"key":"200_CR3","doi-asserted-by":"crossref","unstructured":"Archer, M. (2024). Unsustainable: Measurement, reporting, and the limits of corporate sustainability. New York University Press.","DOI":"10.18574\/nyu\/9781479822034.001.0001"},{"key":"200_CR2","doi-asserted-by":"publisher","unstructured":"Assefa, S. A., Dervovic, D., Mahfouz, M., Tillman, R. E., Reddy, P., & Veloso, M. (2020). Generating synthetic data in finance: Opportunities, challenges and pitfalls. ACM International Conference on AI in Finance, 1\u20138. https:\/\/doi.org\/10.1145\/3383455.3422554","DOI":"10.1145\/3383455.3422554"},{"key":"200_CR6","doi-asserted-by":"crossref","unstructured":"Barad, K. (2007). Meeting the universe halfway: Quantum physics and the entanglement of matter and meaning. Duke University Press.","DOI":"10.2307\/j.ctv12101zq"},{"issue":"2","key":"200_CR7","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1080\/03085140220123162","volume":"31","author":"A Barry","year":"2002","unstructured":"Barry, A. (2002). The anti-political economy. Economy and Society, 31(2), 268\u2013284.","journal-title":"Economy and Society"},{"key":"200_CR8","doi-asserted-by":"crossref","unstructured":"Bateson, G. (2000). Steps to an ecology of mind. University of Chicago Press. (Original work published 1972)","DOI":"10.7208\/chicago\/9780226924601.001.0001"},{"key":"200_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artmed.2023.102658","volume":"144","author":"R Baumgartner","year":"2023","unstructured":"Baumgartner, R., Arora, P., Bath, C., Burljaev, D., Ciereszko, K., Custers, B., & Williams, R. (2023). Fair and equitable AI in biomedical research and healthcare: Social science perspectives. Artificial Intelligence in Medicine, 144, 1\u20139.","journal-title":"Artificial Intelligence in Medicine"},{"issue":"4","key":"200_CR10","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1086\/287212","volume":"19","author":"H Becker","year":"1952","unstructured":"Becker, H. (1952). Science, culture, and society. Philosophy of Science, 19(4), 273\u2013287.","journal-title":"Philosophy of Science"},{"key":"200_CR12","doi-asserted-by":"publisher","unstructured":"Beer, D. (2016). Metric power. Palgrave Macmillan. https:\/\/doi.org\/10.1057\/978-1-137-55649-3","DOI":"10.1057\/978-1-137-55649-3"},{"key":"200_CR13","doi-asserted-by":"crossref","unstructured":"Belgodere, B., Dognin, P., Ivankay, A., Melnyk, I., Mroeuh, Y., Mojsilovic, A., Navratil, J., Nitsure, A., Padhi, I., Rigotti, M., Ross, J., Schiff, Y., Vedpathak, R., & Young, R. A. (2024). Auditing and generating synthetic data with controllable trust trade-offs. arXiv.","DOI":"10.1109\/JETCAS.2024.3477976"},{"key":"200_CR14","unstructured":"Birhane, A., Prabhu, V. U., & Kahembwe, E. (2021). Multimodal datasets: Misogyny, pornography, and malignant stereotypes. arXiv."},{"issue":"5","key":"200_CR15","doi-asserted-by":"publisher","first-page":"662","DOI":"10.1080\/1369118X.2012.678878","volume":"15","author":"D Boyd","year":"2012","unstructured":"Boyd, D., & Crawford, K. (2012). Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information Communication & Society, 15(5), 662\u2013679.","journal-title":"Information Communication & Society"},{"issue":"2","key":"200_CR16","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1080\/03085140220123171","volume":"31","author":"M Callon","year":"2002","unstructured":"Callon, M. (2002). Technology, politics And the market: An interview with Michel callon. Economy and Society, 31(2), 285\u2013306.","journal-title":"Economy and Society"},{"issue":"14","key":"200_CR17","doi-asserted-by":"publisher","first-page":"2110","DOI":"10.1080\/1369118X.2020.1754877","volume":"23","author":"K de Vries","year":"2020","unstructured":"de Vries, K. (2020). You never fake alone. Creative AI in action. Information, Communication & Society, 23(14), 2110\u20132127.","journal-title":"Information Communication & Society"},{"issue":"2","key":"200_CR18","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1214\/24-STS927","volume":"39","author":"J Drechsler","year":"2024","unstructured":"Drechsler, J., & Haensch, A-C. (2024). 30 Years of synthetic data. Statistical Science, 39(2), 221\u2013242.","journal-title":"Statistical Science"},{"issue":"2","key":"200_CR19","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1080\/1600910X.2022.2137546","volume":"24","author":"F Ferrari","year":"2022","unstructured":"Ferrari, F., & McKelvey, F. (2022). Hyperproduction: A social theory of deep generative models. Distinktion: Journal of Social Theory, 24(2), 338\u2013360.","journal-title":"Distinktion: Journal of Social Theory"},{"key":"200_CR20","doi-asserted-by":"crossref","unstructured":"Flusser, V. (2011). Into the universe of technical images (N. A. Roth, Trans.). University of Minnesota Press. (Original work published 1985).","DOI":"10.5749\/minnesota\/9780816670208.001.0001"},{"key":"200_CR49","unstructured":"Galanos, V. (2024). Between monstrous dystopias and policy utopias in artificial intelligence: Reporting on a ten-year journey into the AI-topia. In V. A. Bruno, A. Campati, P. Caerlli, & A. Sfardini (Eds.), Dystopian worlds beyond storytelling: Representations of dehumanized societies in literature, media, and political discourses: Multidisciplinary perspectives (pp. 51\u201368). ibidem-Verlag."},{"key":"200_CR21","doi-asserted-by":"crossref","unstructured":"Gallie, W. B. (1955). Essentially Contested Concepts. Proceedings of the Aristotelian Society, 56(1), 167\u2013198.","DOI":"10.1093\/aristotelian\/56.1.167"},{"key":"200_CR22","unstructured":"Ganev, G., & De Cristofaro, E. (2023). On the inadequacy of similarity-based privacy metrics: Reconstruction Attacks against \u201ctruly anonymous synthetic data.\u201d arXiv."},{"key":"200_CR23","unstructured":"Gretel. (n.d.). Synthetic data quality. Available at: https:\/\/docs.gretel.ai\/optimize-synthetic-data\/evaluate\/synthetic-data-quality-report. [Accessed: 09\/09\/2024]."},{"key":"200_CR24","unstructured":"Gustavsson, M., & Ljungberg, J. (2021). Algorithms and their work: A performativity perspective. 12th Scandinavian Conference on Information Systems, 7."},{"key":"200_CR25","doi-asserted-by":"publisher","unstructured":"Helm, P., Lipp, B., & Pujadas, R. (2024). Generating reality and Silencing debate: Synthetic data as discursive device. Big Data & Society, 11(2). https:\/\/doi.org\/10.1177\/20539517241249447","DOI":"10.1177\/20539517241249447"},{"key":"200_CR27","unstructured":"Hogan, M., & Lepage-Richer, T. (2024). Extractive AI. Centre for Media, Technology, and Democracy. Available at: https:\/\/static1.squarespace.com\/static\/5ea874746663b45e14a384a4\/t\/6664bcdf2cbd2d1d11ba6c03\/1717877983870\/CJT_Hogan+%26+Lepage-Richer.pdf. [Accessed: 11\/09\/2024]."},{"key":"200_CR28","doi-asserted-by":"crossref","unstructured":"Hutchinson, B., Rostamzadeh, N., Greer, C., Heller, K., & Prabhakaran, V. (2022). Evaluation gaps in machine learning practice. arXiv.","DOI":"10.1145\/3531146.3533233"},{"key":"200_CR29","unstructured":"International Organization for Migration (IOM). (2024, February 27). IOM releases the Global Synthetic Dataset. Available at: https:\/\/migrantprotection.iom.int\/en\/spotlight\/articles\/publication\/iom-releases-global-synthetic-dataset. [Accessed: 09\/09\/2024]."},{"issue":"1","key":"200_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/20539517221145372","volume":"10","author":"BN Jacobsen","year":"2023","unstructured":"Jacobsen, B. N. (2023). Machine learning and the politics of synthetic data. Big Data & Society, 10(1), 1\u201312.","journal-title":"Big Data & Society"},{"key":"200_CR31","unstructured":"Jordon, J., Szpruch, L., Houssiau, F., Bottarelli, M., Cherubin, G., Maple, C., Cohen, S. N., & Weller, A. (2022). Synthetic Data \u2013 what, why and how? arXiv."},{"issue":"1","key":"200_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-024-01359-3","volume":"8","author":"B Kaabachi","year":"2025","unstructured":"Kaabachi, B., Despraz, J., Meurers, T., Otte, K., Halilovic, M., Kulynych, B., Prasser, F., & Raisaro, J. L. (2025). A scoping review of privacy and utility metrics in medical synthetic data. npj Digital Medicine, 8(1), 1\u20139.","journal-title":"Npj Digital Medicine"},{"key":"200_CR33","doi-asserted-by":"crossref","unstructured":"Lautrup, A. D., Hyrup, T., Zimek, A., & Schneider-Kamp, P. (2024). SynthEval: A framework for detailed utility and privacy evaluation of tabular synthetic data. arXiv.","DOI":"10.1007\/s10618-024-01081-4"},{"key":"200_CR34","doi-asserted-by":"crossref","unstructured":"Le Bui, M., & Noble, S. U. (2020). We\u2019re missing a moral framework of justice in artificial intelligence: On the limits, failings, and ethics of fairness. In M. D. Dubber, F. Pasquale, & S. Das (Eds.), The Oxford Handbook of Ethics of AI (pp. 162\u2013179). Oxford University Press.","DOI":"10.1093\/oxfordhb\/9780190067397.013.9"},{"key":"200_CR35","doi-asserted-by":"publisher","unstructured":"Luitse, D., Blanke, T., & Poell, T. (2024). AI competitions as infrastructures of power in medical imaging. Information, Communication & Society, 1\u201322. https:\/\/doi.org\/10.1080\/1369118X.2024.2334393","DOI":"10.1080\/1369118X.2024.2334393"},{"key":"200_CR36","unstructured":"Mitchell, M., Luccioni, A. S., Lambert, N., Gerchick, M., McMillan-Major, A., Ozoani, E., Rajani, N., Thrush, T., Jernite, Y., & Kiela, D. (2023). Measuring data. arXiv."},{"key":"200_CR37","doi-asserted-by":"publisher","unstructured":"Offenhuber, D. (2024). Shapes and frictions of synthetic data. Big Data & Society, 11(2). https:\/\/doi.org\/10.1177\/20539517241249390","DOI":"10.1177\/20539517241249390"},{"issue":"11","key":"200_CR38","first-page":"1","volume":"18","author":"C Orwat","year":"2024","unstructured":"Orwat, C., Bareis, J., Folberth, A., Jahnel, J., & Wadephul, C. (2024). Normative challenges of risk regulation of artificial intelligence. NanoEthics, 18(11), 1\u201329.","journal-title":"NanoEthics"},{"key":"200_CR39","unstructured":"Raji, I. D., Bender, E. M., Paullada, A., Denton, E., & Hanna, A. (2021). AI and the everything in the whole wide world benchmark. arXiv."},{"key":"200_CR4","doi-asserted-by":"publisher","unstructured":"Ravn, L. (2025). The fabrication of synthetic data promises: Tracing emerging arenas of expectations and boundary work. Big Data & Society, 12(1). https:\/\/doi.org\/10.1177\/20539517241307915","DOI":"10.1177\/20539517241307915"},{"issue":"1","key":"200_CR40","first-page":"246","volume":"2","author":"G Santangelo","year":"2024","unstructured":"Santangelo, G., Nicora, G., Bellazzi, R., & Dagliati, A. (2024). SynthCheck: A dashboard for synthetic data quality assessment. Proceedings of the 17th International Joint Conference on Biomedical Engineering Systems and Technology, 2(1), 246\u2013256.","journal-title":"Proceedings of the 17th International Joint Conference on Biomedical Engineering Systems and Technology"},{"key":"200_CR41","doi-asserted-by":"crossref","unstructured":"Sarmin, F. J., Sarkar, A. R., Wang, Y., & Mohammed, N. (2024). Synthetic data: Revisiting the privacy-utility trade-off. arXiv.","DOI":"10.1007\/s10207-025-01072-6"},{"key":"200_CR42","doi-asserted-by":"crossref","unstructured":"Scheuermann, M. K., Denton, E., & Hanna, A. (2021). Do datasets have politics? Disciplinary values in computer vision dataset development. Proceedings of the ACM on Human-Computer Interaction, 5, 1\u201337.","DOI":"10.1145\/3476058"},{"key":"200_CR5","doi-asserted-by":"publisher","unstructured":"Shanley, D., Hoogenboom, J., Lysen, F., Wee, L., Gomes, A. L., Dekker, A., & Meacham, D. (2024). Getting real about synthetic data ethics: Are AI ethics principles a good starting point for synthetic data ethics? EMBO Reports, 25(5), 2152\u20132155. https:\/\/doi.org\/10.1038\/s44319-024-00101-0","DOI":"10.1038\/s44319-024-00101-0"},{"issue":"6","key":"200_CR43","doi-asserted-by":"publisher","first-page":"3290","DOI":"10.1177\/14614448221099217","volume":"26","author":"J Steinhoff","year":"2022","unstructured":"Steinhoff, J. (2022). Toward a political economy of synthetic data: A data-intensive capitalism that is not surveillance capitalism? New Media & Society, 26(6), 3290\u20133306.","journal-title":"New Media & Society"},{"key":"200_CR44","doi-asserted-by":"crossref","unstructured":"Steinhoff, J., & Hind, S. (2025). Simulation and the reality gap: Moments in a prehistory of synthetic data. Big Data & Society, 12(1).","DOI":"10.1177\/20539517241309884"},{"issue":"4\u20135","key":"200_CR45","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1177\/1367549415577392","volume":"18","author":"T Striphas","year":"2015","unstructured":"Striphas, T. (2015). Algorithmic culture. European Journal of Cultural Studies, 18(4\u20135), 395\u2013412.","journal-title":"European Journal of Cultural Studies"},{"key":"200_CR46","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ijmedinf.2024.105413","volume":"185","author":"VB Vallevik","year":"2024","unstructured":"Vallevik, V. B., Babic, A., Marshall, S. E., Elvatun, S., Br\u00f8gger, B., Alagaratnam, S., Edwin, B., Veeraragavan, N. R., Befring, A. K., & Nyg\u00e5rd, J. F. (2024). Can I trust my fake data - A comprehensive quality assessment framework for synthetic tabular data in healthcare. International Journal of Medical Informatics, 185, 1\u201320.","journal-title":"International Journal of Medical Informatics"},{"key":"200_CR47","unstructured":"Veselovsky, V., Ribeiro, M. H., & West, R. (2023). Artificial artificial artificial intelligence: Crowd workers widely use large language models for text production tasks. arXiv."},{"key":"200_CR48","unstructured":"YData. (n.d.). Synthetic data quality metrics. Available at: https:\/\/ydata.ai\/synthetic-data-quality-metrics. [Accessed: 23\/08\/2024]."}],"container-title":["Digital Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44206-025-00200-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s44206-025-00200-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44206-025-00200-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T15:25:27Z","timestamp":1756913127000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s44206-025-00200-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,4]]},"references-count":47,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,8]]}},"alternative-id":["200"],"URL":"https:\/\/doi.org\/10.1007\/s44206-025-00200-y","relation":{},"ISSN":["2731-4650","2731-4669"],"issn-type":[{"value":"2731-4650","type":"print"},{"value":"2731-4669","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,4]]},"assertion":[{"value":"30 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 April 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 June 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author declares that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}],"article-number":"44"}}