{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T15:15:10Z","timestamp":1787498110785,"version":"build-2736575974"},"reference-count":195,"publisher":"MIT Press","issue":"1","license":[{"start":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T00:00:00Z","timestamp":1727654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,3,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>User-generated content provides a rich resource to study social and behavioral phenomena. Although its application potential is currently limited by the paucity of expert labels and the privacy risks inherent in personal data, synthetic data can help mitigate this bottleneck. In this work, we introduce an evaluation framework to facilitate research on synthetic language data generation for user-generated text. We define a set of aspects for assessing data quality, namely, style preservation, meaning preservation, and divergence, as a proxy for privacy. We introduce metrics corresponding to each aspect. Moreover, through a set of generation strategies and representative tasks and baselines across domains, we demonstrate the relation between the quality aspects of synthetic user generated content, generation strategies, metrics, and downstream performance. To our knowledge, our work is the first unified evaluation framework for user-generated text in relation to the specified aspects, offering both intrinsic and extrinsic evaluation. We envisage it will facilitate developments towards shareable, high-quality synthetic language data.<\/jats:p>","DOI":"10.1162\/coli_a_00540","type":"journal-article","created":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T09:57:55Z","timestamp":1727690275000},"page":"191-233","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":11,"title":["Evaluating Synthetic Data Generation from User Generated Text"],"prefix":"10.1162","volume":"51","author":[{"given":"Jenny","family":"Chim","sequence":"first","affiliation":[{"name":"Queen Mary University of London, School of Electronic Engineering, and Computer Science. c.chim@qmul.ac.uk"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julia","family":"Ive","sequence":"additional","affiliation":[{"name":"Queen Mary University of London, School of Electronic Engineering, and Computer Science. j.ive@qmul.ac.uk"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maria","family":"Liakata","sequence":"additional","affiliation":[{"name":"Queen Mary University of London \/ The Alan Turing Institute, School of Electronic Engineering and Computer Science. m.liakata@qmul.ac.uk"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2025,3,15]]},"reference":[{"issue":"2","key":"2025032113505803300_bib1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1344411.1344413","article-title":"Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace","volume":"26","author":"Abbasi","year":"2008","journal-title":"ACM Transactions on Information Systems"},{"key":"2025032113505803300_bib2","doi-asserted-by":"publisher","first-page":"8687","DOI":"10.18653\/v1\/2021.emnlp-main.684","article-title":"Preventing author profiling through zero-shot multilingual back-translation","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Adelani","year":"2021"},{"key":"2025032113505803300_bib3","article-title":"CUDA: Curriculum of data augmentation for long-tailed recognition","volume-title":"The Eleventh International Conference on Learning Representations","author":"Ahn","year":"2023"},{"key":"2025032113505803300_bib4","article-title":"How faithful is your synthetic data? Sample-level metrics for evaluating and auditing generative models","volume":"abs\/2102.08921","author":"Alaa","year":"2021","journal-title":"ArXiv"},{"key":"2025032113505803300_bib5","doi-asserted-by":"publisher","first-page":"6168","DOI":"10.18653\/v1\/P19-1620","article-title":"Synthetic QA corpora generation with roundtrip consistency","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Alberti","year":"2019"},{"key":"2025032113505803300_bib6","doi-asserted-by":"publisher","first-page":"4242","DOI":"10.18653\/v1\/2021.findings-emnlp.359","article-title":"The topic confusion task: A novel evaluation scenario for authorship attribution","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Altakrori","year":"2021"},{"key":"2025032113505803300_bib7","doi-asserted-by":"publisher","first-page":"2391","DOI":"10.18653\/v1\/2022.emnlp-main.153","article-title":"A multifaceted framework to evaluate evasion, content preservation, and misattribution in authorship obfuscation techniques","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Altakrori","year":"2022"},{"key":"2025032113505803300_bib8","doi-asserted-by":"publisher","first-page":"898","DOI":"10.1109\/TPAMI.2010.161","article-title":"Contour detection and hierarchical image segmentation","volume":"33","author":"Arbel\u00e1ez","year":"2011","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2025032113505803300_bib9","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1145\/1179601.1179608","article-title":"Scanning electronic documents for personally identifiable information","volume-title":"Proceedings of the 5th ACM Workshop on Privacy in Electronic Society","author":"Aura","year":"2006"},{"key":"2025032113505803300_bib10","first-page":"18","article-title":"LAMP: Extracting text from gradients with language model priors","volume-title":"Conference on Neural Information Processing Systems","author":"Balunovi\u0107","year":"2022"},{"key":"2025032113505803300_bib11","first-page":"65","article-title":"METEOR: An automatic metric for MT evaluation with improved correlation with human judgments","volume-title":"Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization","author":"Banerjee","year":"2005"},{"key":"2025032113505803300_bib12","first-page":"1","article-title":"The problem with bias: Allocative versus representational harms in machine learning","volume-title":"9th Annual Conference of the Special Interest Group for Computing, Information and Society","author":"Barocas","year":"2017"},{"key":"2025032113505803300_bib13","doi-asserted-by":"publisher","first-page":"635","DOI":"10.1111\/jcom.12106","article-title":"Self-disclosure in social media: Extending the functional approach to disclosure motivations and characteristics on social network sites","volume":"64","author":"Bazarova","year":"2014","journal-title":"Journal of Communication"},{"key":"2025032113505803300_bib14","doi-asserted-by":"publisher","first-page":"5454","DOI":"10.18653\/v1\/2020.acl-main.485","article-title":"Language (technology) is power: A critical survey of \u201cbias\u201d in NLP","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Blodgett","year":"2020"},{"key":"2025032113505803300_bib15","doi-asserted-by":"publisher","first-page":"1119","DOI":"10.18653\/v1\/D16-1120","article-title":"Demographic dialectal variation in social media: A case study of African-American English","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Blodgett","year":"2016"},{"key":"2025032113505803300_bib16","volume-title":"Individual and Group Privacy","author":"Bloustein","year":"1978"},{"key":"2025032113505803300_bib17","first-page":"4356","article-title":"Man is to computer programmer as woman is to homemaker? Debiasing word embeddings","volume-title":"Proceedings of the 30th International Conference on Neural Information Processing Systems","author":"Bolukbasi","year":"2016"},{"key":"2025032113505803300_bib18","first-page":"21","article-title":"On the resemblance and containment of documents","author":"Broder","year":"1997","journal-title":"Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No.97TB100171)"},{"key":"2025032113505803300_bib19","first-page":"1877","article-title":"Language models are few-shot learners","volume-title":"Advances in Neural Information Processing Systems","author":"Brown","year":"2020"},{"key":"2025032113505803300_bib20","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1186\/1472-6947-10-59","article-title":"Data-driven approach for creating synthetic electronic medical records","volume":"10","author":"Buczak","year":"2010","journal-title":"BMC Medical Informatics and Decision Making"},{"key":"2025032113505803300_bib21","doi-asserted-by":"publisher","first-page":"105367","DOI":"10.1016\/j.clsr.2019.105367","article-title":"The chilling effects of algorithmic profiling: Mapping the issues","volume":"36","author":"B\u00fcchi","year":"2020","journal-title":"Computer Law & Security Review"},{"key":"2025032113505803300_bib22","first-page":"19","article-title":"Extracting training data from large language models","volume-title":"USENIX Security Symposium","author":"Carlini","year":"2021"},{"key":"2025032113505803300_bib23","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1162\/tacl_a_00542","article-title":"An empirical survey of data augmentation for limited data learning in NLP","volume":"11","author":"Chen","year":"2023","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2025032113505803300_bib24","doi-asserted-by":"publisher","first-page":"1504","DOI":"10.18653\/v1\/2023.acl-long.84","article-title":"Marked personas: Using natural language prompts to measure stereotypes in language models","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Cheng","year":"2023"},{"key":"2025032113505803300_bib25","doi-asserted-by":"publisher","first-page":"10853","DOI":"10.18653\/v1\/2023.emnlp-main.669","article-title":"CoMPosT: Characterizing and evaluating caricature in LLM simulations","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Cheng","year":"2023"},{"key":"2025032113505803300_bib26","doi-asserted-by":"publisher","first-page":"792","DOI":"10.18653\/v1\/2023.emnlp-industry.74","article-title":"Batch prompting: Efficient inference with large language model APIs","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track","author":"Cheng","year":"2023"},{"key":"2025032113505803300_bib27","first-page":"286","article-title":"Generating multi-label discrete patient records using generative adversarial networks","volume-title":"Proceedings of the 2nd Machine Learning for Healthcare Conference","author":"Choi","year":"2017"},{"issue":"15","key":"2025032113505803300_bib28","doi-asserted-by":"publisher","first-page":"8344","DOI":"10.1073\/pnas.1914598117","article-title":"Towards formalizing the GDPR\u2019s notion of singling out","volume":"117","author":"Cohen","year":"2020","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2025032113505803300_bib29","article-title":"Promptagator: Few-shot dense retrieval from 8 examples","volume-title":"Eleventh International Conference on Learning Representations","author":"Dai","year":"2023"},{"issue":"12","key":"2025032113505803300_bib30","doi-asserted-by":"publisher","first-page":"2054","DOI":"10.1038\/s41562-023-01750-2","article-title":"Platform-controlled social media APIs threaten open science","volume":"7","author":"Davidson","year":"2023","journal-title":"Nature Human Behaviour"},{"key":"2025032113505803300_bib31","doi-asserted-by":"publisher","first-page":"7331","DOI":"10.18653\/v1\/2022.acl-long.506","article-title":"Evaluating factuality in text simplification","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Devaraj","year":"2022"},{"issue":"3","key":"2025032113505803300_bib32","first-page":"451","article-title":"A computational theory of goal-directed style in syntax","volume":"19","author":"DiMarco","year":"1993","journal-title":"Computational Linguistics"},{"key":"2025032113505803300_bib33","doi-asserted-by":"publisher","first-page":"8173","DOI":"10.18653\/v1\/2020.emnlp-main.656","article-title":"Queens are powerful too: Mitigating gender bias in dialogue generation","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Dinan","year":"2020"},{"key":"2025032113505803300_bib34","doi-asserted-by":"publisher","first-page":"1286","DOI":"10.18653\/v1\/2021.emnlp-main.98","article-title":"Documenting large webtext corpora: A case study on the Colossal Clean Crawled Corpus","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Dodge","year":"2021"},{"key":"2025032113505803300_bib35","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1007\/BF02912493","article-title":"Human conversational behavior","volume":"8","author":"Dunbar","year":"1997","journal-title":"Human Nature"},{"key":"2025032113505803300_bib36","article-title":"Towards measuring the representation of subjective global opinions in language models","author":"Durmus","year":"2023","journal-title":"arXiv preprint arXiv:2306.16388"},{"key":"2025032113505803300_bib37","doi-asserted-by":"publisher","first-page":"1066","DOI":"10.1162\/tacl_a_00506","article-title":"Evaluating attribution in dialogue systems: The BEGIN benchmark","volume":"10","author":"Dziri","year":"2022","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2025032113505803300_bib38","article-title":"TinyStories: How small can language models be and still speak coherent English?","author":"Eldan","year":"2023","journal-title":"arXiv preprint arXiv:2305.07759"},{"key":"2025032113505803300_bib39","doi-asserted-by":"publisher","first-page":"15044","DOI":"10.18653\/v1\/2023.acl-long.839","article-title":"Don\u2019t forget your ABC\u2019s: Evaluating the state-of-the-art in chat-oriented dialogue systems","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Finch","year":"2023"},{"key":"2025032113505803300_bib40","doi-asserted-by":"publisher","first-page":"2019","DOI":"10.1109\/COMST.2014.2321628","article-title":"Online social networks: Threats and solutions","volume":"16","author":"Fire","year":"2013","journal-title":"IEEE Communications Surveys & Tutorials"},{"issue":"2","key":"2025032113505803300_bib41","doi-asserted-by":"publisher","first-page":"e19","DOI":"10.2196\/mental.7785","article-title":"Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): A randomized controlled trial","volume":"4","author":"Fitzpatrick","year":"2017","journal-title":"JMIR Ment Health"},{"key":"2025032113505803300_bib42","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46608-8_5","article-title":"Group privacy: A defence and an interpretation","volume-title":"Group Privacy","author":"Floridi","year":"2017"},{"key":"2025032113505803300_bib43","doi-asserted-by":"publisher","first-page":"82","DOI":"10.3115\/1609843.1609855","article-title":"GenERRate: Generating errors for use in grammatical error detection","volume-title":"Proceedings of the Fourth Workshop on Innovative Use of NLP for Building Educational Applications","author":"Foster","year":"2009"},{"key":"2025032113505803300_bib44","first-page":"46","article-title":"Results of WMT22 metrics shared task: Stop using BLEU \u2013 neural metrics are better and more robust","volume-title":"Proceedings of the Seventh Conference on Machine Translation (WMT)","author":"Freitag","year":"2022"},{"key":"2025032113505803300_bib45","doi-asserted-by":"publisher","first-page":"478","DOI":"10.18653\/v1\/2021.findings-acl.42","article-title":"GO FIGURE: A meta evaluation of factuality in summarization","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Gabriel","year":"2021"},{"key":"2025032113505803300_bib46","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.552","article-title":"The Pile: An 800GB dataset of diverse text for language modeling","volume":"abs\/2101.00027","author":"Gao","year":"2021","journal-title":"CoRR"},{"key":"2025032113505803300_bib47","doi-asserted-by":"publisher","first-page":"6894","DOI":"10.18653\/v1\/2021.emnlp-main.552","article-title":"SimCSE: Simple contrastive learning of sentence embeddings","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Gao","year":"2021"},{"key":"2025032113505803300_bib48","doi-asserted-by":"publisher","first-page":"96","DOI":"10.18653\/v1\/2021.gem-1.10","article-title":"The GEM benchmark: Natural language generation, its evaluation and metrics","volume-title":"Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)","author":"Gehrmann","year":"2021"},{"key":"2025032113505803300_bib49","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1109\/ICASSP.1992.225858","article-title":"SWITCHBOARD: Telephone speech corpus for research and development","volume":"1","author":"Godfrey","year":"1992","journal-title":"[Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing"},{"key":"2025032113505803300_bib50","doi-asserted-by":"publisher","first-page":"175","DOI":"10.2478\/popets-2020-0068","article-title":"Effective writing style transfer via combinatorial paraphrasing","volume":"2020","author":"Gr\u00f6ndahl","year":"2020","journal-title":"Proceedings on Privacy Enhancing Technologies"},{"key":"2025032113505803300_bib51","doi-asserted-by":"publisher","first-page":"3785","DOI":"10.18653\/v1\/2022.acl-long.263","article-title":"DialFact: A benchmark for fact-checking in dialogue","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Gupta","year":"2022"},{"key":"2025032113505803300_bib52","doi-asserted-by":"publisher","first-page":"1322","DOI":"10.1109\/IJCNN.2008.4633969","article-title":"ADASYN: Adaptive synthetic sampling approach for imbalanced learning","volume-title":"2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence)","author":"He","year":"2008"},{"key":"2025032113505803300_bib53","doi-asserted-by":"publisher","first-page":"826","DOI":"10.1162\/tacl_a_00492","article-title":"Generate, annotate, and learn: NLP with synthetic text","volume":"10","author":"He","year":"2022","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2025032113505803300_bib54","first-page":"1693","article-title":"Teaching machines to read and comprehend","volume-title":"Advances in Neural Information Processing Systems","author":"Hermann","year":"2015"},{"key":"2025032113505803300_bib55","first-page":"6629","article-title":"GANs trained by a two time-scale update rule converge to a local Nash equilibrium","volume":"30","author":"Heusel","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2025032113505803300_bib56","doi-asserted-by":"publisher","first-page":"3791","DOI":"10.18653\/v1\/2023.eacl-main.274","article-title":"Creation and evaluation of timelines for longitudinal user posts","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Hills","year":"2023"},{"key":"2025032113505803300_bib57","article-title":"End-to-end conversation modeling track in DSTC6","author":"Hori","year":"2017","journal-title":"ArXiv preprint arXiv:1706.07440"},{"issue":"6245","key":"2025032113505803300_bib58","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1126\/science.aac4520","article-title":"Data, privacy, and the greater good","volume":"349","author":"Horvitz","year":"2015","journal-title":"Science"},{"issue":"8","key":"2025032113505803300_bib59","doi-asserted-by":"publisher","first-page":"e12432","DOI":"10.1111\/lnc3.12432","article-title":"Five sources of bias in natural language processing","volume":"15","author":"Hovy","year":"2021","journal-title":"Language and Linguistics Compass"},{"key":"2025032113505803300_bib60","article-title":"Disfluencies in dialogues with patients with schizophrenia","volume-title":"Proceedings of the 39th Annual Meeting of the Cognitive Science Society","author":"Howes","year":"2017"},{"key":"2025032113505803300_bib61","doi-asserted-by":"publisher","first-page":"5137","DOI":"10.18653\/v1\/2023.findings-acl.318","article-title":"Code-switched text synthesis in unseen language pairs","volume-title":"Findings of the Association for Computational Linguistics: ACL 2023","author":"Hsu","year":"2023"},{"key":"2025032113505803300_bib62","first-page":"1587","article-title":"Toward controlled generation of text","volume-title":"International Conference on Machine Learning","author":"Hu","year":"2017"},{"key":"2025032113505803300_bib63","article-title":"How much is too much? Privacy issues on Twitter","volume-title":"Conference of International Communication Association","author":"Humphreys","year":"2010"},{"key":"2025032113505803300_bib64","doi-asserted-by":"publisher","first-page":"13914","DOI":"10.18653\/v1\/2023.findings-acl.874","article-title":"DP-BART for privatized text rewriting under local differential privacy","volume-title":"Findings of the Association for Computational Linguistics: ACL 2023","author":"Igamberdiev","year":"2023"},{"issue":"3","key":"2025032113505803300_bib65","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1037\/a0020386","article-title":"Language style matching in writing: Synchrony in essays, correspondence, and poetry.","volume":"99","author":"Ireland","year":"2010","journal-title":"Journal of Personality and Social Psychology"},{"key":"2025032113505803300_bib66","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1038\/s41746-020-0267-x","article-title":"Generation and evaluation of artificial mental health records for natural language processing","volume":"3","author":"Ive","year":"2020","journal-title":"NPJ Digital Medicine"},{"key":"2025032113505803300_bib67","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1145\/3571730","article-title":"Survey of hallucination in natural language generation","author":"Ji","year":"2022","journal-title":"ACM Computing Surveys"},{"key":"2025032113505803300_bib68","article-title":"Mixtral of experts","author":"Jiang","year":"2024","journal-title":"ArXiv preprint arXiv:2401.04088"},{"key":"2025032113505803300_bib69","first-page":"22","article-title":"Evaluating and inducing personality in pre-trained language models","volume-title":"Thirty-seventh Conference on Neural Information Processing Systems","author":"Jiang","year":"2023"},{"issue":"3","key":"2025032113505803300_bib70","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1561\/1500000005","article-title":"Authorship attribution","volume":"1","author":"Juola","year":"2006","journal-title":"Foundations and Trends in Information Retrieval"},{"key":"2025032113505803300_bib71","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1109\/ASRU.1997.658992","article-title":"Automatic detection of discourse structure for speech recognition and understanding","volume-title":"1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings","author":"Jurafsky","year":"1997"},{"key":"2025032113505803300_bib72","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1007\/978-3-319-65813-1_18","article-title":"The case for being average: A mediocrity approach to style masking and author obfuscation","volume-title":"Experimental IR Meets Multilinguality, Multimodality, and Interaction","author":"Karadzhov","year":"2017"},{"key":"2025032113505803300_bib73","doi-asserted-by":"publisher","first-page":"3540","DOI":"10.18653\/v1\/2022.naacl-main.259","article-title":"Bidimensional leaderboards: Generate and evaluate language hand in hand","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Kasai","year":"2022"},{"key":"2025032113505803300_bib74","doi-asserted-by":"publisher","DOI":"10.1145\/3394231.3397919","article-title":"Analysing privacy leakage of life events on Twitter","volume-title":"Proceedings of the 12th ACM Conference on Web Science","author":"Kek\u00fcll\u00fco\u011flu","year":"2020"},{"issue":"1","key":"2025032113505803300_bib75","doi-asserted-by":"publisher","first-page":"103116","DOI":"10.1016\/j.ipm.2022.103116","article-title":"Evaluating the generalisability of neural rumour verification models","volume":"60","author":"Kochkina","year":"2023","journal-title":"Information Processing & Management"},{"issue":"1","key":"2025032113505803300_bib76","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1007\/s10579-009-9111-2","article-title":"Authorship attribution in the wild","volume":"45","author":"Koppel","year":"2011","journal-title":"Language Resources and Evaluation"},{"key":"2025032113505803300_bib77","doi-asserted-by":"publisher","first-page":"737","DOI":"10.18653\/v1\/2020.emnlp-main.55","article-title":"Reformulating unsupervised style transfer as paraphrase generation","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Krishna","year":"2020"},{"key":"2025032113505803300_bib78","article-title":"Towards a human-like open-domain chatbot","author":"Kulshreshtha","year":"2020"},{"key":"2025032113505803300_bib79","doi-asserted-by":"crossref","first-page":"18","DOI":"10.18653\/v1\/2020.lifelongnlp-1.3","article-title":"Data augmentation using pre-trained transformer models","volume-title":"Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems","author":"Kumar","year":"2020"},{"key":"2025032113505803300_bib80","doi-asserted-by":"publisher","first-page":"1169","DOI":"10.18653\/v1\/2021.emnlp-main.89","article-title":"We\u2019ve had this conversation before: A novel approach to measuring dialog similarity","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Lavi","year":"2021"},{"key":"2025032113505803300_bib81","first-page":"1078","article-title":"Adversarial filters of dataset biases","volume-title":"International Conference on Machine Learning","author":"Le Bras","year":"2020"},{"key":"2025032113505803300_bib82","doi-asserted-by":"publisher","first-page":"8424","DOI":"10.18653\/v1\/2022.acl-long.577","article-title":"Deduplicating training data makes language models better","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Lee","year":"2022"},{"key":"2025032113505803300_bib83","article-title":"Textbooks are all you need II: phi-1.5 technical report","author":"Li","year":"2023","journal-title":"arXiv preprint arXiv:2309.05463"},{"key":"2025032113505803300_bib84","doi-asserted-by":"publisher","first-page":"10443","DOI":"10.18653\/v1\/2023.emnlp-main.647","article-title":"Synthetic data generation with large language models for text classification: Potential and limitations","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Li","year":"2023"},{"issue":"1","key":"2025032113505803300_bib85","doi-asserted-by":"publisher","first-page":"140","DOI":"10.1111\/nyas.15007","article-title":"Holistic evaluation of language models","volume":"525","author":"Liang","year":"2022","journal-title":"Annals of the New York Academy of Sciences"},{"key":"2025032113505803300_bib86","doi-asserted-by":"publisher","first-page":"4188","DOI":"10.18653\/v1\/2021.acl-long.323","article-title":"Anonymisation models for text data: State of the art, challenges and future directions","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Lison","year":"2021"},{"key":"2025032113505803300_bib87","doi-asserted-by":"publisher","first-page":"2122","DOI":"10.18653\/v1\/D16-1230","article-title":"How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Liu","year":"2016"},{"key":"2025032113505803300_bib88","first-page":"2170","article-title":"Using context information for dialog act classification in DNN framework","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Liu","year":"2017"},{"key":"2025032113505803300_bib89","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","author":"Liu","year":"2019","journal-title":"ArXiv"},{"key":"2025032113505803300_bib90","doi-asserted-by":"publisher","first-page":"296","DOI":"10.1016\/j.beth.2014.11.002","article-title":"More than reflections: Empathy in motivational interviewing includes language style synchrony between therapist and client","volume":"463","author":"Lord","year":"2015","journal-title":"Behavior Therapy"},{"key":"2025032113505803300_bib91","unstructured":"Louviere, Jordan J. and George G.Woodworth. 1991. Best-worst scaling: A model for the largest difference judgments. Technical report, Working paper."},{"key":"2025032113505803300_bib92","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1075\/ijcl.22.3.02lov","article-title":"The spoken BNC2014: Designing and building a spoken corpus of everyday conversations","volume":"22","author":"Love","year":"2017","journal-title":"International Journal of Corpus Linguistics"},{"key":"2025032113505803300_bib93","article-title":"Teacher\u2019s pet: Understanding and mitigating biases in distillation","author":"Lukasik","year":"2022","journal-title":"Transactions on Machine Learning Research"},{"key":"2025032113505803300_bib94","doi-asserted-by":"publisher","first-page":"2116","DOI":"10.18653\/v1\/2021.naacl-main.171","article-title":"StylePTB: A compositional benchmark for fine-grained controllable text style transfer","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Lyu","year":"2021"},{"issue":"4","key":"2025032113505803300_bib95","doi-asserted-by":"publisher","first-page":"54","DOI":"10.2478\/popets-2019-0058","article-title":"A girl has no name: Automated authorship obfuscation using Mutant-X","volume":"2019","author":"Mahmood","year":"2019","journal-title":"Proceedings on Privacy Enhancing Technologies"},{"key":"2025032113505803300_bib96","article-title":"When less is more: Investigating data pruning for pretraining LLMs at scale","author":"Marion","year":"2023","journal-title":"arXiv preprint arXiv:2309.04564"},{"key":"2025032113505803300_bib97","doi-asserted-by":"publisher","first-page":"4860","DOI":"10.18653\/v1\/2022.emnlp-main.323","article-title":"Differentially private language models for secure data sharing","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Mattern","year":"2022"},{"key":"2025032113505803300_bib98","doi-asserted-by":"publisher","first-page":"11330","DOI":"10.18653\/v1\/2023.findings-acl.719","article-title":"Membership inference attacks against language models via neighbourhood comparison","volume-title":"Findings of the Association for Computational Linguistics: ACL 2023","author":"Mattern","year":"2023"},{"key":"2025032113505803300_bib99","doi-asserted-by":"publisher","first-page":"1906","DOI":"10.18653\/v1\/2020.acl-main.173","article-title":"On faithfulness and factuality in abstractive summarization","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Maynez","year":"2020"},{"key":"2025032113505803300_bib100","doi-asserted-by":"publisher","first-page":"35","DOI":"10.18653\/v1\/W19-1905","article-title":"Towards automatic generation of shareable synthetic clinical notes using neural language models","volume-title":"Proceedings of the 2nd Clinical Natural Language Processing Workshop","author":"Melamud","year":"2019"},{"key":"2025032113505803300_bib101","doi-asserted-by":"publisher","first-page":"312","DOI":"10.18653\/v1\/P18-2050","article-title":"Extreme adaptation for personalized neural machine translation","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Michel","year":"2018"},{"key":"2025032113505803300_bib102","doi-asserted-by":"publisher","first-page":"2009","DOI":"10.18653\/v1\/2021.emnlp-main.152","article-title":"Style pooling: Automatic text style obfuscation for improved classification fairness","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Mireshghallah","year":"2021"},{"key":"2025032113505803300_bib103","doi-asserted-by":"publisher","first-page":"8770","DOI":"10.18653\/v1\/2022.emnlp-main.600","article-title":"ArtELingo: A million emotion annotations of WikiArt with emphasis on diversity over language and culture","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Mohamed","year":"2022"},{"key":"2025032113505803300_bib104","article-title":"Is a prompt and a few samples all you need? Using GPT-4 for data augmentation in low-resource classification tasks","author":"M\u00f8ller","year":"2023","journal-title":"arXiv preprint arXiv:2304.13861"},{"key":"2025032113505803300_bib105","doi-asserted-by":"publisher","first-page":"1640","DOI":"10.18653\/v1\/2023.eacl-main.120","article-title":"Modelling temporal document sequences for clinical ICD coding","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Ng","year":"2023"},{"issue":"3","key":"2025032113505803300_bib106","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1162\/COLI_a_00258","article-title":"Computational sociolinguistics: A survey","volume":"42","author":"Nguyen","year":"2016","journal-title":"Computational Linguistics"},{"key":"2025032113505803300_bib107","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1162\/tacl_a_00027","article-title":"Polite dialogue generation without parallel data","volume":"6","author":"Niu","year":"2018","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2025032113505803300_bib108","doi-asserted-by":"publisher","first-page":"5136","DOI":"10.18653\/v1\/2021.emnlp-main.417","article-title":"Unsupervised paraphrasing with pretrained language models","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Niu","year":"2021"},{"key":"2025032113505803300_bib109","doi-asserted-by":"publisher","first-page":"13","DOI":"10.3389\/fdata.2019.00013","article-title":"Social data: Biases, methodological pitfalls, and ethical boundaries","volume":"2","author":"Olteanu","year":"2019","journal-title":"Frontiers in Big Data"},{"key":"2025032113505803300_bib110","doi-asserted-by":"publisher","first-page":"311","DOI":"10.3115\/1073083.1073135","article-title":"Bleu: a method for automatic evaluation of machine translation","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni","year":"2002"},{"key":"2025032113505803300_bib111","article-title":"Overview of the 5th Author Profiling task at PAN 2017: Gender and language variety identification in Twitter","volume-title":"CLEF","author":"Pardo","year":"2017"},{"key":"2025032113505803300_bib112","doi-asserted-by":"crossref","DOI":"10.1145\/3526113.3545616","article-title":"Social simulacra: Creating populated prototypes for social computing systems","volume-title":"Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology","author":"Park","year":"2022"},{"key":"2025032113505803300_bib113","doi-asserted-by":"publisher","first-page":"267","DOI":"10.3115\/1067807.1067843","article-title":"Language independent authorship attribution with character level n-grams","volume-title":"10th Conference of the European Chapter of the Association for Computational Linguistics","author":"Peng","year":"2003"},{"key":"2025032113505803300_bib114","doi-asserted-by":"publisher","first-page":"1296","DOI":"10.1037\/0022-3514.77.6.1296","article-title":"Linguistic styles: Language use as an individual difference","volume":"77","author":"Pennebaker","year":"1999","journal-title":"Journal of Personality and Social Psychology"},{"key":"2025032113505803300_bib115","doi-asserted-by":"publisher","first-page":"392","DOI":"10.18653\/v1\/W15-3049","article-title":"chrF: character n-gram F-score for automatic MT evaluation","volume-title":"Proceedings of the Tenth Workshop on Statistical Machine Translation","author":"Popovi\u0107","year":"2015"},{"key":"2025032113505803300_bib116","article-title":"Author obfuscation: Attacking the state of the art in authorship verification","volume-title":"Conference and Labs of the Evaluation Forum","author":"Potthast","year":"2016"},{"key":"2025032113505803300_bib117","doi-asserted-by":"publisher","first-page":"1754","DOI":"10.3115\/v1\/P15-1169","article-title":"An analysis of the user occupational class through Twitter content","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Preo\u0163iuc-Pietro","year":"2015"},{"key":"2025032113505803300_bib118","doi-asserted-by":"publisher","first-page":"9496","DOI":"10.18653\/v1\/2022.emnlp-main.646","article-title":"Perturbation augmentation for fairer NLP","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Qian","year":"2022"},{"key":"2025032113505803300_bib119","doi-asserted-by":"publisher","first-page":"1339","DOI":"10.18653\/v1\/2023.emnlp-main.85","article-title":"Is ChatGPT a general-purpose natural language processing task solver?","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Qin","year":"2023"},{"key":"2025032113505803300_bib120","doi-asserted-by":"publisher","first-page":"5596","DOI":"10.1109\/ICASSP.2011.5947628","article-title":"Simultaneous dialog act segmentation and classification from human-human spoken conversations","author":"Quarteroni","year":"2011","journal-title":"2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)"},{"key":"2025032113505803300_bib121","doi-asserted-by":"publisher","first-page":"1074","DOI":"10.18653\/v1\/E17-1101","article-title":"Personalized machine translation: Preserving original author traits","volume-title":"Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers","author":"Rabinovich","year":"2017"},{"key":"2025032113505803300_bib122","article-title":"Scaling language models: Methods, analysis & insights from training gopher","author":"Rae","year":"2021","journal-title":"CoRR"},{"issue":"4","key":"2025032113505803300_bib123","doi-asserted-by":"publisher","first-page":"777","DOI":"10.1162\/coli_a_00486","article-title":"Measuring attribution in natural language generation models","volume":"49","author":"Rashkin","year":"2023","journal-title":"Computational Linguistics"},{"key":"2025032113505803300_bib124","doi-asserted-by":"publisher","first-page":"17","DOI":"10.18653\/v1\/W16-5603","article-title":"Obfuscating gender in social media writing","volume-title":"Proceedings of the First Workshop on NLP and Computational Social Science","author":"Reddy","year":"2016"},{"issue":"3","key":"2025032113505803300_bib125","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1162\/coli_a_00322","article-title":"A structured review of the validity of BLEU","volume":"44","author":"Reiter","year":"2018","journal-title":"Computational Linguistics"},{"issue":"1","key":"2025032113505803300_bib126","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1017\/S1351324997001502","article-title":"Building applied natural language generation systems","volume":"3","author":"Reiter","year":"1997","journal-title":"Natural Language Engineering"},{"key":"2025032113505803300_bib127","doi-asserted-by":"publisher","first-page":"913","DOI":"10.18653\/v1\/2021.emnlp-main.70","article-title":"Learning universal authorship representations","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Rivera-Soto","year":"2021"},{"key":"2025032113505803300_bib128","first-page":"13","article-title":"Evaluating story generation systems using automated linguistic analyses","volume-title":"SIGKDD 2017 Workshop on Machine Learning for Creativity","author":"Roemmele","year":"2017"},{"key":"2025032113505803300_bib129","first-page":"763","article-title":"Age prediction in blogs: A study of style, content, and online behavior in pre- and post-social media generations","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Rosenthal","year":"2011"},{"key":"2025032113505803300_bib130","doi-asserted-by":"publisher","first-page":"7219","DOI":"10.18653\/v1\/2021.emnlp-main.575","article-title":"Perturbation CheckLists for evaluating NLG evaluation metrics","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Sai","year":"2021"},{"key":"2025032113505803300_bib131","doi-asserted-by":"publisher","first-page":"189","DOI":"10.18653\/v1\/P17-2030","article-title":"Error-repair dependency parsing for ungrammatical texts","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Sakaguchi","year":"2017"},{"key":"2025032113505803300_bib132","article-title":"DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter","author":"Sanh","year":"2019","journal-title":"arXiv preprint arXiv:1910.01108"},{"key":"2025032113505803300_bib133","first-page":"29971","article-title":"Whose opinions do language models reflect?","volume-title":"International Conference on Machine Learning","author":"Santurkar","year":"2023"},{"key":"2025032113505803300_bib134","doi-asserted-by":"publisher","first-page":"1668","DOI":"10.18653\/v1\/P19-1163","article-title":"The risk of racial bias in hate speech detection","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Sap","year":"2019"},{"key":"2025032113505803300_bib135","doi-asserted-by":"publisher","first-page":"93","DOI":"10.3115\/v1\/N15-1010","article-title":"Not all character n-grams are created equal: A study in authorship attribution","volume-title":"Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Sapkota","year":"2015"},{"key":"2025032113505803300_bib136","first-page":"199","article-title":"Effects of age and gender on blogging","volume-title":"AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs","author":"Schler","year":"2006"},{"key":"2025032113505803300_bib137","doi-asserted-by":"publisher","first-page":"6594","DOI":"10.18653\/v1\/2021.emnlp-main.529","article-title":"QuestEval: Summarization asks for fact-based evaluation","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Scialom","year":"2021"},{"key":"2025032113505803300_bib138","doi-asserted-by":"publisher","first-page":"1702","DOI":"10.18653\/v1\/N19-1170","article-title":"What makes a good conversation? How controllable attributes affect human judgments","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"See","year":"2019"},{"key":"2025032113505803300_bib139","doi-asserted-by":"publisher","first-page":"7881","DOI":"10.18653\/v1\/2020.acl-main.704","article-title":"BLEURT: Learning robust metrics for text generation","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Sellam","year":"2020"},{"key":"2025032113505803300_bib140","doi-asserted-by":"publisher","first-page":"5248","DOI":"10.18653\/v1\/2020.acl-main.468","article-title":"Predictive biases in natural language processing models: A conceptual framework and overview","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Shah","year":"2020"},{"key":"2025032113505803300_bib141","doi-asserted-by":"publisher","first-page":"4275","DOI":"10.18653\/v1\/2021.acl-long.330","article-title":"Societal biases in language generation: Progress and challenges","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Sheng","year":"2021"},{"key":"2025032113505803300_bib142","first-page":"1633","article-title":"A4NT: Author attribute anonymity by adversarial training of neural machine translation","volume-title":"27th USENIX Security Symposium (USENIX Security 18)","author":"Shetty","year":"2018"},{"key":"2025032113505803300_bib143","article-title":"The curse of recursion: Training on generated data makes models forget","author":"Shumailov","year":"2023","journal-title":"arXiv preprint arXiv:2305.17493"},{"key":"2025032113505803300_bib144","first-page":"223","article-title":"A study of translation edit rate with targeted human annotation","volume-title":"Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers","author":"Snover","year":"2006"},{"key":"2025032113505803300_bib145","doi-asserted-by":"publisher","first-page":"254","DOI":"10.3115\/1613715.1613751","article-title":"Cheap and fast \u2013 but is it good? Evaluating non-expert annotations for natural language tasks","volume-title":"Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing","author":"Snow","year":"2008"},{"key":"2025032113505803300_bib146","first-page":"95","article-title":"Beyond the imitation game: Quantifying and extrapolating the capabilities of language models","author":"Srivastava","year":"2023","journal-title":"Transactions on Machine Learning Research"},{"key":"2025032113505803300_bib147","article-title":"Beyond memorization: Violating privacy via inference with large language models","volume-title":"Twelfth International Conference on Learning Representations","author":"Staab","year":"2024"},{"key":"2025032113505803300_bib148","doi-asserted-by":"publisher","first-page":"518","DOI":"10.1007\/978-3-319-24027-5_49","article-title":"Overview of the PAN\/CLEF 2015 evaluation lab","volume-title":"Proceedings of the 6th International Conference on Experimental IR Meets Multilinguality, Multimodality, and Interaction - Volume 9283","author":"Stamatatos","year":"2015"},{"key":"2025032113505803300_bib149","doi-asserted-by":"publisher","first-page":"1102","DOI":"10.18653\/v1\/2023.findings-acl.71","article-title":"One embedder, any task: Instruction-finetuned text embeddings","volume-title":"Findings of the Association for Computational Linguistics: ACL 2023","author":"Su","year":"2023"},{"key":"2025032113505803300_bib150","doi-asserted-by":"publisher","first-page":"5176","DOI":"10.18653\/v1\/2021.emnlp-main.420","article-title":"AESOP: Paraphrase generation with adaptive syntactic control","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Sun","year":"2021"},{"key":"2025032113505803300_bib151","doi-asserted-by":"publisher","first-page":"9275","DOI":"10.18653\/v1\/2020.emnlp-main.746","article-title":"Dataset cartography: Mapping and diagnosing datasets with training dynamics","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Swayamdipta","year":"2020"},{"issue":"5","key":"2025032113505803300_bib152","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1142\/S0218488502001648","article-title":"k-anonymity: A model for protecting privacy","volume":"10","author":"Sweeney","year":"2002","journal-title":"International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems"},{"key":"2025032113505803300_bib153","doi-asserted-by":"publisher","first-page":"2920","DOI":"10.18653\/v1\/2020.acl-main.263","article-title":"It\u2019s morphin\u2019 time! Combating linguistic discrimination with inflectional perturbations","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Tan","year":"2020"},{"key":"2025032113505803300_bib154","doi-asserted-by":"publisher","first-page":"3450","DOI":"10.18653\/v1\/2021.findings-acl.304","article-title":"Multilingual translation from denoising pre-training","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Tang","year":"2021"},{"key":"2025032113505803300_bib155","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/IISA52424.2021.9555521","article-title":"Differentially private synthetic mixed-type data generation for unsupervised learning","volume-title":"2021 12th International Conference on Information, Intelligence, Systems & Applications (IISA)","author":"Tantipongpipat","year":"2021"},{"key":"2025032113505803300_bib156","first-page":"44","article-title":"The sensitivity of annotator bias to task definitions in argument mining","volume-title":"Proceedings of the 16th Linguistic Annotation Workshop (LAW-XVI) within LREC2022","author":"Thorn Jakobsen","year":"2022"},{"key":"2025032113505803300_bib157","article-title":"Llama 2: Open foundation and fine-tuned chat models","author":"Touvron","year":"2023","journal-title":"arXiv preprint arXiv:2307.09288"},{"key":"2025032113505803300_bib158","doi-asserted-by":"publisher","first-page":"4647","DOI":"10.18653\/v1\/2022.acl-long.318","article-title":"Identifying moments of change from longitudinal user text","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Tsakalidis","year":"2022"},{"key":"2025032113505803300_bib159","article-title":"Beyond privacy: Navigating the opportunities and challenges of synthetic data","author":"van Breugel","year":"2023","journal-title":"arXiv preprint arXiv:2304.03722"},{"key":"2025032113505803300_bib160","doi-asserted-by":"publisher","first-page":"140","DOI":"10.18653\/v1\/2021.inlg-1.14","article-title":"Underreporting of errors in NLG output, and what to do about it","volume-title":"Proceedings of the 14th International Conference on Natural Language Generation","author":"van Miltenburg","year":"2021"},{"key":"2025032113505803300_bib161","article-title":"Generating faithful synthetic data with large language models: A case study in computational social science","volume":"abs\/2305.15041","author":"Veselovsky","year":"2023","journal-title":"ArXiv"},{"issue":"3","key":"2025032113505803300_bib162","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1093\/jamia\/ocx079","article-title":"Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record","volume":"25","author":"Walonoski","year":"2018","journal-title":"Journal of the American Medical Informatics Association"},{"key":"2025032113505803300_bib163","article-title":"Large language models cannot replace human participants because they cannot portray identity groups","author":"Wang","year":"2024","journal-title":"arXiv preprint arXiv:2402.01908"},{"key":"2025032113505803300_bib164","first-page":"10882","article-title":"Directional bias amplification","volume-title":"International Conference on Machine Learning","author":"Wang","year":"2021"},{"key":"2025032113505803300_bib165","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1162\/tacl_a_00113","article-title":"The galactic dependencies treebanks: Getting more data by synthesizing new languages","volume":"4","author":"Wang","year":"2016","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2025032113505803300_bib166","doi-asserted-by":"publisher","first-page":"1193","DOI":"10.18653\/v1\/2021.naacl-main.94","article-title":"Towards modeling the style of translators in neural machine translation","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Wang","year":"2021"},{"key":"2025032113505803300_bib167","article-title":"Self-instruct: Aligning language model with self generated instructions","author":"Wang","year":"2022","journal-title":"arXiv preprint arXiv:2212.10560"},{"key":"2025032113505803300_bib168","doi-asserted-by":"publisher","first-page":"240","DOI":"10.18653\/v1\/W19-5026","article-title":"Is artificial data useful for biomedical natural language processing algorithms?","volume-title":"Proceedings of the 18th BioNLP Workshop and Shared Task","author":"Wang","year":"2019"},{"key":"2025032113505803300_bib169","doi-asserted-by":"publisher","first-page":"33","DOI":"10.18653\/v1\/D18-1004","article-title":"It\u2019s going to be okay: Measuring access to support in online communities","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Wang","year":"2018"},{"issue":"1","key":"2025032113505803300_bib170","doi-asserted-by":"publisher","first-page":"1075","DOI":"10.1609\/icwsm.v16i1.19359","article-title":"Using authorship verification to mitigate abuse in online communities","volume":"16","author":"Weerasinghe","year":"2022","journal-title":"Proceedings of the International AAAI Conference on Web and Social Media"},{"key":"2025032113505803300_bib171","doi-asserted-by":"publisher","first-page":"7109","DOI":"10.18653\/v1\/2021.emnlp-main.569","article-title":"Does it capture STEL? A modular, similarity-based linguistic style evaluation framework","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Wegmann","year":"2021"},{"key":"2025032113505803300_bib172","doi-asserted-by":"publisher","first-page":"1121","DOI":"10.1145\/1553374.1553517","article-title":"Herding dynamical weights to learn","volume-title":"Proceedings of the 26th Annual International Conference on Machine Learning","author":"Welling","year":"2009"},{"issue":"1","key":"2025032113505803300_bib173","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1017\/S104909651500116X","article-title":"On the ethics of crowdsourced research","volume":"49","author":"Williamson","year":"2016","journal-title":"PS: Political Science 38; Politics"},{"key":"2025032113505803300_bib174","doi-asserted-by":"publisher","first-page":"4699","DOI":"10.18653\/v1\/2021.findings-emnlp.402","article-title":"Fight fire with fire: Fine-tuning hate detectors using large samples of generated hate speech","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Wullach","year":"2021"},{"key":"2025032113505803300_bib175","doi-asserted-by":"publisher","first-page":"2113","DOI":"10.1145\/3630106.3659029","article-title":"Fairness feedback loops: Training on synthetic data amplifies bias","volume-title":"Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency","author":"Wyllie","year":"2024"},{"key":"2025032113505803300_bib176","doi-asserted-by":"crossref","first-page":"5786","DOI":"10.18653\/v1\/P19-1579","article-title":"Generalized data augmentation for low-resource translation","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Xia","year":"2019"},{"key":"2025032113505803300_bib177","doi-asserted-by":"publisher","first-page":"2192","DOI":"10.18653\/v1\/P19-1579","article-title":"Assessing dialogue systems with distribution distances","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Xiang","year":"2021"},{"key":"2025032113505803300_bib178","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-acl.193","article-title":"Modeling tabular data using conditional GAN","volume":"32","author":"Xu","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2025032113505803300_bib179","doi-asserted-by":"publisher","first-page":"6567","DOI":"10.18653\/v1\/2020.emnlp-main.532","article-title":"Personal information leakage detection in conversations","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Xu","year":"2020"},{"key":"2025032113505803300_bib180","doi-asserted-by":"publisher","first-page":"247","DOI":"10.18653\/v1\/W19-8633","article-title":"Privacy-aware text rewriting","volume-title":"Proceedings of the 12th International Conference on Natural Language Generation","author":"Xu","year":"2019"},{"key":"2025032113505803300_bib181","first-page":"2899","article-title":"Paraphrasing for style","volume-title":"Proceedings of COLING 2012","author":"Xu","year":"2012"},{"key":"2025032113505803300_bib182","article-title":"Magpie: Alignment data synthesis from scratch by prompting aligned LLMs with nothing","author":"Xu","year":"2024","journal-title":"arXiv preprint arXiv:2406.08464"},{"key":"2025032113505803300_bib183","first-page":"5753","article-title":"XLNet: Generalized autoregressive pretraining for language understanding","volume":"32","author":"Yang","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2025032113505803300_bib184","doi-asserted-by":"publisher","first-page":"11653","DOI":"10.18653\/v1\/2022.emnlp-main.801","article-title":"ZeroGen: Efficient zero-shot learning via dataset generation","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Ye","year":"2022"},{"key":"2025032113505803300_bib185","doi-asserted-by":"publisher","first-page":"2225","DOI":"10.18653\/v1\/2021.findings-emnlp.192","article-title":"GPT3Mix: Leveraging large-scale language models for text augmentation","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Yoo","year":"2021"},{"key":"2025032113505803300_bib186","first-page":"45","article-title":"Large language model as attributed training data generator: A tale of diversity and bias","volume-title":"Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track","author":"Yu","year":"2023"},{"key":"2025032113505803300_bib187","doi-asserted-by":"publisher","first-page":"2204","DOI":"10.18653\/v1\/P18-1205","article-title":"Personalizing dialogue agents: I have a dog, do you have pets too?","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Zhang","year":"2018"},{"key":"2025032113505803300_bib188","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-5021","article-title":"BERTScore: Evaluating text generation with BERT","volume-title":"International Conference on Learning Representations","author":"Zhang","year":"2020"},{"key":"2025032113505803300_bib189","doi-asserted-by":"publisher","first-page":"2979","DOI":"10.18653\/v1\/2021.emnlp-main.25","article-title":"Men also like shopping: Reducing gender bias amplification using corpus-level constraints","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Zhao","year":"2017"},{"key":"2025032113505803300_bib190","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1145\/3209978.3210080","article-title":"A unified neural architecture for joint dialog act segmentation and recognition in spoken dialog system","volume-title":"Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue","author":"Zhao","year":"2018"},{"key":"2025032113505803300_bib191","doi-asserted-by":"publisher","first-page":"279","DOI":"10.18653\/v1\/2021.emnlp-main.25","article-title":"Idiosyncratic but not arbitrary: Learning idiolects in online registers reveals distinctive yet consistent individual styles","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Zhu","year":"2021"},{"key":"2025032113505803300_bib192","first-page":"1097","article-title":"Texygen: A benchmarking platform for text generation models","volume-title":"The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval","author":"Zhu","year":"2018"},{"key":"2025032113505803300_bib193","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1162\/coli_a_00502","article-title":"Can large language models transform computational social science?","author":"Ziems","year":"2024","journal-title":"Computational Linguistics"},{"key":"2025032113505803300_bib194","doi-asserted-by":"publisher","first-page":"1651","DOI":"10.18653\/v1\/P19-1161","article-title":"Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Zmigrod","year":"2019"},{"key":"2025032113505803300_bib195","doi-asserted-by":"publisher","first-page":"1163","DOI":"10.1162\/tacl_a_00420","article-title":"What helps transformers recognize conversational structure? Importance of context, punctuation, and labels in dialog act recognition","volume":"9","author":"\u017belasko","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/coli\/article-pdf\/51\/1\/191\/2481447\/coli_a_00540.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/coli\/article-pdf\/51\/1\/191\/2481447\/coli_a_00540.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,21]],"date-time":"2025-03-21T14:09:30Z","timestamp":1742566170000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/51\/1\/191\/124625\/Evaluating-Synthetic-Data-Generation-from-User"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"references-count":195,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,3,15]]},"published-print":{"date-parts":[[2025,3,15]]}},"URL":"https:\/\/doi.org\/10.1162\/coli_a_00540","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025]]},"published":{"date-parts":[[2025]]}}}