{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T13:57:15Z","timestamp":1770731835752,"version":"3.49.0"},"reference-count":49,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2025,6,12]],"date-time":"2025-06-12T00:00:00Z","timestamp":1749686400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The distinction between artificial intelligence (AI)- and human-generated texts has become increasingly significant with the emergence of ChatGPT and other generative AI models, which have garnered millions of users. In this study, we assess Burrows\u2019s Delta, a well-established algorithm in authorship attribution, as a potential AI detector in argumentative essays. Our results prove that Burrows\u2019s Delta is an effective tool for detecting AI-generated content. In addition, the accuracy of distinguishing between AI- and human-generated texts exceeds 80 per cent across varying text lengths, from short excerpts approximately 100 words to full essays of approximately 550 words. However, the accuracy is influenced by both text length and the number of features selected by Burrows\u2019s Delta. More importantly, our findings indicate that AI exhibits distinct linguistic and stylistic authorship markers, which can be identified using conventional authorship attribution methods. We discuss these findings in relation to authorship attribution, stylometry, and the role of large language models in language teaching and learning.<\/jats:p>","DOI":"10.1093\/llc\/fqaf048","type":"journal-article","created":{"date-parts":[[2025,6,12]],"date-time":"2025-06-12T16:51:44Z","timestamp":1749747104000},"page":"1064-1075","source":"Crossref","is-referenced-by-count":3,"title":["Detecting authorship between generative AI models and humans: a Burrows\u2019s Delta approach"],"prefix":"10.1093","volume":"40","author":[{"given":"Hongao","family":"Zhu","sequence":"first","affiliation":[{"name":"School of Foreign Languages, Shanghai Jiao Tong University , Shanghai, 200240,","place":["China"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3366-1855","authenticated-orcid":false,"given":"Lei","family":"Lei","sequence":"additional","affiliation":[{"name":"Institute of Language Sciences, Shanghai International Studies University , Shanghai, 201620,","place":["China"]}]}],"member":"286","published-online":{"date-parts":[[2025,6,12]]},"reference":[{"key":"2025090303520417700_fqaf048-B1","doi-asserted-by":"crossref","first-page":"693","DOI":"10.25205\/1995-4328-2022-16-2-693-705","article-title":"Testing Burrows\u2019 Delta on Ancient Greek Authors","volume":"16","author":"Alieva","year":"2022","journal-title":"\u03a3\u03a7\u039f\u039b\u0397. Ancient Philosophy and the Classical Tradition"},{"key":"2025090303520417700_fqaf048-B2","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1093\/llc\/fqn003","article-title":"Interpreting Burrows\u2019s Delta: Geometric and Probabilistic Foundations","volume":"23","author":"Argamon","year":"2007","journal-title":"Literary and Linguistic Computing"},{"key":"2025090303520417700_fqaf048-B3","first-page":"15","author":"Bhatt","year":"2021"},{"key":"2025090303520417700_fqaf048-B4","first-page":"283","author":"Bolea","year":"2020"},{"key":"2025090303520417700_fqaf048-B5","doi-asserted-by":"crossref","first-page":"e32","DOI":"10.1017\/dap.2021.20","article-title":"The Role of Artificial Intelligence in Disinformation","volume":"3","author":"Bontridder","year":"2021","journal-title":"Data & Policy"},{"key":"2025090303520417700_fqaf048-B6","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1093\/llc\/17.3.267","article-title":"\u2018Delta\u2019: A Measure of Stylistic Difference and a Guide to Likely Authorship\u2019,","volume":"17","author":"Burrows","year":"2002","journal-title":"Literary and Linguistic Computing"},{"key":"2025090303520417700_fqaf048-B7","doi-asserted-by":"crossref","first-page":"101426","DOI":"10.1016\/j.xcrp.2023.101426","article-title":"Distinguishing Academic Science Writing from Humans or ChatGPT with over 99% Accuracy Using Off-the-Shelf Machine Learning Tools\u2019,","volume":"4","author":"Desaire","year":"2023","journal-title":"Cell Reports Physical Science"},{"key":"2025090303520417700_fqaf048-B8","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1093\/llc\/fqt039","article-title":"Mind Your Corpus: Systematic Errors in Authorship Attribution\u2019","volume":"28","author":"Eder","year":"2013","journal-title":"Literary and Linguistic Computing"},{"key":"2025090303520417700_fqaf048-B9","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1093\/llc\/fqt066","article-title":"Does Size Matter? Authorship Attribution, Small Samples, Big Problem","volume":"30","author":"Eder","year":"2015","journal-title":"Digital Scholarship in the Humanities"},{"key":"2025090303520417700_fqaf048-B10","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1080\/09296174.2022.2122751","article-title":"Stylistic Fingerprints, POS-Tags and Inflected Languages: A Case Study in Polish","volume":"30","author":"Eder","year":"2023","journal-title":"Journal of Quantitative Linguistics"},{"key":"2025090303520417700_fqaf048-B11","doi-asserted-by":"crossref","first-page":"ii4","DOI":"10.1093\/llc\/fqx023","article-title":"Understanding and Explaining Delta Measures for Authorship Attribution","volume":"32","author":"Evert","year":"2017","journal-title":"Digital Scholarship in the Humanities"},{"key":"2025090303520417700_fqaf048-B12","doi-asserted-by":"crossref","first-page":"i","DOI":"10.1002\/j.2333-8504.1998.tb01791.x","article-title":"Computer Analysis of The TOFEL Test of Written English","volume":"1998","author":"Frase","year":"1998","journal-title":"ETS Research Report Series"},{"key":"2025090303520417700_fqaf048-B13","doi-asserted-by":"crossref","first-page":"e443","DOI":"10.7717\/peerj-cs.443","article-title":"Feature-Based Detection of Automated Language Models: Tackling GPT-2, GPT-3 and Grover\u2019,","volume":"7","author":"Fr\u00f6hling","year":"2021","journal-title":"PeerJ Computer Science"},{"key":"2025090303520417700_fqaf048-B14","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1093\/llc\/fqv026","article-title":"Stylometry and Collaborative Authorship: Eddy, Lovecraft, and \u2018The Loved Dead\u2019","volume":"32","author":"Gladwin","year":"2017","journal-title":"Digital Scholarship in the Humanities"},{"key":"2025090303520417700_fqaf048-B15","first-page":"15","author":"Guo","year":"2023"},{"key":"2025090303520417700_fqaf048-B16","doi-asserted-by":"crossref","first-page":"18617","DOI":"10.1038\/s41598-023-45644-9","article-title":"A Large-Scale Comparison of Human-Written versus ChatGPT-Generated Essays\u2019,","volume":"13","author":"Herbold","year":"2023","journal-title":"Scientific Reports"},{"key":"2025090303520417700_fqaf048-B17","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1093\/llc\/19.4.453","article-title":"Testing Burrows\u2019s Delta","volume":"19","author":"Hoover","year":"2004","journal-title":"Literary and Linguistic Computing"},{"key":"2025090303520417700_fqaf048-B18","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1093\/llc\/fqad079","article-title":"Methodological Observations Concerning Word Rankings and z -Score Refinements","volume":"39","author":"Ilsemann","year":"2024","journal-title":"Digital Scholarship in the Humanities"},{"key":"2025090303520417700_fqaf048-B19","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1126\/science.aaa8415","article-title":"Machine Learning: Trends, Perspectives, and Prospects\u2019,","volume":"349","author":"Jordan","year":"2015","journal-title":"Science"},{"key":"2025090303520417700_fqaf048-B20","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1561\/1500000005","article-title":"Authorship Attribution","volume":"1","author":"Juola","year":"2007","journal-title":"Foundations and Trends\u00ae in Information Retrieval"},{"key":"2025090303520417700_fqaf048-B21","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1093\/llc\/fqt063","article-title":"Collaborative Authorship in the Twelfth Century: A Stylometric Study of Hildegard of Bingen and Guibert of Gembloux","volume":"30","author":"Kestemont","year":"2015","journal-title":"Digital Scholarship in the Humanities"},{"key":"2025090303520417700_fqaf048-B22","doi-asserted-by":"crossref","first-page":"e48904","DOI":"10.2196\/48904","article-title":"Differentiate ChatGPT-Generated and Human-Written Medical Texts","volume":"9","author":"Liao","year":"2023","journal-title":"JMIR Medical Education"},{"key":"2025090303520417700_fqaf048-B23","first-page":"15","author":"Liu","year":"2023"},{"key":"2025090303520417700_fqaf048-B24","author":"Liu","year":"2024"},{"key":"2025090303520417700_fqaf048-B25","first-page":"24950","author":"Mitchell","year":"2023"},{"key":"2025090303520417700_fqaf048-B26","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1192\/bjp.2023.136","article-title":"Artificial Intelligence and Increasing Misinformation\u2019,","volume":"224","author":"Monteith","year":"2024","journal-title":"The British Journal of Psychiatry"},{"key":"2025090303520417700_fqaf048-B27","doi-asserted-by":"crossref","DOI":"10.1075\/z.208","volume-title":"Making and Using Word Lists for Language Learning and Testing","author":"Nation","year":"2016"},{"key":"2025090303520417700_fqaf048-B28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3597307","article-title":"Biases in Large Language Models: Origins, Inventory, and Discussion","volume":"15","author":"Navigli","year":"2023","journal-title":"Journal of Data and Information Quality"},{"key":"2025090303520417700_fqaf048-B29","first-page":"31","author":"OpenAI","year":"2023"},{"key":"2025090303520417700_fqaf048-B30","author":"O'Sullivan","year":"2024"},{"key":"2025090303520417700_fqaf048-B31","first-page":"2825","article-title":"Scikit-Learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"The Journal of Machine Learning Research"},{"key":"2025090303520417700_fqaf048-B32","author":"Proisl","year":"2018"},{"key":"2025090303520417700_fqaf048-B33","first-page":"3","article-title":"Style as Choice and Deviation","volume":"6","author":"Rosengren","year":"1972","journal-title":"Style"},{"key":"2025090303520417700_fqaf048-B34","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1093\/llc\/fqr031","article-title":"Deeper Delta across Genres and Languages: Do We Really Need the Most Frequent Words?\u2019,","volume":"26","author":"Rybicki","year":"2011","journal-title":"Literary and Linguistic Computing"},{"key":"2025090303520417700_fqaf048-B35","first-page":"669","author":"Shrestha","year":"2017"},{"key":"2025090303520417700_fqaf048-B36","first-page":"75","article-title":"The Search for Units of Meaning","volume":"1","author":"Sinclair","year":"1996","journal-title":"Textus"},{"key":"2025090303520417700_fqaf048-B37","author":"Singh","year":"2024"},{"key":"2025090303520417700_fqaf048-B38","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1093\/llc\/fqad012","article-title":"Hacking Stylometry with Multiple Voices: Imaginary Writers Can Override Authorial Signal in Delta","volume":"38","author":"Skorinkin","year":"2023","journal-title":"Digital Scholarship in the Humanities"},{"key":"2025090303520417700_fqaf048-B3400293","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1080\/09296174.2011.533591","article-title":"Improving Authorship Attribution: Optimizing Burrows' Delta Method*","volume":"18","author":"Smith,","journal-title":"Journal of Quantitative Linguistics"},{"key":"2025090303520417700_fqaf048-B39","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1002\/asi.21001","article-title":"A Survey of Modern Authorship Attribution Methods","volume":"60","author":"Stamatatos","year":"2009","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"2025090303520417700_fqaf048-B40","author":"Stamatatos","year":"2015"},{"key":"2025090303520417700_fqaf048-B41","author":"Tang","year":"2023"},{"key":"2025090303520417700_fqaf048-B42","first-page":"8384","author":"Uchendu","year":"2020"},{"key":"2025090303520417700_fqaf048-B43","author":"Vareilles","year":"2022"},{"key":"2025090303520417700_fqaf048-B44","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python\u2019,","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nature Methods"},{"key":"2025090303520417700_fqaf048-B45","author":"Warstadt","year":"2024"},{"key":"2025090303520417700_fqaf048-B46","author":"Xiao","year":"2023"},{"key":"2025090303520417700_fqaf048-B47","doi-asserted-by":"crossref","DOI":"10.1561\/116.00000250","article-title":"Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text","volume":"13","author":"Yang","year":"2024","journal-title":"APSIPA Transactions on Signal and Information Processing"},{"key":"2025090303520417700_fqaf048-B48","article-title":"Instruction Tuning for Large Language Models: A Survey\u2019, arXiv","author":"Zhang","year":"2024"}],"container-title":["Digital Scholarship in the Humanities"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/dsh\/article-pdf\/40\/3\/1064\/63480542\/fqaf048.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/dsh\/article-pdf\/40\/3\/1064\/63480542\/fqaf048.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T07:52:16Z","timestamp":1756885936000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/dsh\/article\/40\/3\/1064\/8161203"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,12]]},"references-count":49,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,6,12]]},"published-print":{"date-parts":[[2025,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/llc\/fqaf048","relation":{},"ISSN":["2055-7671","2055-768X"],"issn-type":[{"value":"2055-7671","type":"print"},{"value":"2055-768X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,9]]},"published":{"date-parts":[[2025,6,12]]}}}