{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T03:19:29Z","timestamp":1777778369692,"version":"3.51.4"},"reference-count":102,"publisher":"SAGE Publications","issue":"3","license":[{"start":{"date-parts":[[2024,4,7]],"date-time":"2024-04-07T00:00:00Z","timestamp":1712448000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62132017, 62302435"],"award-info":[{"award-number":["62132017, 62302435"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["LD24F020011"],"award-info":[{"award-number":["LD24F020011"]}]},{"name":"\u201cPioneer\u201d and \u201cLeading Goose\u201d R&D Program of Zhejiang","award":["2024C01167"],"award-info":[{"award-number":["2024C01167"]}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Information Visualization"],"published-print":{"date-parts":[[2024,7]]},"abstract":"<jats:p>Large language models (LLMs) have gained popularity in various fields for their exceptional capability of generating human-like text. Their potential misuse has raised social concerns about plagiarism in academic contexts. However, effective artificial scientific text detection is a non-trivial task due to several challenges, including (1) the lack of a clear understanding of the differences between machine-generated and human-written scientific text, (2) the poor generalization performance of existing methods caused by out-of-distribution issues, and (3) the limited support for human-machine collaboration with sufficient interpretability during the detection process. In this paper, we first identify the critical distinctions between machine-generated and human-written scientific text through a quantitative experiment. Then, we propose a mixed-initiative workflow that combines human experts\u2019 prior knowledge with machine intelligence, along with a visual analytics system to facilitate efficient and trustworthy scientific text detection. Finally, we demonstrate the effectiveness of our approach through two case studies and a controlled user study. We also provide design implications for interactive artificial text detection tools in high-stakes decision-making scenarios.<\/jats:p>","DOI":"10.1177\/14738716241240156","type":"journal-article","created":{"date-parts":[[2024,4,8]],"date-time":"2024-04-08T00:47:45Z","timestamp":1712537265000},"page":"272-291","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":3,"title":["Towards an understanding and explanation for mixed-initiative artificial scientific text detection"],"prefix":"10.1177","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4350-9141","authenticated-orcid":false,"given":"Luoxuan","family":"Weng","sequence":"first","affiliation":[{"name":"State Key Lab of CAD&CG, Zhejiang University, Hangzhou, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shi","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Lab of CAD&CG, Zhejiang University, Hangzhou, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hang","family":"Zhu","sequence":"additional","affiliation":[{"name":"State Key Lab of CAD&CG, Zhejiang University, Hangzhou, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiashun","family":"Sun","sequence":"additional","affiliation":[{"name":"State Key Lab of CAD&CG, Zhejiang University, Hangzhou, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wong","family":"Kam-Kwai","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongming","family":"Han","sequence":"additional","affiliation":[{"name":"State Key Lab of CAD&CG, Zhejiang University, Hangzhou, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minfeng","family":"Zhu","sequence":"additional","affiliation":[{"name":"State Key Lab of CAD&CG, Zhejiang University, Hangzhou, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Chen","sequence":"additional","affiliation":[{"name":"State Key Lab of CAD&CG, Zhejiang University, Hangzhou, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2024,4,7]]},"reference":[{"key":"bibr1-14738716241240156","author":"Crothers E","year":"2022","journal-title":"arXiv preprint arXiv"},{"key":"bibr2-14738716241240156","author":"Xie T","year":"2023","journal-title":"arXiv preprint arXiv"},{"key":"bibr3-14738716241240156","unstructured":"OpenAI. Chatgpt, https:\/\/chat.openai.com\/ (2023, accessed 20 March 2024)."},{"issue":"1","key":"bibr4-14738716241240156","first-page":"1146","volume":"29","author":"Strobelt H","year":"2023","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"bibr5-14738716241240156","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.09036"},{"key":"bibr6-14738716241240156","unstructured":"Zellers R, Holtzman A, Rashkin H, et al. Defending against neural fake news. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc, https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/3e9f0fc9b2f89e043bc6233994dfcf76-Paper.pdf."},{"key":"bibr7-14738716241240156","unstructured":"Pagnoni A, Graciarena M, Tsvetkov Y. Threat scenarios and best practices to detect neural fake news. In Proceedings of the 29th International Conference on Computational Linguistics. Gyeongju, Republic of Korea, pp. 1233\u20131249, https:\/\/aclanthology.org\/2022.coling-1.106."},{"key":"bibr8-14738716241240156","doi-asserted-by":"publisher","DOI":"10.24251\/HICSS.2022.184"},{"key":"bibr9-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2022.103006"},{"key":"bibr10-14738716241240156","author":"Ma Y","year":"2023","journal-title":"arXiv preprint arXiv"},{"key":"bibr11-14738716241240156","first-page":"1213","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Rodriguez J"},{"key":"bibr12-14738716241240156","unstructured":"ACL. Acl 2023 policy on ai writing assistance, https:\/\/2023.aclweb.org\/blog\/ACL-2023-policy\/ (2023, accessed 20 March 2024)."},{"key":"bibr13-14738716241240156","unstructured":"ICML. Clarification on large language model policy llm, https:\/\/icml.cc\/Conferences\/2023\/llm-policy (2023, accessed 20 March 2024)."},{"key":"bibr14-14738716241240156","unstructured":"GPTZero. Gptzero, https:\/\/gptzero.me\/ (2023, accessed 20 March 2024)."},{"key":"bibr15-14738716241240156","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.443"},{"key":"bibr16-14738716241240156","first-page":"1504","volume-title":"2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)","author":"Nguyen-Son H"},{"key":"bibr17-14738716241240156","author":"Mitrovi\u0107 S","year":"2023","journal-title":"arXiv preprint arXiv"},{"key":"bibr18-14738716241240156","author":"Guo B","year":"2023","journal-title":"arXiv preprint arXiv:230107597"},{"key":"bibr19-14738716241240156","first-page":"7250","author":"Dou Y","journal-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"},{"key":"bibr20-14738716241240156","first-page":"1808","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Ippolito D"},{"key":"bibr21-14738716241240156","author":"Jawahar G","year":"2020","journal-title":"arXiv preprint arXiv"},{"key":"bibr22-14738716241240156","first-page":"111","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations","author":"Gehrmann S"},{"key":"bibr23-14738716241240156","first-page":"1","volume-title":"2022 International Joint Conference on Neural Networks (IJCNN)","author":"Crothers E"},{"key":"bibr24-14738716241240156","author":"Solaiman I","year":"2019","journal-title":"arXiv preprint arXiv:190809203"},{"key":"bibr25-14738716241240156","author":"Holtzman A","year":"2019","journal-title":"arXiv preprint arXiv:190409751"},{"key":"bibr26-14738716241240156","first-page":"19","volume-title":"2023 IEEE Symposium on Security and Privacy (SP)","author":"Pu J"},{"key":"bibr27-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1007\/s41060-021-00299-5"},{"issue":"8","key":"bibr28-14738716241240156","first-page":"9","volume":"1","author":"Radford A","year":"2019","journal-title":"OpenAI blog"},{"key":"bibr29-14738716241240156","unstructured":"Bakhtin A, Gross S, Ott M, et al. Real or fake? learning to discriminate machine from human generated text. arXiv preprint arXiv:190603351, 2019."},{"key":"bibr30-14738716241240156","doi-asserted-by":"crossref","unstructured":"Dugan L, Ippolito D, Kirubarajan A, et al. Real or fake text? investigating human ability to detect boundaries between human-written and machine-generated text. In: The 37th AAAI Conference on Artificial Intelligence (AAAI 2023), Washington DC, USA, p. 104979, https:\/\/par.nsf.gov\/biblio\/10390703.","DOI":"10.1609\/aaai.v37i11.26501"},{"key":"bibr31-14738716241240156","first-page":"7282","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Clark E"},{"key":"bibr32-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2864812"},{"key":"bibr33-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2016.2598831"},{"key":"bibr34-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030418"},{"key":"bibr35-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1145\/3359786"},{"key":"bibr36-14738716241240156","doi-asserted-by":"publisher","DOI":"10.3390\/electronics8080832"},{"key":"bibr37-14738716241240156","first-page":"0210","author":"Do\u0161ilovi\u0107 F","journal-title":"2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO)"},{"key":"bibr38-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2870052"},{"key":"bibr39-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2843369"},{"key":"bibr40-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1007\/s41095-020-0191-7"},{"key":"bibr41-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2864838"},{"key":"bibr42-14738716241240156","author":"Yang W","year":"2023","journal-title":"arXiv preprint arXiv"},{"key":"bibr43-14738716241240156","author":"Molnar C","journal-title":"Interpretable machine learning"},{"key":"bibr44-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2019.113141"},{"key":"bibr45-14738716241240156","doi-asserted-by":"publisher","DOI":"10.5120\/ijca2017913660"},{"key":"bibr46-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2893141"},{"key":"bibr47-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2865043"},{"key":"bibr48-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030354"},{"key":"bibr49-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030342"},{"key":"bibr50-14738716241240156","unstructured":"Lundberg SM, Lee SI. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc, https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/8a20a8621978632d76c43dfd28b67767-Paper.pdf."},{"key":"bibr51-14738716241240156","first-page":"1135","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Ribeiro M"},{"key":"bibr52-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11491"},{"key":"bibr53-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2023.3327168"},{"key":"bibr54-14738716241240156","doi-asserted-by":"crossref","unstructured":"Shapley L, et al. A value for n-person games 1953.","DOI":"10.1515\/9781400881970-018"},{"key":"bibr55-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114836"},{"key":"bibr56-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2022.3146806"},{"key":"bibr57-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114864"},{"key":"bibr58-14738716241240156","first-page":"159","volume-title":"Proceedings of the SIGCHI conference on Human Factors in Computing Systems","author":"Horvitz E"},{"key":"bibr59-14738716241240156","first-page":"9","volume-title":"2015 IEEE conference on visual analytics science and technology (VAST)","author":"Cook K"},{"key":"bibr60-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14555"},{"key":"bibr61-14738716241240156","first-page":"40","volume-title":"2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)","author":"Wenskovitch J"},{"key":"bibr62-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2016.2598460"},{"key":"bibr63-14738716241240156","first-page":"121","volume-title":"2021 IEEE Visualization Conference (VIS)","author":"Husain F"},{"key":"bibr64-14738716241240156","first-page":"46","volume-title":"2019 IEEE Conference on Visual Analytics Science and Technology (VAST)","author":"Cabrera \u00c1A"},{"key":"bibr65-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1007\/s12650-021-00780-0"},{"issue":"1","key":"bibr66-14738716241240156","doi-asserted-by":"crossref","first-page":"809","DOI":"10.1109\/TVCG.2022.3209445","volume":"29","author":"Zhou J","year":"2023","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"bibr67-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445591"},{"key":"bibr68-14738716241240156","author":"Lu J","year":"2024","journal-title":"arXiv preprint arXiv"},{"key":"bibr69-14738716241240156","author":"Liu Y","year":"2023","journal-title":"arXiv preprint arXiv"},{"issue":"1","key":"bibr70-14738716241240156","first-page":"756","volume":"29","author":"Zhang W","year":"2023","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"bibr71-14738716241240156","author":"Zhang W","year":"2023","journal-title":"arXiv"},{"key":"bibr72-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030370"},{"key":"bibr73-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2023.3245609"},{"key":"bibr74-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2331979"},{"key":"bibr75-14738716241240156","first-page":"57","volume-title":"2019 IEEE Conference on Visual Analytics Science and Technology (VAST)","author":"Xiang S"},{"key":"bibr76-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2023.3345340"},{"key":"bibr77-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2745078"},{"key":"bibr78-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030347"},{"key":"bibr79-14738716241240156","first-page":"153","volume-title":"Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology","author":"Felix C"},{"key":"bibr80-14738716241240156","first-page":"1","volume-title":"Proceedings of the 2019 CHI conference on human factors in computing systems","author":"Choi M"},{"key":"bibr81-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1177\/1071181322661356"},{"key":"bibr82-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2744818"},{"key":"bibr83-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-018-1500-3"},{"key":"bibr84-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1145\/3439333"},{"key":"bibr85-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1145\/3429448"},{"key":"bibr86-14738716241240156","first-page":"167","volume-title":"2023 IEEE 16th Pacific Visualization Symposium (PacificVis)","author":"Zhang X"},{"key":"bibr87-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114820"},{"key":"bibr88-14738716241240156","unstructured":"Grammarly. Grammarly, https:\/\/www.grammarly.com\/ (2023, accessed 20 March 2024)."},{"key":"bibr89-14738716241240156","author":"Gleason JB","year":"2022","journal-title":"The development of language"},{"key":"bibr90-14738716241240156","first-page":"3615","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Beltagy I"},{"key":"bibr91-14738716241240156","unstructured":"Christiano P, Leike J, Brown T, et al. Deep reinforcement learning from human preferences. Advances in neural information processing systems 2017; 30: 4302\u20134310, https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf."},{"key":"bibr92-14738716241240156","author":"Monti F","year":"2019","journal-title":"arXiv preprint arXiv"},{"key":"bibr93-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-019-8208-z"},{"key":"bibr94-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v8i1.7469"},{"key":"bibr95-14738716241240156","author":"McInnes L","year":"2018","journal-title":"arXiv preprint arXiv"},{"key":"bibr96-14738716241240156","unstructured":"Ke G, Meng Q, Finley T, et al. Lightgbm: A highly efficient gradient boosting decision tree. Adv Neural Inf Process Syst 2017: 30, https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf; https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/6449f44a102fde848669bdd9eb6b76fa-Abstract.html"},{"key":"bibr97-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3128157"},{"key":"bibr98-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2021.103502"},{"key":"bibr99-14738716241240156","volume-title":"Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems","author":"Ehsan U"},{"key":"bibr100-14738716241240156","doi-asserted-by":"publisher","DOI":"10.1016\/j.visinf.2021.06.002"},{"key":"bibr101-14738716241240156","author":"Kirchenbauer J","year":"2023","journal-title":"arXiv preprint arXiv"},{"key":"bibr102-14738716241240156","author":"Mitchell E","year":"2023","journal-title":"arXiv preprint arXiv"}],"container-title":["Information Visualization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14738716241240156","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/14738716241240156","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14738716241240156","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T19:19:20Z","timestamp":1777490360000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/14738716241240156"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,7]]},"references-count":102,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,7]]}},"alternative-id":["10.1177\/14738716241240156"],"URL":"https:\/\/doi.org\/10.1177\/14738716241240156","relation":{},"ISSN":["1473-8716","1473-8724"],"issn-type":[{"value":"1473-8716","type":"print"},{"value":"1473-8724","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,7]]}}}