{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,21]],"date-time":"2026-11-21T01:59:19Z","timestamp":1795226359338,"version":"4.10.3"},"reference-count":93,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2025,9,23]],"date-time":"2025-09-23T00:00:00Z","timestamp":1758585600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,9,23]],"date-time":"2025-09-23T00:00:00Z","timestamp":1758585600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["72304274"],"award-info":[{"award-number":["72304274"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012325","name":"National Social Science Fund of China","doi-asserted-by":"publisher","award":["22CTQ032"],"award-info":[{"award-number":["22CTQ032"]}],"id":[{"id":"10.13039\/501100012325","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["asistdl.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Asso for Info Science &amp;amp; Tech"],"published-print":{"date-parts":[[2026,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Timely detection of problematic research is essential for safeguarding scientific integrity. To explore whether social media commentary can serve as an early indicator of potentially problematic articles, this study analyzed 3815 tweets referencing 604 retracted articles and 3373 tweets referencing 668 comparable non\u2010retracted articles. Tweets critical of the articles were identified through both human annotation and large language models (LLMs). Human annotation revealed that 8.3% of retracted articles were associated with at least one critical tweet prior to retraction, compared to only 1.5% of non\u2010retracted articles, highlighting the potential of tweets as early warning signals of retraction. However, critical tweets identified by LLMs (GPT\u20104o mini, Gemini 2.0 Flash\u2010Lite, and Claude 3.5 Haiku) only partially aligned with human annotation, suggesting that fully automated monitoring of post\u2010publication discourse should be applied with caution. A human\u2013AI collaborative approach may offer a more reliable and scalable alternative, with human expertise helping to filter out tweets critical of issues unrelated to the research integrity of the articles. Overall, this study provides insights into how social media signals, combined with generative AI technologies, may support efforts to strengthen research integrity.<\/jats:p>","DOI":"10.1002\/asi.70028","type":"journal-article","created":{"date-parts":[[2025,9,24]],"date-time":"2025-09-24T03:55:53Z","timestamp":1758686153000},"page":"624-639","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Can social media provide early warning of retraction? Evidence from critical tweets identified by human annotation and large language models"],"prefix":"10.1002","volume":"77","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8759-3643","authenticated-orcid":false,"given":"Er\u2010Te","family":"Zheng","sequence":"first","affiliation":[{"name":"School of Information, Journalism and Communication The University of Sheffield  Sheffield UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1534-9374","authenticated-orcid":false,"given":"Hui\u2010Zhen","family":"Fu","sequence":"additional","affiliation":[{"name":"Department of Information Resources Management Zhejiang University  Hangzhou China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6065-205X","authenticated-orcid":false,"given":"Mike","family":"Thelwall","sequence":"additional","affiliation":[{"name":"School of Information, Journalism and Communication The University of Sheffield  Sheffield UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3802-2227","authenticated-orcid":false,"given":"Zhichao","family":"Fang","sequence":"additional","affiliation":[{"name":"School of Information Resource Management Renmin University of China  Beijing China"},{"name":"Centre for Science and Technology Studies (CWTS) Leiden University  Leiden The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2025,9,23]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"publisher","DOI":"10.22495\/cocv22i1art8"},{"key":"e_1_2_10_3_1","doi-asserted-by":"publisher","DOI":"10.1002\/nop2.51"},{"key":"e_1_2_10_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jdent.2024.104840"},{"key":"e_1_2_10_5_1","doi-asserted-by":"publisher","DOI":"10.1162\/qss_a_00298"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-023-04871-7"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-017-2242-0"},{"key":"e_1_2_10_8_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1001772"},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.1128\/mbio.00809-16"},{"key":"e_1_2_10_10_1","doi-asserted-by":"publisher","DOI":"10.23668\/psycharchives.13062"},{"key":"e_1_2_10_11_1","doi-asserted-by":"publisher","DOI":"10.1177\/1536867X0900900402"},{"key":"e_1_2_10_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-020-03536-z"},{"key":"e_1_2_10_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-018-2698-6"},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1108\/AJIM-12-2014-0180"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.362.6413.390"},{"key":"e_1_2_10_16_1","unstructured":"Brown T. B. Mann B. Ryder N. Subbiah M. Kaplan J. Dhariwal P. Neelakantan A. Shyam P. Sastry G. Askell A. Agarwal S. Herbert\u2010Voss A. Krueger G. Henighan T. Child R. Ramesh A. Ziegler D. M. Wu J. Winter C. \u2026Amodei D.(2020).Language models are few\u2010shot learners. Proceedings of the 34th International Conference on Neural Information Processing Systems 1877\u20131901."},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-99680-2_40"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.3000860"},{"key":"e_1_2_10_19_1","doi-asserted-by":"publisher","DOI":"10.1108\/JKM-04-2020-0252"},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-023-03144-w"},{"key":"e_1_2_10_21_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-023-03074-7"},{"key":"e_1_2_10_22_1","doi-asserted-by":"crossref","unstructured":"COPE Council. (2019 November 2).COPE retraction guidelines\u2014English.https:\/\/doi.org\/10.24318\/cope.2019.1.4","DOI":"10.24318\/cope.2019.1.4"},{"key":"e_1_2_10_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23309"},{"key":"e_1_2_10_24_1","doi-asserted-by":"publisher","DOI":"10.1139\/facets-2018-0002"},{"key":"e_1_2_10_25_1","unstructured":"Dambanemuya H. K. Abhari R. Vincent N. &Horv\u00e1t E.\u2010\u00c1.(2024).Online engagement with retracted articles: Who when and how? (No. arXiv:2203.04228; Version 3).arXiv.http:\/\/arxiv.org\/abs\/2203.04228"},{"key":"e_1_2_10_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.joi.2018.08.002"},{"key":"e_1_2_10_27_1","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.2.1.e4"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1212247109"},{"key":"e_1_2_10_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-020-03405-9"},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-022-04468-6"},{"key":"e_1_2_10_31_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00524\/2471010"},{"key":"e_1_2_10_32_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-025-00648-5"},{"key":"e_1_2_10_33_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2305016120"},{"key":"e_1_2_10_34_1","doi-asserted-by":"publisher","DOI":"10.3389\/fcell.2023.1339390"},{"key":"e_1_2_10_35_1","unstructured":"Guo Y. Guo M. Su J. Yang Z. Zhu M. Li H. Qiu M. &Liu S. S.(2024).Bias in large language models: Origin evaluation and mitigation (No. arXiv:2411.10915).arXiv.https:\/\/doi.org\/10.48550\/arXiv.2411.10915"},{"key":"e_1_2_10_36_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocae210"},{"key":"e_1_2_10_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-021-03962-7"},{"key":"e_1_2_10_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-02511-3_28"},{"key":"e_1_2_10_39_1","doi-asserted-by":"publisher","DOI":"10.1093\/bjps\/axz029"},{"key":"e_1_2_10_40_1","first-page":"1880","volume-title":"Proceedings of the 31st international conference on computational linguistics","author":"Hosseini P.","year":"2025"},{"key":"e_1_2_10_41_1","doi-asserted-by":"publisher","DOI":"10.1093\/pan\/mpr013"},{"key":"e_1_2_10_42_1","doi-asserted-by":"publisher","DOI":"10.56042\/alis.v71i4.14254"},{"key":"e_1_2_10_43_1","first-page":"367","volume-title":"Impacts of Generative AI on the Future of Research and Education","author":"Jan R.","year":"2025"},{"key":"e_1_2_10_44_1","doi-asserted-by":"crossref","unstructured":"Karmakar M. Banshal S. K. &Singh V. K.(2023 September 27).Exploring twitter for scientific and public engagement with scholarly articles.27th International Conference on Science Technology and Innovation Indicators(STI 2023) Leiden The Netherlands.https:\/\/doi.org\/10.55835\/6442b5d9f1b0f0c586cb14ac","DOI":"10.55835\/6442b5d9f1b0f0c586cb14ac"},{"key":"e_1_2_10_45_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1539-6924.1988.tb01168.x"},{"key":"e_1_2_10_46_1","doi-asserted-by":"publisher","DOI":"10.1080\/08989621.2023.2223996"},{"key":"e_1_2_10_47_1","doi-asserted-by":"publisher","DOI":"10.3346\/jkms.2022.37.e44"},{"key":"e_1_2_10_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.clsr.2010.09.002"},{"key":"e_1_2_10_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11948-016-9841-7"},{"key":"e_1_2_10_50_1","unstructured":"Kostina A. Dikaiakos M. D. Stefanidis D. &Pallis G.(2025).Large language models for text classification: Case study and comprehensive review (No. arXiv:2501.08457).arXiv.https:\/\/doi.org\/10.48550\/arXiv.2501.08457"},{"key":"e_1_2_10_51_1","doi-asserted-by":"crossref","unstructured":"Kumar H. Vincentius J. Jordan E. &Anderson A.(2025).Human creativity in the age of LLMs: Randomized experiments on divergent and convergent thinking.Proceedings of the 2025 CHI conference on human factors in computing systems 1\u201318.https:\/\/doi.org\/10.1145\/3706598.3714198","DOI":"10.1145\/3706598.3714198"},{"key":"e_1_2_10_52_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.adu8276"},{"key":"e_1_2_10_53_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2796.1995.tb01222.x"},{"key":"e_1_2_10_54_1","doi-asserted-by":"publisher","DOI":"10.1056\/AIoa2400196"},{"key":"e_1_2_10_55_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-024-03784-6"},{"key":"e_1_2_10_56_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa2007621"},{"key":"e_1_2_10_57_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0197265"},{"key":"e_1_2_10_58_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-023-03635-w"},{"key":"e_1_2_10_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.joi.2018.06.001"},{"key":"e_1_2_10_60_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2119086119"},{"key":"e_1_2_10_61_1","doi-asserted-by":"crossref","unstructured":"Perkovi\u0107 G. Drobnjak A. &Boti\u010dki I.(2024).Hallucinations in LLMs: Understanding and addressing challenges. 2024 47th MIPRO ICT and Electronics Convention (MIPRO) 2084\u20102088.https:\/\/doi.org\/10.1109\/MIPRO60963.2024.10569238","DOI":"10.1109\/MIPRO60963.2024.10569238"},{"key":"e_1_2_10_62_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.aas9490"},{"key":"e_1_2_10_63_1","doi-asserted-by":"publisher","DOI":"10.1002\/meet.14504701201"},{"key":"e_1_2_10_64_1","unstructured":"Ranjan R. Gupta S. &Singh S. N.(2024).A comprehensive survey of bias in LLMs: Current landscape and future directions (No. arXiv:2409.16430).arXiv.https:\/\/doi.org\/10.48550\/arXiv.2409.16430"},{"key":"e_1_2_10_65_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0183551"},{"key":"e_1_2_10_66_1","unstructured":"Sathish V. Lin H. Kamath A. K. &Nyayachavadi A.(2024).LLeMpower: Understanding disparities in the control and access of large language models (No. arXiv:2404.09356).arXiv.https:\/\/doi.org\/10.48550\/arXiv.2404.09356"},{"key":"e_1_2_10_67_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0248625"},{"key":"e_1_2_10_68_1","doi-asserted-by":"publisher","DOI":"10.1515\/iwp-2019-2006"},{"key":"e_1_2_10_69_1","doi-asserted-by":"publisher","DOI":"10.1259\/bjr.20230430"},{"key":"e_1_2_10_70_1","doi-asserted-by":"publisher","DOI":"10.2218\/ijdc.v7i1.224"},{"key":"e_1_2_10_71_1","doi-asserted-by":"publisher","DOI":"10.1177\/0165551520945853"},{"key":"e_1_2_10_72_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23833"},{"key":"e_1_2_10_73_1","doi-asserted-by":"publisher","DOI":"10.1088\/1402-4896\/aceb40"},{"key":"e_1_2_10_74_1","doi-asserted-by":"publisher","DOI":"10.1108\/JICES-05-2018-0051"},{"key":"e_1_2_10_75_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0064841"},{"key":"e_1_2_10_76_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2025.104123"},{"key":"e_1_2_10_77_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0268999"},{"key":"e_1_2_10_78_1","unstructured":"Vajjala S. &Shimangaud S.(2025).Text classification in the LLM era\u2014Where do we stand? (arXiv:2502.11830).arXiv.https:\/\/doi.org\/10.48550\/arXiv.2502.11830"},{"key":"e_1_2_10_79_1","doi-asserted-by":"publisher","DOI":"10.1038\/478026a"},{"key":"e_1_2_10_80_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-023-03974-8"},{"key":"e_1_2_10_81_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-023-02554-0"},{"key":"e_1_2_10_82_1","doi-asserted-by":"publisher","DOI":"10.6087\/kcse.187"},{"key":"e_1_2_10_83_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10049-011-1543-8"},{"key":"e_1_2_10_84_1","doi-asserted-by":"publisher","DOI":"10.3390\/vaccines9070693"},{"key":"e_1_2_10_85_1","doi-asserted-by":"crossref","unstructured":"Wang Z. Pang Y. Lin Y. &Zhu X.(2024).Adaptable and reliable text classification using large language models (No. arXiv:2405.10523).arXiv.https:\/\/doi.org\/10.48550\/arXiv.2405.10523","DOI":"10.1109\/ICDMW65004.2024.00015"},{"key":"e_1_2_10_86_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2288-13-61"},{"key":"e_1_2_10_87_1","doi-asserted-by":"publisher","DOI":"10.1097\/MD.0000000000037589"},{"key":"e_1_2_10_88_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.joi.2019.08.001"},{"key":"e_1_2_10_89_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.joi.2017.02.011"},{"key":"e_1_2_10_90_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.24830"},{"key":"e_1_2_10_91_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.surfin.2024.104081"},{"key":"e_1_2_10_92_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.joi.2025.101682"},{"key":"e_1_2_10_93_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00502"},{"key":"e_1_2_10_94_1","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2023.1350306"}],"container-title":["Journal of the Association for Information Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/asi.70028","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/asi.70028","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/asi.70028","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T04:52:59Z","timestamp":1773031979000},"score":1,"resource":{"primary":{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/10.1002\/asi.70028"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,23]]},"references-count":93,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,4]]}},"alternative-id":["10.1002\/asi.70028"],"URL":"https:\/\/doi.org\/10.1002\/asi.70028","archive":["Portico"],"relation":{},"ISSN":["2330-1635","2330-1643"],"issn-type":[{"value":"2330-1635","type":"print"},{"value":"2330-1643","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,23]]},"assertion":[{"value":"2024-12-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-12","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}