{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T19:22:53Z","timestamp":1791919373527,"version":"4.3.4"},"reference-count":33,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Generative artificial intelligence (GenAI) tools are increasingly used in scientific research to support literature searches, evidence synthesis, and manuscript preparation. While these systems promise substantial efficiency gains, concerns have emerged regarding their reliability, particularly their tendency to cite inaccurate, fabricated, or retracted literature. The unrecognized inclusion of retracted studies poses a serious risk to research integrity and evidence-based decision-making. Whether commonly used GenAI tools can reliably detect, exclude, or transparently communicate the retraction status of scientific publications remains unclear.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate the ability of freely available GenAI tools to correctly handle retracted scientific articles during literature searches. Primary and secondary outcomes focused on accuracy, reliability, and consistency in recognizing retracted literature.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n In this pragmatic trial, nine widely used free-access GenAI tools (ChatGPT 4, ChatGPT 5, Claude, Gemini, Perplexity, Microsoft Copilot, SciSpace, ScienceOS, and Consensus) were evaluated. Each tool was asked five predefined, standardized questions addressing topic overview, article identification, article summarization, and explicit assessment of retraction status. Overall, 15 retracted articles (the 10 most cited and 5 most recently retracted as of May 23, 2025) were selected from the Retraction Watch database. All questions were repeated twice to assess intratool consistency. Responses were independently rated as correct or incorrect by 2 researchers. Descriptive statistics summarized performance, and comparisons between general-purpose and research-focused AI tools were conducted using descriptive statistics. Interreviewer agreement was assessed using Cohen kappa coefficient.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n None of the evaluated AI tools consistently handled retracted articles correctly. No model achieved perfect accuracy across all question sets. ChatGPT 5 performed best, defined by the primary outcome of achieving fully correct responses to all five predefined tasks (5\/5) for the highest number of retracted articles, correctly answering all five questions for 8 of 15 articles (53.3%). Research-focused tools (SciSpace, ScienceOS, and Consensus) failed to produce a single fully correct response set. Retracted articles were frequently included in topic overviews without warning, with error rates exceeding 40% in several tools. When specifically asked about retraction status, most systems failed to provide correct or complete information. OpenEvidence only reported data for a subset of our retracted articles as it is only used in health care literature. It demonstrated strong performance in topic overviews but low accuracy in identifying retracted articles.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Freely available GenAI tools are currently not able to detect, exclude, or appropriately flag retracted scientific literature. The widespread and confident reproduction of retracted studies represents a substantial threat to research integrity, particularly in medical and evidence-based fields. Until retraction-aware verification mechanisms are systematically integrated, independent source checking remains essential when using AI-assisted literature tools.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/88766","type":"journal-article","created":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T14:50:07Z","timestamp":1774709407000},"page":"e88766-e88766","source":"Crossref","is-referenced-by-count":5,"title":["Performance of AI Tools in Citing Retracted Literature : Content Analysis"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6877-1528","authenticated-orcid":false,"given":"Sebastian","family":"Labenbacher","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5330-8138","authenticated-orcid":false,"given":"Maximilian","family":"Niederer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-4062-2572","authenticated-orcid":false,"given":"Sascha","family":"Hammer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4620-6042","authenticated-orcid":false,"given":"Matthias","family":"Bader","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3691-8457","authenticated-orcid":false,"given":"Nikolaus","family":"Schreiber","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1201-3752","authenticated-orcid":false,"given":"Helmar","family":"Bornemann-Cimenti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,5,1]]},"reference":[{"issue":"6","key":"R1","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1038\/s41585-023-00746-x","article-title":"Artificial intelligence in academic writing: a paradigm-shifting technological advance","volume":"20","author":"Golan","journal-title":"Nat Rev Urol"},{"key":"R2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpbup.2024.100145","article-title":"Using artificial intelligence in academic writing and research: an essential productivity tool","volume":"5","author":"Khalifa","journal-title":"Computer Methods and Programs in Biomedicine Update"},{"key":"R3","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijinfomgt.2023.102642","article-title":"Opinion Paper: \u201cSo what if ChatGPT wrote it?\u201d Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy","volume":"71","author":"Dwivedi","journal-title":"Int J Inf Manage"},{"issue":"2","key":"R4","doi-asserted-by":"publisher","first-page":"615","DOI":"10.5114\/biolsport.2023.125623","article-title":"From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing","volume":"40","author":"Dergaa","journal-title":"Biol Sport"},{"issue":"10","key":"R5","doi-asserted-by":"publisher","first-page":"2171","DOI":"10.1007\/s00607-023-01181-x","article-title":"Artificial intelligence to automate the systematic review of scientific literature","volume":"105","author":"de la Torre-L\u00f3pez","journal-title":"Computing"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","DOI":"10.1186\/s13643-024-02682-2","article-title":"Leveraging artificial intelligence to enhance systematic reviews in health research: advanced tools and challenges","volume":"13","author":"Ge","journal-title":"Syst Rev"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.2196\/47049","article-title":"The potential and concerns of using AI in scientific research: ChatGPT performance evaluation","volume":"9","author":"Khlaif","journal-title":"JMIR Med Educ"},{"key":"R8","doi-asserted-by":"publisher","DOI":"10.2196\/46924","article-title":"Artificial intelligence can generate fraudulent but authentic-looking scientific medical articles: pandora\u2019s box has been opened","volume":"25","author":"M\u00e1jovsk\u00fd","journal-title":"J Med Internet Res"},{"issue":"4","key":"R9","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1016\/j.pop.2025.07.009","article-title":"Ethical and legal considerations of medical artificial intelligence","volume":"52","author":"Montalbano","journal-title":"Prim Care"},{"key":"R10","doi-asserted-by":"crossref","unstructured":"Jan R . Examining the reliability of ChatGPT: identifying retracted scientific literature and ensuring accurate citations and references. In: Impacts of Generative AI on the Future of Research and Education. IGI Global Scientific Publishing; 2024:367-392. [doi: 10.4018\/979-8-3693-0884-4.ch014]","DOI":"10.4018\/979-8-3693-0884-4.ch014"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.1016\/j.jare.2025.03.020","article-title":"Alarm: retracted articles on cancer imaging are not only continuously cited by publications but also used by ChatGPT to answer questions","volume":"71","author":"Gu","journal-title":"J Adv Res"},{"issue":"44","key":"R12","doi-asserted-by":"publisher","DOI":"10.3346\/jkms.2025.40.e280","article-title":"Analysis of retracted publications on artificial intelligence: trends, ethical concerns, and scientific integrity","volume":"40","author":"Kocyigit","journal-title":"J Korean Med Sci"},{"issue":"2","key":"R13","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1136\/bmjebm-2022-111921","article-title":"Reducing the residue of retractions in evidence synthesis: ways to minimise inappropriate citation and use of retracted data","volume":"29","author":"Bakker","journal-title":"BMJ Evid Based Med"},{"key":"R14","unstructured":"AI-assisted knowledge gathering and its pitfalls with retracted articles. Open Science Framework (OSF). URL: https:\/\/doi.org\/10.17605\/OSF.IO\/B6J2W [Accessed 13-04-2026]"},{"issue":"1","key":"R15","doi-asserted-by":"publisher","DOI":"10.1080\/0167482X.2024.2318194","article-title":"Prevalence and knowledge of polycystic ovary syndrome (PCOS) and health-related practices among women of Syria: a cross-sectional study","volume":"45","author":"Bohsas","journal-title":"J Psychosom Obstet Gynaecol"},{"issue":"14","key":"R16","doi-asserted-by":"publisher","first-page":"1279","DOI":"10.1056\/NEJMoa1200303","article-title":"Primary prevention of cardiovascular disease with a Mediterranean diet","volume":"368","author":"Estruch","journal-title":"N Engl J Med"},{"issue":"1","key":"R17","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-79925-8","article-title":"Burnout components, perceived stress and hair cortisol in healthcare professionals during the second wave of COVID 19 pandemic","volume":"14","author":"Fortuna","journal-title":"Sci Rep"},{"issue":"5708","key":"R18","doi-asserted-by":"publisher","first-page":"426","DOI":"10.1126\/science.1097243","article-title":"RETRACTED: Visfatin: a protein secreted by visceral fat that mimics the effects of insulin","volume":"307","author":"Fukuhara","journal-title":"Science"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","first-page":"105949","DOI":"10.1016\/j.ijantimicag.2020.105949","article-title":"RETRACTED: Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial","volume":"56","author":"Gautret","journal-title":"Int J Antimicrob Agents"},{"issue":"10270","key":"R20","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1016\/S0140-6736(20)32656-8","article-title":"6-month consequences of COVID-19 in patients discharged from hospital: a cohort study","volume":"397","author":"Huang","journal-title":"Lancet"},{"issue":"6893","key":"R21","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1038\/nature00870","article-title":"Pluripotency of mesenchymal stem cells derived from adult marrow","volume":"418","author":"Jiang","journal-title":"Nature New Biol"},{"issue":"7082","key":"R22","doi-asserted-by":"publisher","first-page":"352","DOI":"10.1038\/nature04533","article-title":"A specific amyloid-beta protein assembly in the brain impairs memory","volume":"440","author":"Lesn\u00e9","journal-title":"Nature New Biol"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-25387-9","article-title":"A non-enzymatic, isothermal strand displacement and amplification assay for rapid detection of SARS-CoV-2 RNA","volume":"12","author":"Mohammadniaei","journal-title":"Nat Commun"},{"issue":"5","key":"R24","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1111\/j.1751-7176.2008.07572.x","article-title":"Predictive validity of a medication adherence measure in an outpatient setting","volume":"10","author":"Morisky","journal-title":"J Clin Hypertens (Greenwich)"},{"issue":"1","key":"R25","doi-asserted-by":"publisher","DOI":"10.1186\/s13099-021-00398-5","article-title":"Detection of SARS-CoV-2 from patient fecal samples by whole genome sequencing","volume":"13","author":"Papoutsis","journal-title":"Gut Pathog"},{"issue":"3","key":"R26","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1016\/j.pmatsci.2004.04.001","article-title":"RETRACTED: recent progress in processing and properties of ZnO","volume":"50","author":"Pearton","journal-title":"Prog Mater Sci"},{"issue":"6","key":"R27","doi-asserted-by":"publisher","DOI":"10.3390\/mi13060831","article-title":"RETRACTED: surface hydrophilic modification for chip of centrifugal microfluidic immunoassay system","volume":"13","author":"Shi","journal-title":"Micromachines (Basel)"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/j.ygyno.2008.04.033","article-title":"MicroRNA signatures of tumor-derived exosomes as diagnostic biomarkers of ovarian cancer","volume":"110","author":"Taylor","journal-title":"Gynecol Oncol"},{"issue":"9103","key":"R29","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1016\/S0140-6736(97)11096-0","article-title":"RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children","volume":"351","author":"Wakefield","journal-title":"The Lancet"},{"key":"R30","unstructured":"Top 10 most highly cited retracted papers. Retraction Watch. URL: https:\/\/retractionwatch.com\/the-retraction-watch-leaderboard\/top-10-most-highly-cited-retracted-papers\/ [Accessed 13-04-2026]"},{"key":"R31","unstructured":"Retraction watch data. GitLab. URL: https:\/\/gitlab.com\/crossref\/retraction-watch-data [Accessed 13-04-2026]"},{"issue":"2","key":"R32","doi-asserted-by":"publisher","DOI":"10.1002\/leap.1667","article-title":"The citation of retracted papers and impact on the integrity of the scientific biomedical literature","volume":"38","author":"Teixeira da Silva","journal-title":"Learn Publ"},{"issue":"9","key":"R33","doi-asserted-by":"publisher","first-page":"24","DOI":"10.59022\/ijlp.225","article-title":"UNESCO\u2019s AI ethics principles: challenges and opportunities","volume":"2","author":"AllahRakha","journal-title":"Irshad J Law and Policy"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2026\/1\/e88766","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:15:10Z","timestamp":1782839710000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e88766"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,1]]},"references-count":33,"URL":"https:\/\/doi.org\/10.2196\/88766","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,1]]},"article-number":"v28i6e88766"}}