{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,26]],"date-time":"2026-10-26T20:12:17Z","timestamp":1793045537760,"version":"4.5.0"},"reference-count":49,"publisher":"American Medical Association (AMA)","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Netw Open"],"abstract":"\n Importance<\/jats:title>\n Systematic reviews are the criterion standard for evidence synthesis in the life sciences, yet their reliability and integrity are threatened by citation contamination from fabricated publications produced by paper mills. Despite growing awareness, the extent and implications of this issue remain unclear.<\/jats:p>\n <\/jats:sec>\n \n Objectives<\/jats:title>\n To analyze the prevalence, characteristics, affected subject areas, and citation patterns of retracted paper mill articles cited in systematic reviews.<\/jats:p>\n <\/jats:sec>\n \n Design, Setting, and Participants<\/jats:title>\n This cross-sectional study analyzed systematic reviews published between 2013 and 2024, indexed in Web of Science (WoS). References were matched against the Retraction Watch dataset, and full texts were reviewed to identify retracted paper mill articles incorporated into the evidence synthesis.<\/jats:p>\n <\/jats:sec>\n \n Main Outcomes and Measures<\/jats:title>\n The study assessed (1) contamination prevalence, defined as the proportion of systematic reviews incorporating retracted paper mill articles into the evidence synthesis; (2) geographic distribution of citing authors according to institutional affiliations; (3) citation timing and trends, including the time lag between incorporation and article retraction; (4) affected research areas, categorized by WoS subject classifications; and (5) citation patterns, including highly contaminated reviews (\u22653 incorporations of retracted articles).<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Of the total of 200\u202f000 systematic reviews, 299 incorporated at least 1 retracted paper mill article into the evidence synthesis (contamination rate, 0.15%). Among them, 256 (85.6%) included a single retracted article, and 43 (14.4%) included multiple such articles. Of 1802 author affiliations associated with the contaminated reviews, 660 (36.6%) were from institutions in China. Of 385 total citations, 124 (32.2%) occurred after retraction, including 13 occurring more than 500 days after the retraction date. Oncology was the most affected field (48 of 299 [16.1%]). Five reviews each included 5 or more retracted articles, all published in journals under questionable publishers.<\/jats:p>\n <\/jats:sec>\n \n Conclusions and Relevance<\/jats:title>\n In this cross-sectional study of life sciences systematic reviews, contamination remained low but increased over time, posing a risk to research integrity. Continued citation of retracted articles, even after retraction, highlights the need for rigorous screening practices. Correcting contaminated reviews and developing automated detection tools are essential to preserving the credibility of systematic reviews.<\/jats:p>\n <\/jats:sec>","DOI":"10.1001\/jamanetworkopen.2025.15160","type":"journal-article","created":{"date-parts":[[2025,6,12]],"date-time":"2025-06-12T11:00:57Z","timestamp":1749726057000},"page":"e2515160","source":"Crossref","is-referenced-by-count":29,"title":["Citation Contamination by Paper Mill Articles in Systematic Reviews of the Life Sciences"],"prefix":"10.1001","volume":"8","author":[{"given":"Gengyan","family":"Tang","sequence":"first","affiliation":[{"name":"Werklund School of Education, University of Calgary, Calgary, Alberta, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Cai","sequence":"additional","affiliation":[{"name":"The First Affiliated Hospital, Chongqing Medical University, Chongqing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","published-online":{"date-parts":[[2025,6,12]]},"reference":[{"issue":"3","key":"zoi250492r1","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1080\/19439342.2012.710641","article-title":"Narrative approaches to systematic review and synthesis of evidence for international development policy and practice.","volume":"4","author":"Snilstveit","year":"2012","journal-title":"J Dev Effect"},{"issue":"2","key":"zoi250492r2","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1177\/1747493017743796","article-title":"How to do a systematic review.","volume":"13","author":"Pollock","year":"2018","journal-title":"Int J Stroke"},{"issue":"2","key":"zoi250492r3","doi-asserted-by":"publisher","first-page":"103","DOI":"10.4097\/kjae.2018.71.2.103","article-title":"Introduction to systematic review and meta-analysis.","volume":"71","author":"Ahn","year":"2018","journal-title":"Korean J Anesthesiol"},{"issue":"3","key":"zoi250492r4","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1002\/ejp.1905","article-title":"Systematic reviews do not (yet) represent the \u2018gold standard\u2019 of evidence: a position paper.","volume":"26","author":"Moore","year":"2022","journal-title":"Eur J Pain"},{"issue":"7514","key":"zoi250492r5","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1136\/bmj.38478.497164.F7","article-title":"Systematic review of publication bias in studies on publication bias.","volume":"331","author":"Dubben","year":"2005","journal-title":"BMJ"},{"issue":"2","key":"zoi250492r6","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1136\/bmjebm-2018-111107","article-title":"Catalogue of bias: publication bias.","volume":"24","author":"DeVito","year":"2019","journal-title":"BMJ Evid Based Med"},{"issue":"3","key":"zoi250492r7","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1503\/jpn.120065","article-title":"Publication bias: what are the challenges and can they be overcome?","volume":"37","author":"Joober","year":"2012","journal-title":"J Psychiatry Neurosci"},{"issue":"11","key":"zoi250492r8","doi-asserted-by":"publisher","first-page":"e109","DOI":"10.1016\/j.jid.2016.08.021","article-title":"Research techniques made simple: assessing risk of bias in systematic reviews.","volume":"136","author":"Drucker","year":"2016","journal-title":"J Invest Dermatol"},{"issue":"4","key":"zoi250492r9","doi-asserted-by":"publisher","first-page":"551","DOI":"10.1590\/1982-021620171941117","article-title":"How to avoid bias in systematic reviews of observational studies.","volume":"19","author":"de Almeida","year":"2017","journal-title":"Rev CEFAC"},{"issue":"3","key":"zoi250492r10","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1002\/jrsm.1106","article-title":"Searching for grey literature for systematic reviews: challenges and benefits.","volume":"5","author":"Mahood","year":"2014","journal-title":"Res Synth Methods"},{"issue":"2","key":"zoi250492r11","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1017\/S0266462312000086","article-title":"The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies.","volume":"28","author":"Morrison","year":"2012","journal-title":"Int J Technol Assess Health Care"},{"issue":"2","key":"zoi250492r12","first-page":"156","article-title":"Clinical epidemiology workshop 11\u2014sources of bias in studies of systematic reviews with or without meta-analysis.","volume":"19","author":"Yu","year":"2013","journal-title":"Hong Kong Med J"},{"issue":"71","key":"zoi250492r13","doi-asserted-by":"publisher","first-page":"n71","DOI":"10.1136\/bmj.n71","article-title":"The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.","volume":"372","author":"Page","year":"2021","journal-title":"BMJ"},{"key":"zoi250492r14","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2024.111549","article-title":"Paper mill challenges: past, present, and future.","volume":"176","author":"Parker","year":"2024","journal-title":"J Clin Epidemiol"},{"issue":"7851","key":"zoi250492r15","doi-asserted-by":"publisher","first-page":"516","DOI":"10.1038\/d41586-021-00733-5","article-title":"The fight against fake-paper factories that churn out sham science.","volume":"591","author":"Else","year":"2021","journal-title":"Nature"},{"issue":"18","key":"zoi250492r16","doi-asserted-by":"publisher","first-page":"3027","DOI":"10.1002\/1873-3468.13201","article-title":"Systematic fabrication of scientific images revealed.","volume":"592","author":"Christopher","year":"2018","journal-title":"FEBS Lett"},{"issue":"13","key":"zoi250492r17","doi-asserted-by":"publisher","first-page":"1751","DOI":"10.1002\/1873-3468.14143","article-title":"The raw truth about paper mills.","volume":"595","author":"Christopher","year":"2021","journal-title":"FEBS Lett"},{"issue":"6725","key":"zoi250492r18","doi-asserted-by":"publisher","first-page":"955","DOI":"10.1126\/science.adu8281","article-title":"Fake papers compromise research syntheses.","volume":"386","author":"Else","year":"2024","journal-title":"Science"},{"issue":"11","key":"zoi250492r19","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.3002931","article-title":"A call for research to address the threat of paper mills.","volume":"22","author":"Byrne","year":"2024","journal-title":"PLoS Biol"},{"issue":"4","key":"zoi250492r20","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1002\/leap.1574","article-title":"Publication and collaboration anomalies in academic papers originating from a paper mill: evidence from a Russia-based paper mill.","volume":"36","author":"Abalkina","year":"2023","journal-title":"Learn Publ"},{"issue":"4","key":"zoi250492r21","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1002\/1873-3468.13747","article-title":"Digital magic, or the dark arts of the 21st century: how can journals and peer reviewers detect manuscripts and publications from paper mills?","volume":"594","author":"Byrne","year":"2020","journal-title":"FEBS Lett"},{"issue":"3","key":"zoi250492r22","doi-asserted-by":"publisher","first-page":"1471","DOI":"10.1007\/s11192-016-2209-6","article-title":"Striking similarities between publications from China describing single gene knockdown experiments in human cancer cell lines.","volume":"110","author":"Byrne","year":"2016","journal-title":"Scientometrics"},{"key":"zoi250492r23","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2024.111608","article-title":"Yet another problem with systematic reviews: a living review update.","volume":"177","author":"Uttley","year":"2025","journal-title":"J Clin Epidemiol"},{"issue":"5","key":"zoi250492r27","doi-asserted-by":"publisher","first-page":"1137","DOI":"10.3390\/cancers13051137","article-title":"MicroRNA as epigenetic modifiers in endometrial cancer: a systematic review.","volume":"13","author":"Favier","year":"2021","journal-title":"Cancers (Basel)"},{"issue":"16","key":"zoi250492r28","doi-asserted-by":"publisher","first-page":"9354","DOI":"10.3390\/ijms23169354","article-title":"Expression of MicroRNAs in sepsis-related organ dysfunction: a systematic review.","volume":"23","author":"Maiese","year":"2022","journal-title":"Int J Mol Sci"},{"issue":"11","key":"zoi250492r29","doi-asserted-by":"publisher","first-page":"3118","DOI":"10.3390\/cancers12113118","article-title":"MicroRNA in papillary thyroid carcinoma: a systematic review from 2018 to June 2020.","volume":"12","author":"Hitu","year":"2020","journal-title":"Cancers (Basel)"},{"key":"zoi250492r30","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2022.879704","article-title":"The mechanism of long non-coding RNA in cancer radioresistance\/radiosensitivity: a systematic review.","volume":"13","author":"Wu","year":"2022","journal-title":"Front Pharmacol"},{"issue":"14","key":"zoi250492r31","doi-asserted-by":"publisher","first-page":"3393","DOI":"10.3390\/cancers13143393","article-title":"The role of miRNAs in the regulation of endometrial cancer invasiveness and metastasis\u2014a systematic review.","volume":"13","author":"Klicka","year":"2021","journal-title":"Cancers (Basel)"},{"issue":"s3","key":"zoi250492r33","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1034\/j.1600-051X.29.s3.15.x","article-title":"A guide to systematic reviews.","volume":"29","author":"Needleman","year":"2002","journal-title":"J Clin Periodontol"},{"issue":"4","key":"zoi250492r34","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1136\/qhc.12.4.298","article-title":"Systematic reviews of the effectiveness of quality improvement strategies and programmes.","volume":"12","author":"Grimshaw","year":"2003","journal-title":"Qual Saf Health Care"},{"issue":"1","key":"zoi250492r35","doi-asserted-by":"publisher","DOI":"10.1136\/bmjmed-2023-000604","article-title":"Assessing the methodological quality and risk of bias of systematic reviews: primer for authors of overviews of systematic reviews.","volume":"3","author":"Lunny","year":"2024","journal-title":"BMJ Med"},{"issue":"3","key":"zoi250492r36","doi-asserted-by":"publisher","DOI":"10.5867\/medwave.2020.03.7887","article-title":"Clinical practice guidelines: concepts, limitations and challenges.","volume":"20","author":"Franco","year":"2020","journal-title":"Medwave"},{"issue":"6","key":"zoi250492r37","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1016\/j.jclinepi.2013.01.008","article-title":"Systematic reviews supporting practice guideline recommendations lack protection against bias.","volume":"66","author":"Brito","year":"2013","journal-title":"J Clin Epidemiol"},{"key":"zoi250492r38","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/j.jclinepi.2021.08.013","article-title":"Clinical Epidemiology in China series. Paper 4: The reporting and methodological quality of Chinese clinical practice guidelines published between 2014 and 2018: a systematic review.","volume":"140","author":"Zhou","year":"2021","journal-title":"J Clin Epidemiol"},{"key":"zoi250492r39","doi-asserted-by":"publisher","DOI":"10.1017\/S0950268821001758","article-title":"The quality of systematic reviews and other synthesis in the time of COVID-19.","volume":"149","author":"Baumeister","year":"2021","journal-title":"Epidemiol Infect"},{"issue":"24","key":"zoi250492r40","doi-asserted-by":"publisher","DOI":"10.3346\/jkms.2021.36.e165","article-title":"Retractions, fake peer reviews, and paper mills.","volume":"36","author":"Rivera","year":"2021","journal-title":"J Korean Med Sci"},{"key":"zoi250492r41","doi-asserted-by":"publisher","DOI":"10.15626\/MP.2022.3422","article-title":"Paper mills: a novel form of publishing malpractice affecting psychology.","volume":"7","author":"Bishop","year":"2023","journal-title":"Meta-Psychology."},{"issue":"12","key":"zoi250492r42","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2024.e33007","article-title":"Recent trends: retractions of articles in the oncology field.","volume":"10","author":"Qi","year":"2024","journal-title":"Heliyon"},{"issue":"3","key":"zoi250492r43","doi-asserted-by":"publisher","first-page":"29","DOI":"10.3390\/publications10030029","article-title":"Sustained rise in retractions in the life sciences literature during the pandemic years 2020 and 2021.","volume":"10","author":"Yeo-Teh","year":"2022","journal-title":"Publications"},{"key":"zoi250492r44","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2024.111397","article-title":"Retracted papers originating from paper mills: a cross-sectional analysis of references and citations.","volume":"172","author":"Candal-Pedreira","year":"2024","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"zoi250492r45","doi-asserted-by":"publisher","first-page":"47","DOI":"10.5195\/jmla.2022.1280","article-title":"Retracted publications in pharmacy systematic reviews.","volume":"110","author":"Brown","year":"2022","journal-title":"J Med Libr Assoc"},{"issue":"4","key":"zoi250492r46","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1002\/cre2.292","article-title":"Retracted publications and their citation in dental literature: a systematic review.","volume":"6","author":"Rapani","year":"2020","journal-title":"Clin Exp Dent Res"},{"issue":"10","key":"zoi250492r47","doi-asserted-by":"publisher","first-page":"8363","DOI":"10.1007\/s11192-021-04125-4","article-title":"Team size and retracted citations reveal the patterns of retractions from 1981 to 2020.","volume":"126","author":"Sharma","year":"2021","journal-title":"Scientometrics"},{"issue":"3","key":"zoi250492r49","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1093\/reseval\/rvab020","article-title":"Journal citation reports and the definition of a predatory journal: the case of the Multidisciplinary Digital Publishing Institute (MDPI).","volume":"30","author":"Oviedo-Garc\u00eda","year":"2021","journal-title":"Res Eval"},{"key":"zoi250492r24","unstructured":"Retraction Watch. Retraction Watch data. GitLab. 2025. Accessed March 8, 2025. https:\/\/gitlab.com\/crossref\/retraction-watch-data"},{"key":"zoi250492r25","unstructured":"Web of Science. Research areas. Clarivate.com. 2021. Accessed February 27, 2025. https:\/\/webofscience.help.clarivate.com\/Content\/research-areas.html?Highlight=research%20area"},{"key":"zoi250492r26","unstructured":"Altmetric. Accessed March 14, 2025. https:\/\/www.altmetric.com\/"},{"key":"zoi250492r32","unstructured":"Centers for Disease Control and Prevention, Department of Health and Human Services. World Trade Center (WTC) Health Program: addition of uterine cancer to the list of WTC-related health conditions. January 17, 2023. Accessed March 21, 2025. https:\/\/www.cdc.gov\/wtc\/uterineCancer_20230118.html"},{"key":"zoi250492r48","unstructured":"Petrou? C. Guest post: reputation and publication volume at MDPI and Frontiers. The Scholarly Kitchen. September 18, 2023. Accessed March 8, 2025. https:\/\/scholarlykitchen.sspnet.org\/2023\/09\/18\/guest-post-reputation-and-publication-volume-at-mdpi-and-frontiers-the-1b-question\/"}],"container-title":["JAMA Network Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/articlepdf\/2835228\/tang_2025_oi_250492_1749071628.96475.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,12]],"date-time":"2025-06-12T11:00:59Z","timestamp":1749726059000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/fullarticle\/2835228"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,12]]},"references-count":49,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,6,2]]}},"URL":"https:\/\/doi.org\/10.1001\/jamanetworkopen.2025.15160","relation":{},"ISSN":["2574-3805"],"issn-type":[{"value":"2574-3805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,12]]}}}