{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T06:29:03Z","timestamp":1785392943931,"version":"3.55.0"},"reference-count":38,"publisher":"BMJ","issue":"4","license":[{"start":{"date-parts":[[2025,2,20]],"date-time":"2025-02-20T00:00:00Z","timestamp":1740009600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ EBM"],"accepted":{"date-parts":[[2025,1,7]]},"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>We aimed to determine to what extent systematic reviewers and post-preprint and post-publication peer review identified methodological and reporting issues in COVID-19 trials that could be easily resolved by the authors.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Design<\/jats:title>\n                  <jats:p>Qualitative study.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Data sources<\/jats:title>\n                  <jats:p>COVID-NMA living systematic review (<jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/covid-nma.com\">covid-nma.com<\/jats:ext-link>), PubPeer, medRxiv, Research Square, SSRN.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>We considered randomised controlled trials (RCTs) in COVID-NMA that evaluated pharmacological treatments for COVID-19 and retrieved systematic reviewers\u2019 assessments of the risk of bias and outcome reporting bias. We also searched for commentary data on PubPeer and preprint servers up to 6 November 2023. We employed qualitative content analysis to develop themes and domains of methodological and reporting issues identified by commenters.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We identified 500 eligible RCTs. Systematic reviewers identified methodological and reporting issues in 446 (89%) RCTs. In 391 (78%) RCTs, the issues could be easily resolved by the trial authors; issues included incomplete reporting (49%), selection of the reported results (52%) and no access to the pre-specified plan (25%). Alternatively, 74 (15%) RCTs had received at least one comment on PubPeer or preprint servers, totalling 348 comments. In 46 (9%) RCTs, the issues identified by post-preprint and post-publication peer review comments could be easily resolved by the trial authors; the issues were related to incomplete reporting (6%), errors (5%), statistical analysis (3%), inconsistent reporting of methods and analyses (2%), spin (2%), selection of the reported results (1%) and no access to the raw data\/pre-specified plan (1%).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Without changing their process, systematic reviewers identified issues in most RCTs that could be easily resolved by the trial authors; however, the lack of an established author feedback mechanism represents a wasted opportunity for facilitating improvement and enhancing the overall manuscript quality. On the other hand, despite the existing feedback loop to authors present in post-publication peer review, it demonstrated limited effectiveness in identifying methodological and reporting issues.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1136\/bmjebm-2024-113068","type":"journal-article","created":{"date-parts":[[2025,2,20]],"date-time":"2025-02-20T13:20:15Z","timestamp":1740057615000},"page":"233-240","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":3,"title":["Post-publication peer review and the identification of methodological and reporting issues in COVID-19 trials: a qualitative study"],"prefix":"10.1136","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9991-8035","authenticated-orcid":false,"given":"Mauricia","family":"Davidson","sequence":"first","affiliation":[{"name":"Universit\u00e9 Paris Cit\u00e9 and Universit\u00e9 Sorbonne Paris Nord, Inserm, INRAE, Centre for Research in Epidemiology and Statistics (CRESS), Paris, \u00cele-de-France, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2159-7662","authenticated-orcid":false,"given":"Christoffer Bruun","family":"Korfitsen","sequence":"additional","affiliation":[{"name":"Open Patient Data Explorative Network (OPEN), Odense University Hospital, Odense, Denmark"},{"name":"Cochrane Denmark & Centre for Evidence-Based Medicine Odense (CEBMO), Department of Clinical Research, University of Southern Denmark, Odense, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carolina","family":"Riveros","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris Cit\u00e9 and Universit\u00e9 Sorbonne Paris Nord, Inserm, INRAE, Centre for Research in Epidemiology and Statistics (CRESS), Paris, \u00cele-de-France, France"},{"name":"Cochrane Centre France, Paris, \u00cele-de-France, France"},{"name":"Centre d\u2019Epid\u00e9miologie Clinique, AP-HP, H\u00f4pital H\u00f4tel Dieu, Paris, \u00cele-de-France, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anna","family":"Chaimani","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris Cit\u00e9 and Universit\u00e9 Sorbonne Paris Nord, Inserm, INRAE, Centre for Research in Epidemiology and Statistics (CRESS), Paris, \u00cele-de-France, France"},{"name":"Cochrane Centre France, Paris, \u00cele-de-France, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Isabelle","family":"Boutron","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris Cit\u00e9 and Universit\u00e9 Sorbonne Paris Nord, Inserm, INRAE, Centre for Research in Epidemiology and Statistics (CRESS), Paris, \u00cele-de-France, France"},{"name":"Cochrane Centre France, Paris, \u00cele-de-France, France"},{"name":"Centre d\u2019Epid\u00e9miologie Clinique, AP-HP, H\u00f4pital H\u00f4tel Dieu, Paris, \u00cele-de-France, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2025,2,20]]},"reference":[{"key":"2025082722260154000_30.4.233.1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-7799(02)01985-6"},{"key":"2025082722260154000_30.4.233.2","doi-asserted-by":"crossref","DOI":"10.1186\/s12916-019-1347-0","article-title":"A scoping review on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals","volume":"17","author":"Glonti","year":"2019","journal-title":"BMC Med"},{"key":"2025082722260154000_30.4.233.3","doi-asserted-by":"publisher","DOI":"10.1002\/aris.2011.1440450112"},{"key":"2025082722260154000_30.4.233.4","doi-asserted-by":"crossref","DOI":"10.1186\/s12916-022-02567-y","article-title":"Transparency and reporting characteristics of COVID-19 randomized controlled trials","volume":"20","author":"Kapp","year":"2022","journal-title":"BMC Med"},{"key":"2025082722260154000_30.4.233.5","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2016.04.012"},{"key":"2025082722260154000_30.4.233.6","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.g4145","article-title":"Impact of peer review on reports of randomised trials published in open peer review journals: retrospective before and after study","volume":"349","author":"Hopewell","year":"2014","journal-title":"BMJ"},{"key":"2025082722260154000_30.4.233.7","doi-asserted-by":"publisher","DOI":"10.1016\/j.anpede.2023.05.006"},{"key":"2025082722260154000_30.4.233.8","doi-asserted-by":"crossref","DOI":"10.1186\/s41073-020-00092-1","article-title":"The limitations to our understanding of peer review","volume":"5","author":"Tennant","year":"2020","journal-title":"Res Integr Peer Rev"},{"key":"2025082722260154000_30.4.233.9","doi-asserted-by":"crossref","DOI":"10.1098\/rsos.220139","article-title":"Post-publication critique at top-ranked journals across scientific disciplines: a cross-sectional assessment of policies and practice","volume":"9","author":"Hardwicke","year":"2022","journal-title":"R Soc Open Sci"},{"key":"2025082722260154000_30.4.233.10","doi-asserted-by":"crossref","DOI":"10.3389\/fncom.2012.00063","article-title":"Post-publication peer review: opening up scientific conversation","volume":"6","author":"Hunter","year":"2012","journal-title":"Front Comput Neurosci"},{"key":"2025082722260154000_30.4.233.11","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-020-03536-z"},{"key":"2025082722260154000_30.4.233.12","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-statistics-031219-041104"},{"key":"2025082722260154000_30.4.233.13","unstructured":"PubPeer . Search publications and join the conversation. Available: https:\/\/blog.pubpeer.com\/ [Accessed 21 Jan 2024]."},{"key":"2025082722260154000_30.4.233.14","doi-asserted-by":"publisher","DOI":"10.29024\/sar.26"},{"key":"2025082722260154000_30.4.233.15","doi-asserted-by":"publisher","DOI":"10.1016\/j.biopha.2018.10.114"},{"key":"2025082722260154000_30.4.233.16","unstructured":"Marcus AA . Journals acknowledge that a critical \u2018reader\u2019 has a name: Elisabeth BiK [Retraction Watch]. Available: https:\/\/retractionwatch.com\/2022\/04\/08\/journals-acknowledge-that-a-critical-reader-has-a-name-elisabeth-bik\/ [Accessed 19 May 2023]."},{"key":"2025082722260154000_30.4.233.17","unstructured":"Dimensions . Digital science. 2018. Available: https:\/\/app.dimensions.ai [Accessed 19 Nov 2024]."},{"key":"2025082722260154000_30.4.233.18","doi-asserted-by":"publisher","DOI":"10.1093\/reseval\/rvaa037"},{"key":"2025082722260154000_30.4.233.19","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pbio.3000959","article-title":"The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape","volume":"19","author":"Fraser","year":"2021","journal-title":"PLoS Biol"},{"key":"2025082722260154000_30.4.233.20","doi-asserted-by":"publisher","DOI":"10.7326\/M20-5261"},{"key":"2025082722260154000_30.4.233.21","unstructured":"Living overview of the evidence (LOVE). Available: https:\/\/app.iloveevidence.com\/loves\/5e6fdb9669c00e4ac072701d [Accessed 19 May 2021]."},{"key":"2025082722260154000_30.4.233.22","unstructured":"\"Cochrane covid-19 study register,\u201d cochrane covid-19 study register. Available: https:\/\/covid-19.cochrane.org\/?sf=publishedDate&sd=desc [Accessed 19 May 2021]."},{"key":"2025082722260154000_30.4.233.23","unstructured":"Retracted coronavirus (covid-19) papers [Retraction Watch]. Available: https:\/\/retractionwatch.com\/retracted-coronavirus-covid-19-papers\/ [Accessed 01 Jun 2023]."},{"key":"2025082722260154000_30.4.233.24","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-021-03900-7"},{"key":"2025082722260154000_30.4.233.25","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.l4898","article-title":"RoB 2: a revised tool for assessing risk of bias in randomised trials","volume":"366","author":"Sterne","year":"2019","journal-title":"BMJ"},{"key":"2025082722260154000_30.4.233.26","unstructured":"Cochrane handbook for systematic reviews of interventions. Available: https:\/\/training.cochrane.org\/handbook [Accessed 25 May 2023]."},{"key":"2025082722260154000_30.4.233.27","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.k3802","article-title":"Outcome reporting bias in trials: a methodological approach for assessment and adjustment in systematic reviews","volume":"362","author":"Kirkham","year":"2018","journal-title":"BMJ"},{"key":"2025082722260154000_30.4.233.28","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2023.31410","article-title":"Characterization of Comments About bioRxiv and medRxiv Preprints","volume":"6","author":"Carneiro","year":"2023","journal-title":"JAMA Netw Open"},{"key":"2025082722260154000_30.4.233.29","unstructured":"R Core Team . R: a language and environment for statistical computing. r foundation for statistical computing. Vienna, Austria, 2022. Available: https:\/\/www.r-project.org\/"},{"key":"2025082722260154000_30.4.233.30","doi-asserted-by":"publisher","DOI":"10.21105\/joss.01686"},{"key":"2025082722260154000_30.4.233.31","doi-asserted-by":"publisher","DOI":"10.1002\/asi.24568"},{"key":"2025082722260154000_30.4.233.32","doi-asserted-by":"publisher","DOI":"10.11613\/BM.2021.020201"},{"key":"2025082722260154000_30.4.233.33","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0257093","article-title":"Evaluation of reporting quality of randomized controlled trials in patients with COVID-19 using the CONSORT statement","volume":"16","author":"Yin","year":"2021","journal-title":"PLoS One"},{"key":"2025082722260154000_30.4.233.34","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0292860","article-title":"Poor reporting quality of randomized controlled trials comparing treatments of COVID-19-A retrospective cross-sectional study on the first year of publications","volume":"18","author":"Gr\u00fc\u00dfer","year":"2023","journal-title":"PLoS One"},{"key":"2025082722260154000_30.4.233.35","doi-asserted-by":"crossref","DOI":"10.1186\/s12916-021-01920-x","article-title":"Following the science? Comparison of methodological and reporting quality of covid-19 and other research from the first wave of the pandemic","volume":"19","author":"Quinn","year":"2021","journal-title":"BMC Med"},{"key":"2025082722260154000_30.4.233.36","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2020.01.024"},{"key":"2025082722260154000_30.4.233.37","doi-asserted-by":"crossref","DOI":"10.1007\/s11948-023-00447-z","article-title":"Post-publication Peer Review with an Intention to Uncover Data\/Result Irregularities and Potential Research Misconduct in Scientific Research: Vigilantism or Volunteerism?","volume":"29","author":"Yeo-Teh","year":"2023","journal-title":"Sci Eng Ethics"},{"key":"2025082722260154000_30.4.233.38","unstructured":"Lapinski PS . Information access and scholarly communication in post-publication peer review online social networks. 2021. Available: https:\/\/dash.harvard.edu\/handle\/1\/37367697 [Accessed 23 Jan 2024]."}],"container-title":["BMJ Evidence-Based Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjebm-2024-113068","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,28]],"date-time":"2025-08-28T05:26:14Z","timestamp":1756358774000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebm.bmj.com\/lookup\/doi\/10.1136\/bmjebm-2024-113068"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,20]]},"references-count":38,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,7,21]]},"published-print":{"date-parts":[[2025,8]]}},"alternative-id":["10.1136\/bmjebm-2024-113068"],"URL":"https:\/\/doi.org\/10.1136\/bmjebm-2024-113068","relation":{},"ISSN":["2515-446X","2515-4478"],"issn-type":[{"value":"2515-446X","type":"print"},{"value":"2515-4478","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,20]]}}}