{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T19:49:54Z","timestamp":1791316194091,"version":"4.3.1"},"reference-count":57,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Res Protoc"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Reproducible study findings are those that are shown to be identical or nearly identical when an original study is repeated using similar methods. Findings from many randomized controlled trials (RCTs) in cardiovascular surgery have not been reproduced or are small, statistically fragile, and potentially influenced by funding source. They also demonstrate variable end point selection and protocol adherence, all of which may influence reproducibility of their reported results.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aims to assess the reproducibility of RCTs conducted in the most commonly studied disease states in cardiovascular surgery that compared at least one open surgical procedure to an alternative open, interventional catheter-based, or sham procedure or medical therapy.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We will conduct a meta-epidemiological study of the reproducibility of RCTs using umbrella review methodology. We designed the study protocol using the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols) statement and published recommendations on conducting umbrella reviews. We will search MEDLINE, Embase, and the CENTRAL from their inception for systematic reviews published in the English language that included data from phase 3 RCTs or those with a clinically important primary outcome and compared an open cardiovascular surgery (performed in the most commonly studied disease states across these specialties) to either an alternative open, catheter-based, or sham procedure or medical therapy. Two investigators will independently review the individual RCTs from these systematic reviews and extract data and assess risk of bias (using the Cochrane Risk of Bias-2 tool), the fragility or reverse fragility indices (which provide a measurement of the statistical robustness of the trials), and the authors\u2019 interpretation of the results (also referred to as spin). RCTs will be classified as the original study if it was the first chronological study (based on participant enrollment date) to evaluate the compared interventions and as reproduction attempts if they reevaluate the compared interventions in a different set of participants. The interpretation of a study\u2019s results will be based on the findings of its primary end point and be used to assess reproducibility. We will also create multivariable regression models to determine whether the adjusted odds of reproducibility are independently predicted by 12 different variables.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n We registered the study protocol but have not yet conducted the search. We aim to complete the study by the end of 2026 and submit for peer review by the summer of 2027.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n This study protocol may be used as a framework for future reproducibility research. The results will help guide which cardiovascular surgery practices warrant careful interpretation and which are in need of reproduction studies. They will also identify characteristics of an original RCT in this field that predicts future study reproducibility before reproducibility studies are published.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/93997","type":"journal-article","created":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T13:35:05Z","timestamp":1780580105000},"page":"e93997-e93997","source":"Crossref","is-referenced-by-count":0,"title":["Reproducibility of Published Randomized Controlled Trial Results in Cardiovascular Surgery: Protocol for a Meta-Epidemiological Umbrella Review"],"prefix":"10.2196","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6615-7295","authenticated-orcid":false,"given":"Allen","family":"Li","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7541-2857","authenticated-orcid":false,"given":"Arshia P","family":"Javidan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3253-7211","authenticated-orcid":false,"given":"Daniel J","family":"Niven","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1231-1490","authenticated-orcid":false,"given":"Henry T","family":"Stelfox","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3389-2485","authenticated-orcid":false,"given":"Dean A","family":"Fergusson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9241-8890","authenticated-orcid":false,"given":"Mamas A","family":"Mamas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0538-4456","authenticated-orcid":false,"given":"Prasad","family":"Jetty","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7177-7480","authenticated-orcid":false,"given":"Charles","family":"de Mestral","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0388-5812","authenticated-orcid":false,"given":"Risa","family":"Shorr","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4018-8533","authenticated-orcid":false,"given":"Subodh","family":"Verma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2815-8697","authenticated-orcid":false,"given":"Munir","family":"Boodhwani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6111-6291","authenticated-orcid":false,"given":"Derek J","family":"Roberts","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,6]]},"reference":[{"issue":"10","key":"R1","doi-asserted-by":"publisher","first-page":"1019","DOI":"10.1001\/jama.2017.0549","article-title":"Acknowledging and overcoming nonreproducibility in basic and preclinical research","volume":"317","author":"Ioannidis","journal-title":"JAMA"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1161\/CIRCRESAHA.114.303819","article-title":"Reproducibility in science: improving the standard for basic and preclinical research","volume":"116","author":"Begley","journal-title":"Circ Res"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-018-1018-6","article-title":"Reproducibility of clinical research in critical care: a scoping review","volume":"16","author":"Niven","journal-title":"BMC Med"},{"key":"R4","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.h4320","article-title":"Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence","volume":"351","author":"Le Noury","journal-title":"BMJ"},{"issue":"341","key":"R5","doi-asserted-by":"publisher","DOI":"10.1126\/scitranslmed.aaf5027","article-title":"What does research reproducibility mean?","volume":"8","author":"Goodman","journal-title":"Sci Transl Med"},{"issue":"2","key":"R6","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1001\/jama.294.2.218","article-title":"Contradicted and initially stronger effects in highly cited clinical research","volume":"294","author":"Ioannidis","journal-title":"JAMA"},{"issue":"18","key":"R7","doi-asserted-by":"publisher","first-page":"1675","DOI":"10.1001\/archinternmed.2011.295","article-title":"The frequency of medical reversal","volume":"171","author":"Prasad","journal-title":"Arch Intern Med"},{"issue":"8","key":"R8","doi-asserted-by":"publisher","first-page":"790","DOI":"10.1016\/j.mayocp.2013.05.012","article-title":"A decade of reversal: an analysis of 146 contradicted medical practices","volume":"88","author":"Prasad","journal-title":"Mayo Clin Proc"},{"issue":"6","key":"R9","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1177\/1745691612460688","article-title":"Replications in psychology research: how often do they really occur?","volume":"7","author":"Makel","journal-title":"Perspect Psychol Sci"},{"issue":"6","key":"R10","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1177\/1745691612459058","article-title":"Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability","volume":"7","author":"Nosek","journal-title":"Perspect Psychol Sci"},{"issue":"8","key":"R11","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.0020124","article-title":"Why most published research findings are false","volume":"2","author":"Ioannidis","journal-title":"PLoS Med"},{"issue":"6","key":"R12","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.14494","article-title":"Characteristics of randomized clinical trials in surgery from 2008 to 2020: a systematic review","volume":"4","author":"Robinson","journal-title":"JAMA Netw Open"},{"issue":"7","key":"R13","doi-asserted-by":"publisher","first-page":"993","DOI":"10.1001\/jamainternmed.2020.1670","article-title":"Characteristics of contemporary randomized clinical trials and their association with the trial funding source in invasive cardiovascular interventions","volume":"180","author":"Gaudino","journal-title":"JAMA Intern Med"},{"issue":"7351","key":"R14","doi-asserted-by":"publisher","first-page":"1448","DOI":"10.1136\/bmj.324.7351.1448","article-title":"Randomised trials in surgery: problems and possible solutions","volume":"324","author":"McCulloch","journal-title":"BMJ"},{"issue":"1","key":"R15","doi-asserted-by":"publisher","DOI":"10.1186\/s12893-019-0586-5","article-title":"Challenges and potential solutions to the evaluation, monitoring, and regulation of surgical innovations","volume":"19","author":"Roberts","journal-title":"BMC Surg"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1186\/1745-6215-10-9","article-title":"The challenges faced in the design, conduct and analysis of surgical randomised controlled trials","volume":"10","author":"Cook","journal-title":"Trials"},{"issue":"4","key":"R17","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1161\/CIRCULATIONAHA.120.053216","article-title":"Spending on cardiovascular disease and cardiovascular risk factors in the United States: 1996 to 2016","volume":"144","author":"Birger","journal-title":"Circulation"},{"key":"R18","unstructured":"Cardiovascular diseases (CVDs). World Health Organization. Jul 31, 2025. URL: https:\/\/www.who.int\/news-room\/fact-sheets\/detail\/cardiovascular-diseases-(cvds) [Accessed 01-08-2026]"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","DOI":"10.1186\/s13063-019-3944-9","article-title":"Global mapping of randomised trials related articles published in high-impact-factor medical journals: a cross-sectional analysis","volume":"21","author":"Catal\u00e1-L\u00f3pez","journal-title":"Trials"},{"issue":"20","key":"R20","doi-asserted-by":"publisher","first-page":"1415","DOI":"10.1056\/NEJM199811123392002","article-title":"Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators","volume":"339","author":"Barnett","journal-title":"N Engl J Med"},{"issue":"10","key":"R21","doi-asserted-by":"publisher","first-page":"961","DOI":"10.1056\/NEJMoa0804626","article-title":"Percutaneous coronary intervention versus coronary-artery bypass grafting for severe coronary artery disease","volume":"360","author":"Serruys","journal-title":"N Engl J Med"},{"issue":"3","key":"R22","doi-asserted-by":"publisher","first-page":"1074","DOI":"10.1016\/j.jvs.2021.09.051","article-title":"Analysis of spin in vascular surgery randomized controlled trials with nonsignificant outcomes","volume":"75","author":"Nguyen","journal-title":"J Vasc Surg"},{"issue":"23","key":"R23","doi-asserted-by":"publisher","first-page":"1859","DOI":"10.1161\/CIRCULATIONAHA.119.044697","article-title":"Mortality and paclitaxel-coated devices: an individual patient data meta-analysis","volume":"141","author":"Rocha-Singh","journal-title":"Circulation"},{"issue":"24","key":"R24","doi-asserted-by":"publisher","DOI":"10.1161\/JAHA.118.011245","article-title":"Risk of death following application of paclitaxel-coated balloons and stents in the femoropopliteal artery of the leg: a systematic review and meta-analysis of randomized controlled trials","volume":"7","author":"Katsanos","journal-title":"J Am Heart Assoc"},{"key":"R25","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2024-080881","article-title":"Plain versus drug balloon and stenting in severe ischaemia of the leg (BASIL-3): open label, three arm, randomised, multicentre, phase 3 trial","volume":"388","author":"Bradbury","journal-title":"BMJ"},{"issue":"25","key":"R26","doi-asserted-by":"publisher","first-page":"2305","DOI":"10.1056\/NEJMoa2207899","article-title":"Surgery or endovascular therapy for chronic limb-threatening ischemia","volume":"387","author":"Farber","journal-title":"N Engl J Med"},{"issue":"10508","key":"R27","doi-asserted-by":"publisher","first-page":"1103","DOI":"10.1016\/S0140-6736(25)01585-5","article-title":"Paclitaxel-coated versus uncoated devices for infrainguinal endovascular revascularisation in chronic limb-threatening ischaemia (SWEDEPAD 1): a multicentre, participant-masked, registry-based, randomised controlled trial","volume":"406","author":"Falkenberg","journal-title":"The Lancet"},{"issue":"4","key":"R28","doi-asserted-by":"publisher","DOI":"10.1111\/jebm.70092","article-title":"Umbrella reviews: concepts, methodological frameworks, and step-by-step implementation","volume":"18","author":"Liu","journal-title":"J Evidence Based Medicine"},{"issue":"1","key":"R29","doi-asserted-by":"publisher","DOI":"10.1136\/bmjmed-2021-000071","article-title":"Conducting umbrella reviews","volume":"1","author":"Belbasis","journal-title":"BMJ Med"},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.g7647","article-title":"Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation","volume":"350","author":"Shamseer","journal-title":"BMJ"},{"key":"R31","doi-asserted-by":"publisher","DOI":"10.1016\/j.avsg.2020.03.033","article-title":"A systematic review of economic evaluations in vascular surgery","volume":"67","author":"Javidan","journal-title":"Ann Vasc Surg"},{"issue":"3","key":"R32","doi-asserted-by":"publisher","first-page":"566","DOI":"10.1016\/j.athoracsur.2022.12.033","article-title":"The society of thoracic surgeons adult cardiac surgery database: 2022 update on outcomes and research","volume":"115","author":"Kim","journal-title":"Ann Thorac Surg"},{"issue":"6","key":"R33","doi-asserted-by":"publisher","first-page":"1646","DOI":"10.1016\/j.athoracsur.2020.03.003","article-title":"The Society of Thoracic Surgeons adult cardiac surgery database: 2020 update on outcomes and research","volume":"109","author":"Bowdish","journal-title":"Ann Thorac Surg"},{"key":"R34","unstructured":"Value of vascular surgery: an essential component of a high-performing hospital\u2019s surgical program. Society for Vascular Surgery (SVS); 2020. URL: https:\/\/vascular.org\/sites\/files\/2022-01\/SVS_value_vascular_surgery_CS_v1.2.pdf [Accessed 01-08-2026]"},{"issue":"3","key":"R35","doi-asserted-by":"publisher","first-page":"738","DOI":"10.1016\/j.athoracsur.2021.07.015","article-title":"The effect of COVID-19 on adult cardiac surgery in the United States in 717 103 patients","volume":"113","author":"Nguyen","journal-title":"Ann Thorac Surg"},{"key":"R36","unstructured":"National Adult Cardiac Surgery Audit (NACSA) 2023 summary report. National Institute for Cardiovascular Outcomes Research (NICOR); 2023. URL: https:\/\/scts.org\/news\/570\/the_nacsa_2023_summary_report\/ [Accessed 01-08-2026]"},{"issue":"2","key":"R37","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1016\/j.jvs.2021.07.230","article-title":"Trends in operative case volumes of Canadian vascular surgery trainees","volume":"75","author":"AlHamzah","journal-title":"J Vasc Surg"},{"issue":"16","key":"R38","doi-asserted-by":"publisher","DOI":"10.1161\/JAHA.120.020513","article-title":"Representation of women in randomized trials in cardiac surgery: a meta-analysis","volume":"10","author":"Gaudino","journal-title":"J Am Heart Assoc"},{"issue":"1","key":"R39","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1016\/j.jvs.2022.12.029","article-title":"Assessing the robustness of negative vascular surgery randomized controlled trials using their reverse fragility index","volume":"78","author":"Li","journal-title":"J Vasc Surg"},{"key":"R40","unstructured":"Li A . Reverse fragility index of vascular surgery trials with statistically nonsignificant primary outcomes. Open Science Framework (OSF). 2021. URL: https:\/\/osf.io\/t8g2b\/overview [Accessed 01-08-2026]"},{"key":"R41","doi-asserted-by":"publisher","DOI":"10.1016\/j.avsg.2022.03.019","article-title":"A systematic review and bibliometric analysis of applications of artificial intelligence and machine learning in vascular surgery","volume":"85","author":"Javidan","journal-title":"Ann Vasc Surg"},{"issue":"1","key":"R42","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1016\/j.jvs.2023.05.051","article-title":"Assessing the robustness of positive vascular surgery randomized controlled trials using their fragility index","volume":"79","author":"Javidan","journal-title":"J Vasc Surg"},{"issue":"6","key":"R43","doi-asserted-by":"publisher","first-page":"1550","DOI":"10.1016\/j.jvs.2023.04.009","article-title":"Completeness of reporting in systematic reviews and meta-analyses in vascular surgery","volume":"78","author":"Javidan","journal-title":"J Vasc Surg"},{"key":"R44","unstructured":"Higgins JPT Green S . Cochrane Handbook for Systematic Reviews of Interventions. John Wiley & Sons; 2011. URL: https:\/\/www.cochrane.org\/authors\/handbooks-and-manuals\/handbook\/archive\/v5.1.0 [Accessed 01-08-2026]"},{"key":"R45","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2016.01.021","article-title":"PRESS peer review of electronic search strategies: 2015 guideline statement","volume":"75","author":"McGowan","journal-title":"J Clin Epidemiol"},{"issue":"11","key":"R46","doi-asserted-by":"publisher","first-page":"726","DOI":"10.7326\/0003-4819-152-11-201006010-00232","article-title":"CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials","volume":"152","author":"Schulz","journal-title":"Ann Intern Med"},{"issue":"1","key":"R47","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1016\/j.ijsu.2011.10.001","article-title":"CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials","volume":"10","author":"Moher","journal-title":"Int J Surg"},{"key":"R48","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.l4898","article-title":"RoB 2: a revised tool for assessing risk of bias in randomised trials","volume":"366","author":"Sterne","journal-title":"BMJ"},{"issue":"3","key":"R49","doi-asserted-by":"publisher","first-page":"872","DOI":"10.1182\/bloodadvances.2020003763","article-title":"American Society of Hematology 2021 guidelines on the use of anticoagulation for thromboprophylaxis in patients with COVID-19","volume":"5","author":"Cuker","journal-title":"Blood Adv"},{"issue":"3","key":"R50","doi-asserted-by":"publisher","first-page":"723","DOI":"10.1016\/j.jvs.2019.11.013","article-title":"Society for Vascular Surgery (SVS) and Society of Thoracic Surgeons (STS) reporting standards for type B aortic dissections","volume":"71","author":"Lombardi","journal-title":"J Vasc Surg"},{"issue":"1","key":"R51","doi-asserted-by":"publisher","first-page":"e1","DOI":"10.1016\/j.jvs.2016.03.420","article-title":"Reporting standards of the Society for Vascular Surgery for endovascular treatment of chronic lower extremity peripheral artery disease","volume":"64","author":"Stoner","journal-title":"J Vasc Surg"},{"issue":"5","key":"R52","doi-asserted-by":"publisher","first-page":"1048","DOI":"10.1067\/mva.2002.123763","article-title":"Reporting standards for endovascular aortic aneurysm repair","volume":"35","author":"Chaikof","journal-title":"J Vasc Surg"},{"issue":"3","key":"R53","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1016\/0197-2456(86)90046-2","article-title":"Meta-analysis in clinical trials","volume":"7","author":"DerSimonian","journal-title":"Control Clin Trials"},{"issue":"20","key":"R54","doi-asserted-by":"publisher","first-page":"2058","DOI":"10.1001\/jama.2010.651","article-title":"Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes","volume":"303","author":"Boutron","journal-title":"JAMA"},{"issue":"6","key":"R55","doi-asserted-by":"publisher","first-page":"622","DOI":"10.1016\/j.jclinepi.2013.10.019","article-title":"The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index","volume":"67","author":"Walsh","journal-title":"J Clin Epidemiol"},{"issue":"3","key":"R56","doi-asserted-by":"publisher","first-page":"493","DOI":"10.1053\/j.semtcvs.2022.05.010","article-title":"Reverse fragility index in negative cardiac procedural randomized controlled trials","volume":"35","author":"Dhingra","journal-title":"Semin Thorac Cardiovasc Surg"},{"issue":"1","key":"R57","doi-asserted-by":"publisher","DOI":"10.1186\/s12961-020-0539-6","article-title":"How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field","volume":"18","author":"Nguyen","journal-title":"Health Res Policy Syst"}],"container-title":["JMIR Research Protocols"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T18:55:58Z","timestamp":1791312958000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.researchprotocols.org\/2026\/1\/e93997"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,6]]},"references-count":57,"URL":"https:\/\/doi.org\/10.2196\/93997","relation":{},"ISSN":["1929-0748"],"issn-type":[{"value":"1929-0748","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,6]]},"article-number":"v15i10e93997"}}