{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T15:09:44Z","timestamp":1776438584604,"version":"3.51.2"},"reference-count":38,"publisher":"BMJ","issue":"4","license":[{"start":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T00:00:00Z","timestamp":1776384000000},"content-version":"unspecified","delay-in-days":16,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000133","name":"Agency for Healthcare Research and Quality","doi-asserted-by":"publisher","award":["75Q80120D00003"],"award-info":[{"award-number":["75Q80120D00003"]}],"id":[{"id":"10.13039\/100000133","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open"],"accepted":{"date-parts":[[2025,12,19]]},"published-print":{"date-parts":[[2026,4]]},"abstract":"\n Objectives<\/jats:title>\n In the USA, an estimated 40\u201350\u2009million operations are performed annually, with high rates of adverse events. Since the 1980s, report cards have been used for outcome measures and to improve safety of surgical care. As part of Making Healthcare Safer IV\u2014an initiative aimed at publishing evidence-based reviews as they are completed to help healthcare leaders, researchers and policymakers act more quickly on evidence-supported practices\u2014we performed an updated review on the certainty of evidence on patient safety practices related to the use of surgical report cards and outcome measurements.<\/jats:p>\n <\/jats:sec>\n \n Design<\/jats:title>\n Systematic review using the Grade of Recommendations Assessment, Development and Evaluation (GRADE) approach.<\/jats:p>\n <\/jats:sec>\n \n Data sources<\/jats:title>\n PubMed, Web of Science, Scopus and the Cochrane Library were searched from November 2011 to May 2023.<\/jats:p>\n <\/jats:sec>\n \n Eligibility criteria for selecting studies<\/jats:title>\n We included primary research studies (randomised control trials or observational studies with a comparison group, including pre\u2013post studies) or observational studies that investigated a surgical report card in adult or paediatric surgical patients at the hospital or surgeon level in inpatient or outpatient settings. Excluded studies included: narrative reviews, scoping reviews, editorials, commentaries, abstracts, studies that measured only patient knowledge or levels of engagement or studies using local surgical dashboard data.<\/jats:p>\n <\/jats:sec>\n \n Data extraction and synthesis<\/jats:title>\n Screening and eligibility were done in duplicate, while data extraction was done by one reviewer and checked by a second reviewer. Specific items in the Risk Of Bias In Non-randomised Studies - of Interventions tool and a modification of the National Institutes of Health Tool were used to assess for bias in studies. Two reviewers assessed each study for risk of bias. A modified version of the GRADE framework was used to assess the certainty of evidence.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n We identified 19 studies that met the inclusion criteria: 13 primary research studies and 6 descriptive studies of surgical collaboratives. Of the primary studies, nine used a pre\u2013post or longitudinal study design and four used a regression discontinuity or concurrent controlled design. Seven of the studies were about the American College of Surgeons National Surgical Quality Improvement Project. Five studies were from single institutions and the remainder included nine to greater than 700 hospitals. Pre\u2013post studies of report cards that prompted quality improvement (QI) programmes all reported improvements in outcomes, longitudinal studies reported benefits in some but not all outcomes and one in four controlled before-and-after studies reported a statistically significant mortality benefit. All studies, except for one, were at moderate or high risk of bias. Six collaboratives were identified with preliminary data.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Based on the above evidence, the theoretical rationale and parallel evidence in other settings, we judged that it was moderate certainty that report cards and outcomes measurements can improve surgical outcomes. However, given the evidence from studies where report cards were actively linked to institutional QI initiatives, we recommend that outcome data must be paired with actionable QI efforts to meaningfully improve patient outcomes.<\/jats:p>\n <\/jats:sec>","DOI":"10.1136\/bmjopen-2025-101718","type":"journal-article","created":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T14:25:25Z","timestamp":1776435925000},"page":"e101718","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":0,"title":["Report cards and outcome measurements to improve the safety of surgical care (North America): an updated systematic review from Making Healthcare Safer IV"],"prefix":"10.1136","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-4557-8362","authenticated-orcid":false,"given":"Tess","family":"Huy","sequence":"first","affiliation":[{"name":"UCLA Medical School, Los Angeles, California, USA"},{"name":"VA Greater Los Angeles Healthcare System, Los Angeles, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mariah B","family":"Blegen","sequence":"additional","affiliation":[{"name":"UCLA Medical School, Los Angeles, California, USA"},{"name":"VA Greater Los Angeles Healthcare System, Los Angeles, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haley","family":"Tupper","sequence":"additional","affiliation":[{"name":"UCLA Medical School, Los Angeles, California, USA"},{"name":"VA Greater Los Angeles Healthcare System, Los Angeles, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alykhan","family":"Premji","sequence":"additional","affiliation":[{"name":"UCLA Medical School, Los Angeles, California, USA"},{"name":"VA Greater Los Angeles Healthcare System, Los Angeles, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aneesa","family":"Motala","sequence":"additional","affiliation":[{"name":"RAND Corporation, Santa Monica, California, USA"},{"name":"University of Southern California, Los Angeles, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Emily","family":"Lawson","sequence":"additional","affiliation":[{"name":"RAND Corporation, Santa Monica, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2764-4610","authenticated-orcid":false,"given":"P G","family":"Shekelle","sequence":"additional","affiliation":[{"name":"VA Greater Los Angeles Healthcare System, Los Angeles, California, USA"},{"name":"RAND Corporation, Santa Monica, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mark","family":"Girgis","sequence":"additional","affiliation":[{"name":"UCLA Medical School, Los Angeles, California, USA"},{"name":"VA Greater Los Angeles Healthcare System, Los Angeles, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"M","family":"Maggard-Gibbons","sequence":"additional","affiliation":[{"name":"UCLA Medical School, Los Angeles, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"239","published-online":{"date-parts":[[2026,4,17]]},"reference":[{"key":"2026041707251437000_16.4.e101718.1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijsu.2020.07.017"},{"key":"2026041707251437000_16.4.e101718.2","doi-asserted-by":"crossref","first-page":"1340","DOI":"10.1007\/s11605-019-04228-7","article-title":"Morbidity and Mortality of Total Gastrectomy: a Comprehensive Analysis of 90-Day Outcomes","volume":"23","author":"Li","year":"2019","journal-title":"J Gastrointest Surg"},{"key":"2026041707251437000_16.4.e101718.3","doi-asserted-by":"crossref","DOI":"10.1186\/s13063-020-4167-9","article-title":"Care after pancreatic resection according to an algorithm for early detection and minimally invasive management of pancreatic fistula versus current practice (PORSCH-trial): design and rationale of a nationwide stepped-wedge cluster-randomized trial","volume":"21","author":"Smits","year":"2020","journal-title":"Trials"},{"key":"2026041707251437000_16.4.e101718.4","doi-asserted-by":"crossref","first-page":"2039","DOI":"10.1001\/jamainternmed.2013.9763","article-title":"Health care-associated infections: a meta-analysis of costs and financial impact on the US health care system","volume":"173","author":"Zimlichman","year":"2013","journal-title":"JAMA Intern Med"},{"key":"2026041707251437000_16.4.e101718.5","doi-asserted-by":"publisher","DOI":"10.1016\/j.jamcollsurg.2016.10.029"},{"key":"2026041707251437000_16.4.e101718.6","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacc.2011.12.051"},{"key":"2026041707251437000_16.4.e101718.7","doi-asserted-by":"publisher","DOI":"10.1136\/bmjqs-2013-002223"},{"key":"2026041707251437000_16.4.e101718.8","doi-asserted-by":"publisher","DOI":"10.1257\/aer.103.7.2875"},{"key":"2026041707251437000_16.4.e101718.9","doi-asserted-by":"publisher","DOI":"10.1001\/jama.1994.03510340051033"},{"key":"2026041707251437000_16.4.e101718.10","unstructured":"Shekelle PG , Wachter RM , Pronovost PJ , et al . Making health care safer II: an updated critical analysis of the evidence for patient safety practices. Evid Rep Technol Assess 2013;1\u2013945."},{"key":"2026041707251437000_16.4.e101718.11","doi-asserted-by":"crossref","unstructured":"Maggard-Gibbons M , Blegen M , Tupper H , et al . Use of Report Cards and Outcome Measurements To Improve the Safety of Surgical Care: Rapid Response. Making Healthcare Safer IV: A Continuous Updating of Patient Safety Harms and Practices. Rockville (MD), 2023.","DOI":"10.23970\/AHRQEPC_MHS4REPORTCARD"},{"key":"2026041707251437000_16.4.e101718.12","unstructured":"Agency for Healthcare Research and Quality . Making healthcare safer IV. secondary making healthcare safer IV. 2024. Available: https:\/\/www.ahrq.gov\/research\/findings\/making-healthcare-safer\/mhs4\/index.html"},{"key":"2026041707251437000_16.4.e101718.13","unstructured":"Agency for Healthcare Research and Quality . Research protocol: making healthcare safer iv: use of report cards and outcome measurements to improve the safety of surgical care. secondary research protocol: making healthcare safer iv: use of report cards and outcome measurements to improve the safety of surgical care. 2003. Available: https:\/\/effectivehealthcare.ahrq.gov\/products\/mhs4-surgical-report-cards\/protocol"},{"key":"2026041707251437000_16.4.e101718.14","unstructured":"ROBINS-I tool . Secondary ROBINS-I tool. 2022. Available: https:\/\/methods.cochrane.org\/methods-cochrane\/robins-i-tool"},{"key":"2026041707251437000_16.4.e101718.15","unstructured":"National Heart, Lung, and Blood Institute . Study quality assessment tools. Secondary Study Quality Assessment Tools; 2020. Available: https:\/\/www.nhlbi.nih.gov\/health-topics\/study-quality-assessment-tools"},{"key":"2026041707251437000_16.4.e101718.16","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1002\/jrsm.1582","article-title":"A framework for synthesizing intervention evidence from multiple sources into a single certainty of evidence rating: Methodological developments from a US National Academies of Sciences, Engineering, and Medicine Committee","volume":"14","author":"Calonge","year":"2023","journal-title":"Res Synth Methods"},{"key":"2026041707251437000_16.4.e101718.17","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.euf.2018.08.025","article-title":"Implementation of a Perioperative Venous Thromboembolism Prophylaxis Program for Patients Undergoing Radical Cystectomy on an Enhanced Recovery After Surgery Protocol","volume":"6","author":"Chiang","year":"2020","journal-title":"Eur Urol Focus"},{"key":"2026041707251437000_16.4.e101718.18","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1177\/000313481508101017","article-title":"Regional Quality Groups Enhance Effectiveness of Vascular Quality Initiative\u00ae","volume":"81","author":"Dunn","year":"2015","journal-title":"Am Surg"},{"key":"2026041707251437000_16.4.e101718.19","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1097\/JTN.0000000000000593","article-title":"Comprehensive Initiative to Decrease Trauma Venous Thromboembolism","volume":"28","author":"Johns","year":"2021","journal-title":"J Trauma Nurs"},{"key":"2026041707251437000_16.4.e101718.20","doi-asserted-by":"crossref","first-page":"896","DOI":"10.1177\/000313481408000924","article-title":"Use of the national surgical quality improvement program to reduce surgical mortality: implementation of intensive preoperative screening and intervention","volume":"80","author":"Konstantinidis","year":"2014","journal-title":"Am Surg"},{"key":"2026041707251437000_16.4.e101718.21","doi-asserted-by":"crossref","DOI":"10.1136\/tsaco-2018-000201","article-title":"Improving survival after an emergency resuscitative thoracotomy: a 5-year review of the Trauma Quality Improvement Program","volume":"3","author":"Joseph","year":"2018","journal-title":"Trauma Surg Acute Care Open"},{"key":"2026041707251437000_16.4.e101718.22","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1016\/j.surg.2013.12.001","article-title":"Quality improvement in gastrointestinal surgical oncology with American College of Surgeons National Surgical Quality Improvement Program","volume":"155","author":"Lucas","year":"2014","journal-title":"Surgery"},{"key":"2026041707251437000_16.4.e101718.23","doi-asserted-by":"crossref","first-page":"e1752","DOI":"10.7205\/MILMED-D-16-00220","article-title":"Improving Surgical Complications and Patient Safety at the Nation\u2019s Largest Military Hospital: An Analysis of National Surgical Quality Improvement Program Data","volume":"182","author":"Maturo","year":"2017","journal-title":"Mil Med"},{"key":"2026041707251437000_16.4.e101718.24","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2011.06.052"},{"key":"2026041707251437000_16.4.e101718.25","first-page":"1447","article-title":"American College of Surgeons\u2019 Committee on Trauma Performance Improvement and Patient Safety Program: Maximal Impact in a Mature Trauma Center","volume":"71","author":"Sarkar","year":"2011","journal-title":"J Trauma"},{"key":"2026041707251437000_16.4.e101718.26","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2015.90"},{"key":"2026041707251437000_16.4.e101718.27","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1001\/jamasurg.2013.3977","article-title":"Effectiveness of nonpublic report cards for reducing trauma mortality","volume":"149","author":"Glance","year":"2014","journal-title":"JAMA Surg"},{"key":"2026041707251437000_16.4.e101718.28","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1001\/jamasurg.2018.0985","article-title":"Association of Hospital Participation in a Regional Trauma Quality Improvement Collaborative With Patient Outcomes","volume":"153","author":"Hemmila","year":"2018","journal-title":"JAMA Surg"},{"key":"2026041707251437000_16.4.e101718.29","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2015.25"},{"key":"2026041707251437000_16.4.e101718.30","doi-asserted-by":"crossref","first-page":"S106","DOI":"10.1097\/BRS.0000000000000579","article-title":"The National Neurosurgery Quality and Outcomes Database (N2QOD): a collaborative North American outcomes registry to advance value-based spine care","volume":"39","author":"Asher","year":"1976","journal-title":"Spine (Phila Pa 1986)"},{"key":"2026041707251437000_16.4.e101718.31","doi-asserted-by":"crossref","DOI":"10.3171\/2015.10.FOCUS15370","article-title":"The Michigan Spine Surgery Improvement Collaborative: a statewide Collaborative Quality Initiative","volume":"39","author":"Chang","year":"2015","journal-title":"Neurosurg Focus"},{"key":"2026041707251437000_16.4.e101718.32","doi-asserted-by":"crossref","unstructured":"Davis CH , Kao LS , Fleming JB , et al . Multi-Institution Analysis of Infection Control Practices Identifies the Subset Associated with Best Surgical Site Infection Performance: A Texas Alliance for Surgical Quality Collaborative Project. J Am Coll Surg 2017. doi:10.1016\/j.jamcollsurg.2017.07.1054","DOI":"10.1016\/j.jamcollsurg.2017.07.824"},{"key":"2026041707251437000_16.4.e101718.33","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.aorn.2016.06.017","article-title":"Outcome Management in Cardiac Surgery Using the Society of Thoracic Surgeons National Database","volume":"104","author":"Halpin","year":"2016","journal-title":"Aorn j"},{"key":"2026041707251437000_16.4.e101718.34","doi-asserted-by":"publisher","DOI":"10.1007\/s10029-016-1477-7"},{"key":"2026041707251437000_16.4.e101718.35","doi-asserted-by":"publisher","DOI":"10.9778\/cmajo.20170166"},{"key":"2026041707251437000_16.4.e101718.36","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0146254","article-title":"Change in Adverse Events After Enrollment in the National Surgical Quality Improvement Program: A Systematic Review and Meta-Analysis","volume":"11","author":"Montroy","year":"2016","journal-title":"PLoS ONE"},{"key":"2026041707251437000_16.4.e101718.37","doi-asserted-by":"publisher","DOI":"10.1136\/bmjqs-2018-008797"},{"key":"2026041707251437000_16.4.e101718.38","doi-asserted-by":"crossref","first-page":"636","DOI":"10.1016\/j.jamcollsurg.2013.04.040","article-title":"Surgeon-Specific Performance Reports in General Surgery: An Observational Study of Initial Implementation and Adoption","volume":"217","author":"Yi","year":"2013","journal-title":"J Am Coll Surg"}],"container-title":["BMJ Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjopen-2025-101718","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T14:25:34Z","timestamp":1776435934000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjopen.bmj.com\/lookup\/doi\/10.1136\/bmjopen-2025-101718"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4]]},"references-count":38,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4,17]]},"published-print":{"date-parts":[[2026,4]]}},"alternative-id":["10.1136\/bmjopen-2025-101718"],"URL":"https:\/\/doi.org\/10.1136\/bmjopen-2025-101718","relation":{},"ISSN":["2044-6055","2044-6055"],"issn-type":[{"value":"2044-6055","type":"print"},{"value":"2044-6055","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4]]}}}