{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T18:27:03Z","timestamp":1793730423306,"version":"4.6.1"},"reference-count":33,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2016,12]]},"abstract":"<jats:sec>\n            <jats:title>Background:<\/jats:title>\n            <jats:p>The Agency for Health Care Research and Quality Patient Safety Indicators (PSIs) and Centers for Medicare and Medicaid Services Hospital-acquired Conditions (HACs) are increasingly being used for pay-for-performance and public reporting despite concerns over their validity. Given the potential for these measures to misinform patients, misclassify hospitals, and misapply financial and reputational harm to hospitals, these need to be rigorously evaluated. We performed a systematic review and meta-analysis to assess PSI and HAC measure validity.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods:<\/jats:title>\n            <jats:p>We searched MEDLINE and the gray literature from January 1, 1990 through January 14, 2015 for studies that addressed the validity of the HAC measures and PSIs. Secondary outcomes included the effects of present on admission (POA) modifiers, and the most common reasons for discrepancies. We developed pooled results for measures evaluated by \u22653 studies. We propose a threshold of 80% for positive predictive value or sensitivity for pay-for-performance and public reporting suitability.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results:<\/jats:title>\n            <jats:p>Only 5 measures, Iatrogenic Pneumothorax (PSI 6\/HAC 17), Central Line\u2013associated Bloodstream Infections (PSI 7), Postoperative hemorrhage\/hematoma (PSI 9), Postoperative deep vein thrombosis\/pulmonary embolus (PSI 12), and Accidental Puncture\/Laceration (PSI 15), had sufficient data for pooled meta-analysis. Only PSI 15 (Accidental Puncture and Laceration) met our proposed threshold for validity (positive predictive value only) but this result was weakened by considerable heterogeneity. Coding errors were the most common reasons for discrepancies between medical record review and administrative databases. POA modifiers may improve the validity of some measures.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion:<\/jats:title>\n            <jats:p>This systematic review finds that there is limited validity for the PSI and HAC measures when measured against the reference standard of a medical chart review. Their use, as they currently exist, for public reporting and pay-for-performance, should be publicly reevaluated in light of these findings.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1097\/mlr.0000000000000550","type":"journal-article","created":{"date-parts":[[2016,4,26]],"date-time":"2016-04-26T13:06:37Z","timestamp":1461675997000},"page":"1105-1111","source":"Crossref","is-referenced-by-count":92,"title":["Validity of the Agency for Health Care Research and Quality Patient Safety Indicators and the Centers for Medicare and Medicaid Hospital-acquired Conditions"],"prefix":"10.1097","volume":"54","author":[{"given":"Bradford D.","family":"Winters","sequence":"first","affiliation":[{"name":"Johns Hopkins School of Medicine Armstrong Institute for Patient Safety and Quality, Baltimore, MD"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aamir","family":"Bharmal","sequence":"additional","affiliation":[{"name":"Health Policy and Management, Johns Hopkins School of Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Renee F.","family":"Wilson","sequence":"additional","affiliation":[{"name":"Health Policy and Management, Johns Hopkins School of Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Allen","family":"Zhang","sequence":"additional","affiliation":[{"name":"Health Policy and Management, Johns Hopkins School of Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lilly","family":"Engineer","sequence":"additional","affiliation":[{"name":"Anesthesiology and Critical Care Medicine, Johns Hopkins School of Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deidre","family":"Defoe","sequence":"additional","affiliation":[{"name":"Johns Hopkins University Bloomberg School of Public Health"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric B.","family":"Bass","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Johns Hopkins School of Medicine, Baltimore, MD"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sydney","family":"Dy","sequence":"additional","affiliation":[{"name":"Health Policy and Management, Johns Hopkins School of Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter J.","family":"Pronovost","sequence":"additional","affiliation":[{"name":"Johns Hopkins School of Medicine Armstrong Institute for Patient Safety and Quality, Baltimore, MD"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","reference":[{"issue":"pt 2","key":"R1-20230821","first-page":"110","article-title":"Patient Safety Indicators: using administrative data to identify potential patient safety concerns","volume":"36","author":"Miller","year":"2001","journal-title":"Health Serv Res"},{"key":"R2-20230821","doi-asserted-by":"crossref","first-page":"700","DOI":"10.1097\/00005650-199407000-00004","article-title":"Identifying complications of care using administrative data","volume":"32","author":"Iezzoni","year":"1994","journal-title":"Med Care"},{"key":"R3-20230821","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1097\/MLR.0b013e31818af83d","article-title":"Identification of hospital-acquired catheter-associated urinary tract infections from Medicare claims: sensitivity and positive predictive value","volume":"47","author":"Zhan","year":"2009","journal-title":"Med Care"},{"key":"R6-20230821","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1001\/jama.2015.52","article-title":"Concerns about using the patient safety indicator-90 composite in pay-for-performance programs","volume":"313","author":"Rajaram","year":"2015","journal-title":"JAMA"},{"key":"R7-20230821","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1377\/hlthaff.2014.0201","article-title":"National hospital ratings systems share few common scores and may generate confusion instead of clarity","volume":"34","author":"Austin","year":"2015","journal-title":"Health Aff"},{"key":"R8-20230821","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1016\/j.jamcollsurg.2010.07.007","article-title":"Validity of selected patient safety indicators: opportunities and concerns","volume":"212","author":"Kaafarani","year":"2011","journal-title":"J Am Coll Surg"},{"key":"R9-20230821","doi-asserted-by":"crossref","first-page":"1117","DOI":"10.1016\/j.surg.2013.04.062","article-title":"Detecting postoperative hemorrhage or hematoma from administrative data: the performance of the AHRQ Patient Safety Indicator","volume":"154","author":"Utter","year":"2013","journal-title":"Surgery"},{"key":"R10-20230821","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1111\/j.1475-6773.2008.00905.x","article-title":"Validity of selected AHRQ patient safety indicators based on VA national surgical quality improvement program data","volume":"44","author":"Romano","year":"2009","journal-title":"Health Serv Res"},{"key":"R11-20230821","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1097\/01.mlr.0000173561.79742.fb","article-title":"Evaluating the patient safety indicators: how well do they perform on Veterans Health Administration data?","volume":"43","author":"Rosen","year":"2005","journal-title":"Med Care"},{"key":"R12-20230821","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1177\/000313481307900617","article-title":"Validity of Agency for Healthcare Research and Quality Patient Safety Indicators at an academic medical center","volume":"79","author":"Ramanathan","year":"2013","journal-title":"Am Surg"},{"key":"R13-20230821","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1097\/MLR.0b013e3182293edf","article-title":"Validating the patient safety indicators in the veterans health administration","volume":"50","author":"Rosen","year":"2012","journal-title":"Med Care"},{"key":"R14-20230821","first-page":"20","article-title":"Designing an abstraction instrument: lessons from efforts to validate the AHRQ patient safety indicators","volume":"37","author":"Utter","year":"2011","journal-title":"Jt Comm J Qual Patient Saf"},{"key":"R16-20230821","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/0197-2456(86)90046-2","article-title":"Meta-analysis in clinical trials","volume":"7","author":"DerSimonian","year":"1986","journal-title":"Control Clin Trials"},{"key":"R17-20230821","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1136\/bmj.327.7414.557","article-title":"Measuring inconsistency in meta-analyses","volume":"327","author":"Higgins","year":"2003","journal-title":"BMJ"},{"key":"R18-20230821","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1056\/NEJM198005153022003","article-title":"The threshold approach to clinical decision making","volume":"302","author":"Pauker","year":"1980","journal-title":"N Engl J Med"},{"key":"R19-20230821","first-page":"8","article-title":"Hospital margins slump due to squeeze from volume, rates, investments","volume":"44","author":"Kutscher","year":"2014","journal-title":"Mod Healthc"},{"key":"R20-20230821","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1111\/j.1945-1474.2010.00114.x","article-title":"Positive predictive value of the Agency for Healthcare Research and Quality Patient Safety Indicator for central line-related bloodstream infection (\u201cselected infections due to medical care\u201d)","volume":"33","author":"Zrelak","year":"2011","journal-title":"J Healthc Qual"},{"key":"R21-20230821","doi-asserted-by":"crossref","first-page":"984","DOI":"10.1016\/j.jamcollsurg.2011.02.005","article-title":"Validity of the AHRQ Patient Safety Indicator \u201ccentral venous catheter-related bloodstream infections\u201d","volume":"212","author":"Cevasco","year":"2011","journal-title":"J Am Coll Surg"},{"key":"R22-20230821","doi-asserted-by":"crossref","first-page":"946","DOI":"10.1016\/j.jamcollsurg.2010.09.033","article-title":"How valid is the AHRQ Patient Safety Indicator \u201cpostoperative hemorrhage or hematoma\u201d?","volume":"212","author":"Borzecki","year":"2011","journal-title":"J Am Coll Surg"},{"key":"R23-20230821","doi-asserted-by":"crossref","first-page":"1237","DOI":"10.1097\/MLR.0b013e3181b58940","article-title":"How valid is the ICD-9-CM based AHRQ patient safety indicator for postoperative venous thromboembolism?","volume":"47","author":"White","year":"2009","journal-title":"Med Care"},{"key":"R24-20230821","doi-asserted-by":"crossref","first-page":"1041","DOI":"10.1097\/SLA.0b013e3181afe095","article-title":"Positive predictive value of the AHRQ accidental puncture or laceration patient safety indicator","volume":"250","author":"Utter","year":"2009","journal-title":"Ann Surg"},{"key":"R25-20230821","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1177\/0272989X14528381","article-title":"\u201cIt\u2019s valid and reliable\u201d is not enough: critical appraisal of reporting of measures in trials evaluating patient decision aids","volume":"34","author":"Sepucha","year":"2014","journal-title":"Med Decis Making"},{"key":"R27-20230821","first-page":"848","article-title":"Automated identification of postoperative complications within an electronic medical record using natural language processing","volume":"306","author":"Murff","year":"2011","journal-title":"JAMA"},{"key":"R28-20230821","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1016\/j.jamcollsurg.2010.11.013","article-title":"Positive predictive value of the AHRq patient safety indicator \u201cpostoperative sepsis\u201d: implications for practice and policy","volume":"212","author":"Cevasco","year":"2011","journal-title":"J Am Coll Surg"},{"key":"R29-20230821","doi-asserted-by":"crossref","first-page":"962","DOI":"10.1016\/j.jamcollsurg.2011.01.053","article-title":"Positive predictive value of the AHRQ Patient Safety Indicator \u201cpostoperative wound dehiscence\u201d","volume":"212","author":"Cevasco","year":"2011","journal-title":"J Am Coll Surg"},{"key":"R30-20230821","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1016\/j.jamcollsurg.2010.04.022","article-title":"Detection of postoperative respiratory failure: how predictive is the Agency for Healthcare Research and Quality\u2019s Patient Safety Indicator?","volume":"211","author":"Utter","year":"2010","journal-title":"J Am Coll Surg"},{"key":"R31-20230821","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1097\/MLR.0b013e31829c8b91","article-title":"Validity of the AHRQ patient safety indicator for postoperative physiologic and metabolic derangement based on a national sample of medical records","volume":"51","author":"Zrelak","year":"2013","journal-title":"Med Care"},{"key":"R32-20230821","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1177\/1062860609359534","article-title":"Cases of iatrogenic pneumothorax can be identified from ICD-9-CM coded data","volume":"25","author":"Sadeghi","year":"2010","journal-title":"Am J Med Qual"},{"key":"R33-20230821","first-page":"326","article-title":"The validity of ICD-9-CM codes in identifying postoperative deep vein thrombosis and pulmonary embolism","volume":"33","author":"Zhan","year":"2007","journal-title":"Jt Comm J Qual Patient Saf"},{"key":"R34-20230821","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1016\/j.jclinepi.2003.01.002","article-title":"Assessing accuracy of diagnosis-type indicators for flagging complications in administrative data","volume":"57","author":"Quan","year":"2004","journal-title":"J Clin Epidemiol"},{"key":"R35-20230821","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1086\/652523","article-title":"Hospital-acquired catheter-associated urinary tract infection: documentation and coding issues may reduce financial impact of Medicare\u2019s new payment policy","volume":"31","author":"Meddings","year":"2010","journal-title":"Infect Control Hosp Epidemiol"},{"key":"R36-20230821","doi-asserted-by":"crossref","first-page":"812","DOI":"10.1086\/518455","article-title":"Validation of coronary artery bypass graft surgical site infection surveillance data from a statewide surveillance system in Australia","volume":"28","author":"Friedman","year":"2007","journal-title":"Infect Control Hosp Epidemiol"},{"key":"R37-20230821","first-page":"77","article-title":"An evaluation of the FASB\u2019s coneptual framework from a user\u2019s perspective","volume":"16","author":"Pike","year":"2012","journal-title":"Acad Account Financ Stud J"}],"updated-by":[{"DOI":"10.1097\/mlr.0000000000000674","type":"correction","label":"Correction","source":"retraction-watch","updated":{"date-parts":[[2016,12,1]],"date-time":"2016-12-01T00:00:00Z","timestamp":1480550400000},"record-id":"8533"}],"container-title":["Medical Care"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.lww.com\/00005650-201612000-00014","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,22]],"date-time":"2023-08-22T04:13:33Z","timestamp":1692677613000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.lww.com\/00005650-201612000-00014"}},"subtitle":["A Systematic Review and Meta-Analysis"],"short-title":[],"issued":{"date-parts":[[2016,12]]},"references-count":33,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2016]]}},"URL":"https:\/\/doi.org\/10.1097\/mlr.0000000000000550","relation":{},"ISSN":["0025-7079"],"issn-type":[{"value":"0025-7079","type":"print"}],"subject":[],"published":{"date-parts":[[2016,12]]}}}