{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T15:51:13Z","timestamp":1770565873308,"version":"3.49.0"},"reference-count":22,"publisher":"Oxford University Press (OUP)","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,3,1]]},"abstract":"<jats:p>Objective: Agency for Healthcare Research and Quality (AHRQ) software applies standardized algorithms to hospital administrative data to identify patient safety indicators (PSIs). The objective of this study was to assess the validity of PSI flags and report reasons for invalid flagging.<\/jats:p><jats:p>Material and Methods: At a 6-hospital academic medical center, a retrospective analysis was conducted of all\u00a0PSIs flagged in fiscal year 2014. A multidisciplinary PSI Quality Team reviewed each flagged PSI based on quarterly reports. The positive predictive value (PPV, the percent of clinically validated cases) was calculated for 12 PSI categories. The documentation for each reversed case was reviewed to determine the reasons for PSI reversal.<\/jats:p><jats:p>Results: Of 657 PSI flags, 185 were reversed. Seven PSI categories had a PPV below 75%. Four broad categories of reasons for reversal were AHRQ algorithm limitations (38%), coding misinterpretations (45%), present upon admission (10%), and documentation insufficiency (7%). AHRQ algorithm limitations included 2 subcategories: an \u201cincident\u201d was inherent to the procedure, or highly likely (eg, vascular tumor bleed), or an \u201cincident\u201d was nonsignificant, easily controlled, and\/or no intervention was needed.<\/jats:p><jats:p>Discussion: These findings support previous research highlighting administrative data problems. Additionally, AHRQ algorithm limitations was an emergent category not considered in previous research. Herein we present potential solutions to address these issues.<\/jats:p><jats:p>Conclusions: If, despite poor validity, US policy continues to rely on PSIs for incentive and penalty programs, improvements are needed in the quality of administrative data and the standardized PSI algorithms. These solutions require national motivation, research attention, and dissemination support.<\/jats:p>","DOI":"10.1093\/jamia\/ocw126","type":"journal-article","created":{"date-parts":[[2016,8,31]],"date-time":"2016-08-31T00:54:19Z","timestamp":1472604859000},"page":"310-315","source":"Crossref","is-referenced-by-count":8,"title":["Navigating a ship with a broken compass: evaluating standard algorithms to measure patient safety"],"prefix":"10.1093","volume":"24","author":[{"given":"Jennifer L","family":"Hefner","sequence":"first","affiliation":[{"name":"Department of Family Medicine, College of Medicine, The Ohio State University, Columbus, Ohio, USA"}]},{"given":"Timothy R","family":"Huerta","sequence":"additional","affiliation":[{"name":"Department of Family Medicine, College of Medicine, The Ohio State University, Columbus, Ohio, USA"},{"name":"Department of Biomedical Informatics, College of Medicine, Ohio State University"}]},{"given":"Ann Scheck","family":"McAlearney","sequence":"additional","affiliation":[{"name":"Department of Family Medicine, College of Medicine, The Ohio State University, Columbus, Ohio, USA"},{"name":"Division of Health Services Management and Policy, College of Public Health, The Ohio State University"}]},{"given":"Barbara","family":"Barash","sequence":"additional","affiliation":[{"name":"Department of Family Medicine, College of Medicine, The Ohio State University, Columbus, Ohio, USA"}]},{"given":"Tina","family":"Latimer","sequence":"additional","affiliation":[{"name":"Quality and Operations, The Ohio State University Wexner Medical Center"}]},{"given":"Susan D","family":"Moffatt-Bruce","sequence":"additional","affiliation":[{"name":"Quality and Operations, The Ohio State University Wexner Medical Center"},{"name":"Department of Surgery, The Ohio State University Wexner Medical Center"}]}],"member":"286","published-online":{"date-parts":[[2016,8,30]]},"reference":[{"issue":"8 Pt 2","key":"2020110612434959000_ocw126-B1","doi-asserted-by":"crossref","first-page":"666","DOI":"10.7326\/0003-4819-127-8_Part_2-199710151-00048","article-title":"Assessing quality using administrative data","volume":"127","author":"Iezzoni","year":"1997","journal-title":"Ann Int Med"},{"issue":"3","key":"2020110612434959000_ocw126-B2","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/S1553-7250(08)34018-5","article-title":"How often are potential patient safety events present on admission?","volume":"34","author":"Houchens","year":"2008","journal-title":"Jt CommJ Qual Patient Safety"},{"issue":"6","key":"2020110612434959000_ocw126-B3","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1016\/j.jamcollsurg.2010.07.007","article-title":"Validity of selected Patient Safety Indicators: opportunities and concerns","volume":"212","author":"Kaafarani","year":"2011","journal-title":"J Am College Surgeons"},{"issue":"5","key":"2020110612434959000_ocw126-B4","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1016\/j.surg.2011.06.020","article-title":"How best to measure surgical quality? Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI) and the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) postoperative adverse events at a single institution","volume":"150","author":"Cima","year":"2011","journal-title":"Surgery"},{"key":"2020110612434959000_ocw126-B5","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1186\/1472-6963-6-77","article-title":"Identifying priorities in methodological research using ICD-9-CM and ICD-10 administrative data: report from an international consortium","volume":"6","author":"De Coster","year":"2006","journal-title":"BMC Health Services Res"},{"issue":"1","key":"2020110612434959000_ocw126-B6","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1093\/oxfordjournals.pubmed.a024721","article-title":"Assessment of the reproducibility of clinical coding in routinely collected hospital activity data: a study in two hospitals","volume":"20","author":"Dixon","year":"1998","journal-title":"J Public Health Med"},{"issue":"6 Pt1","key":"2020110612434959000_ocw126-B7","doi-asserted-by":"crossref","first-page":"1946","DOI":"10.1111\/j.1475-6773.2011.01300.x","article-title":"The accuracy of present-on-admission reporting in administrative data","volume":"46","author":"Goldman","year":"2011","journal-title":"Health Services Res"},{"issue":"11","key":"2020110612434959000_ocw126-B8","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1097\/01.mlr.0000228018.48783.34","article-title":"Quality of diagnosis and procedure coding in ICD-10 administrative data","volume":"44","author":"Henderson","year":"2006","journal-title":"Med Care"},{"issue":"4","key":"2020110612434959000_ocw126-B9","doi-asserted-by":"crossref","first-page":"1424","DOI":"10.1111\/j.1475-6773.2007.00822.x","article-title":"Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database","volume":"43","author":"Quan","year":"2008","journal-title":"Health Services Res"},{"issue":"9","key":"2020110612434959000_ocw126-B10","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1001\/jama.2015.52","article-title":"Concerns about using the patient safety indicator 90 composite in pay-for-performance programs","volume":"313","author":"Rajaram","year":"2015","journal-title":"JAMA"},{"issue":"5","key":"2020110612434959000_ocw126-B11","doi-asserted-by":"crossref","first-page":"1426","DOI":"10.1111\/1475-6773.12180","article-title":"Using estimated true safety event rates versus flagged safety event rates: does it change hospital profiling and payment?","volume":"49","author":"Rosen","year":"2014","journal-title":"Health Services Res"},{"issue":"2","key":"2020110612434959000_ocw126-B12","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1136\/jamia.2009.002451","article-title":"The impact of electronic medical records data sources on an adverse drug event quality measure","volume":"17","author":"Kahn","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2020110612434959000_ocw126-B13","article-title":"Patient Safety Indicators Technical Specifications","author":"AHRQ"},{"issue":"6","key":"2020110612434959000_ocw126-B14","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1177\/000313481307900617","article-title":"Validity of Agency for Healthcare Research and Quality Patient Safety Indicators at an academic medical center","volume":"79","author":"Ramanathan","year":"2013","journal-title":"Am Surg"},{"issue":"1","key":"2020110612434959000_ocw126-B15","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1097\/MLR.0b013e3182293edf","article-title":"Validating the patient safety indicators in the Veterans Health Administration: do they accurately identify true safety events?","volume":"50","author":"Rosen","year":"2012","journal-title":"Med Care"},{"issue":"12","key":"2020110612434959000_ocw126-B16","doi-asserted-by":"crossref","first-page":"1237","DOI":"10.1097\/MLR.0b013e3181b58940","article-title":"How valid is the ICD-9-CM based AHRQ patient safety indicator for postoperative venous thromboembolism?","volume":"47","author":"White","year":"2009","journal-title":"Med Care"},{"issue":"2","key":"2020110612434959000_ocw126-B17","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1177\/1062860611414697","article-title":"A review of patient safety measures based on routinely collected hospital data","volume":"27","author":"Tsang","year":"2012","journal-title":"Am J Med Qual"},{"issue":"10","key":"2020110612434959000_ocw126-B18","doi-asserted-by":"crossref","first-page":"e003716","DOI":"10.1136\/bmjopen-2013-003716","article-title":"Validity of AHRQ patient safety indicators derived from ICD-10 hospital discharge abstract data (chart review study)","volume":"3","author":"Quan","year":"2013","journal-title":"BMJ Open"},{"issue":"1","key":"2020110612434959000_ocw126-B19","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1136\/amiajnl-2013-002491","article-title":"Challenges and remediation for Patient Safety Indicators in the transition to ICD-10-CM","volume":"22","author":"Boyd","year":"2015","journal-title":"J Am Med Inform Assoc"},{"key":"2020110612434959000_ocw126-B20","volume-title":"Quality Indicators Software Instructions","author":"AHRQ"},{"issue":"1","key":"2020110612434959000_ocw126-B21","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1097\/MLR.0b013e318270c0f7","article-title":"Examining the impact of the AHRQ Patient Safety Indicators (PSIs) on the Veterans Health Administration: the case of readmissions","volume":"51","author":"Rosen","year":"2013","journal-title":"Med Care"},{"issue":"4","key":"2020110612434959000_ocw126-B22","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1177\/1062860613499402","article-title":"Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicator rates among veteran dual users","volume":"29","author":"Chen","year":"2013","journal-title":"Am J Med Qual"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/2\/310\/34149261\/ocw126.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/2\/310\/34149261\/ocw126.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T12:20:21Z","timestamp":1657196421000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/24\/2\/310\/2631510"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,8,30]]},"references-count":22,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2016,8,30]]},"published-print":{"date-parts":[[2017,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocw126","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2017,3]]},"published":{"date-parts":[[2016,8,30]]}}}