{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T02:31:48Z","timestamp":1773455508969,"version":"3.50.1"},"reference-count":25,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2020,7,4]],"date-time":"2020-07-04T00:00:00Z","timestamp":1593820800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000133","name":"Agency for Healthcare Research and Quality","doi-asserted-by":"publisher","award":["R01HS023696"],"award-info":[{"award-number":["R01HS023696"]}],"id":[{"id":"10.13039\/100000133","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>The study sought to evaluate the overall performance of hospitals that used the Computerized Physician Order Entry Evaluation Tool in both 2017 and 2018, along with their performance against fatal orders and nuisance orders.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We evaluated 1599 hospitals that took the test in both 2017 and 2018 by using their overall percentage scores on the test, along with the percentage of fatal orders appropriately alerted on, and the percentage of nuisance orders incorrectly alerted on.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Hospitals showed overall improvement; the mean score in 2017 was 58.1%, and this increased to 66.2% in 2018. Fatal order performance improved slightly from 78.8% to 83.0% (P\u2009&amp;lt;\u2009.001), though there was almost no change in nuisance order performance (89.0% to 89.7%; P\u2009=\u2009.43). Hospitals alerting on one or more nuisance orders had a 3-percentage-point increase in their overall score.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Despite the improvement of overall scores in 2017 and 2018, there was little improvement in fatal order performance, suggesting that hospitals are not targeting the deadliest orders first. Nuisance order performance showed almost no improvement, and some hospitals may be achieving higher scores by overalerting, suggesting that the thresholds for which alerts are fired from are too low.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Although hospitals improved overall from 2017 to 2018, there is still important room for improvement for both fatal and nuisance orders. Hospitals that incorrectly alerted on one or more nuisance orders had slightly higher overall performance, suggesting that some hospitals may be achieving higher scores at the cost of overalerting, which has the potential to cause clinician burnout and even worsen safety.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaa098","type":"journal-article","created":{"date-parts":[[2020,5,8]],"date-time":"2020-05-08T11:12:26Z","timestamp":1588936346000},"page":"1252-1258","source":"Crossref","is-referenced-by-count":32,"title":["The tradeoffs between safety and alert fatigue: Data from a national evaluation of hospital medication-related clinical decision support"],"prefix":"10.1093","volume":"27","author":[{"given":"Zoe","family":"Co","sequence":"first","affiliation":[{"name":"Department of General Internal Medicine, Brigham and Women\u2019s Hospital, Boston, Massachusetts, USA"}]},{"given":"A Jay","family":"Holmgren","sequence":"additional","affiliation":[{"name":"Harvard Business School, Harvard University, Boston, Massachusetts, USA"}]},{"given":"David C","family":"Classen","sequence":"additional","affiliation":[{"name":"Division of Clinical Epidemiology, University of Utah, Salt Lake City, Utah, USA"}]},{"given":"Lisa","family":"Newmark","sequence":"additional","affiliation":[{"name":"Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, USA"}]},{"given":"Diane L","family":"Seger","sequence":"additional","affiliation":[{"name":"Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, USA"}]},{"given":"Melissa","family":"Danforth","sequence":"additional","affiliation":[{"name":"Leapfrog Group, Washington DC, USA"}]},{"given":"David W","family":"Bates","sequence":"additional","affiliation":[{"name":"Department of General Internal Medicine, Brigham and Women\u2019s Hospital, Boston, Massachusetts, USA"},{"name":"Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"}]}],"member":"286","published-online":{"date-parts":[[2020,7,4]]},"reference":[{"issue":"8","key":"2020110613074369700_ocaa098-B1","doi-asserted-by":"crossref","first-page":"1416","DOI":"10.1377\/hlthaff.2016.1651","article-title":"HITECH Act drove large gains in hospital electronic health record adoption","volume":"36","author":"Adler-Milstein","year":"2017","journal-title":"Health Aff (Millwood)"},{"issue":"1","key":"2020110613074369700_ocaa098-B2","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1197\/jamia.M2170","article-title":"Medication-related clinical decision support in computerized provider order entry systems: a review","volume":"14","author":"Kuperman","year":"2007","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"2020110613074369700_ocaa098-B3","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1136\/jamia.1999.00660313","article-title":"The impact of computerized physician order entry on medication error prevention","volume":"6","author":"Bates","year":"2007","journal-title":"J Am Med Inform Assoc"},{"issue":"e1","key":"2020110613074369700_ocaa098-B4","doi-asserted-by":"crossref","first-page":"e85","DOI":"10.1136\/amiajnl-2012-001549","article-title":"Relationship between medication event rates and the Leapfrog computerized physician order entry evaluation tool","volume":"20","author":"Leung","year":"2013","journal-title":"J Am Med Informatics Assoc"},{"issue":"15","key":"2020110613074369700_ocaa098-B5","doi-asserted-by":"crossref","first-page":"1311","DOI":"10.1001\/jama.280.15.1311","article-title":"Effect of computerized physician order entry and a team intervention on prevention of serious medication errors","volume":"280","author":"Bates","year":"1998","journal-title":"JAMA"},{"issue":"3","key":"2020110613074369700_ocaa098-B6","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1136\/amiajnl-2012-001241","article-title":"Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems","volume":"20","author":"Radley","year":"2013","journal-title":"J Am Med Inform Assoc"},{"issue":"10","key":"2020110613074369700_ocaa098-B7","doi-asserted-by":"crossref","first-page":"1197","DOI":"10.1001\/jama.293.10.1197","article-title":"Role of computerized physician order entry systems in facilitating medication errors","volume":"293","author":"Koppel","year":"2005","journal-title":"JAMA"},{"key":"2020110613074369700_ocaa098-B8","author":"Chou"},{"issue":"4","key":"2020110613074369700_ocaa098-B9","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1136\/bmjqs-2014-003555","article-title":"Computerised physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems","volume":"24","author":"Schiff","year":"2015","journal-title":"BMJ Qual Saf"},{"issue":"4","key":"2020110613074369700_ocaa098-B10","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1197\/jamia.M2373","article-title":"The extent and importance of unintended consequences related to computerized provider order entry","volume":"14","author":"Ash","year":"2007","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2020110613074369700_ocaa098-B11","doi-asserted-by":"crossref","first-page":"686","DOI":"10.4338\/ACI-2017-01-RA-0003","article-title":"Electronic health record alert-related workload as a predictor of burnout in primary care providers","volume":"8","author":"Gregory","year":"2017","journal-title":"Appl Clin Inform"},{"issue":"5","key":"2020110613074369700_ocaa098-B12","doi-asserted-by":"crossref","first-page":"496","DOI":"10.1093\/jamia\/ocx106","article-title":"Clinical decision support alert malfunctions: analysis and empirically derived taxonomy","volume":"25","author":"Wright","year":"2018","journal-title":"J Am Med Informatics Assoc"},{"issue":"2","key":"2020110613074369700_ocaa098-B13","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1136\/qshc.2005.014969","article-title":"Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems","volume":"15","author":"Kilbridge","year":"2006","journal-title":"Qual Saf Health Care"},{"issue":"1","key":"2020110613074369700_ocaa098-B14","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1197\/jamia.M2248","article-title":"Evaluation and certification of computerized provider order entry systems","volume":"14","author":"Classen","year":"2007","journal-title":"J Am Med Informatics Assoc"},{"issue":"4","key":"2020110613074369700_ocaa098-B15","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1377\/hlthaff.2010.0160","article-title":"Mixed results in the safety performance of computerized physician order entry","volume":"29","author":"Metzger","year":"2010","journal-title":"Health Aff (Millwood)"},{"issue":"2","key":"2020110613074369700_ocaa098-B16","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1093\/jamia\/ocw134","article-title":"National trends in safety performance of electronic health record systems in children\u2019s hospitals","volume":"24","author":"Chaparro","year":"2016","journal-title":"J Am Med Informatics Assoc"},{"key":"2020110613074369700_ocaa098-B17","year":"2019"},{"key":"2020110613074369700_ocaa098-B18","year":"2019"},{"issue":"1","key":"2020110613074369700_ocaa098-B19","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1136\/bmjqs-2019-009609","article-title":"Assessing the safety of electronic health records: a national longitudinal study of medication-related decision support","volume":"29","author":"Holmgren","year":"2020","journal-title":"BMJ Qual Saf"},{"issue":"3","key":"2020110613074369700_ocaa098-B20","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1136\/amiajnl-2012-001089","article-title":"Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records","volume":"20","author":"Phansalkar","year":"2013","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"2020110613074369700_ocaa098-B21","doi-asserted-by":"crossref","first-page":"1142","DOI":"10.1093\/jamia\/ocx080","article-title":"Electronic health record adoption in US hospitals: the emergence of a digital \u201cadvanced use\u201d divide","volume":"24","author":"Adler-Milstein","year":"2017","journal-title":"J Am Med Informatics Assoc"},{"issue":"21","key":"2020110613074369700_ocaa098-B22","doi-asserted-by":"crossref","first-page":"2625","DOI":"10.1001\/archinte.163.21.2625","article-title":"Physicians\u2019 decisions to override computerized drug alerts in primary care","volume":"163","author":"Weingart","year":"2003","journal-title":"Arch Intern Med"},{"issue":"1","key":"2020110613074369700_ocaa098-B23","doi-asserted-by":"crossref","DOI":"10.1186\/s12911-017-0430-8","article-title":"Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system","volume":"17","author":"Ancker","year":"2017","journal-title":"BMC Med Inform Decis Mak"},{"issue":"12","key":"2020110613074369700_ocaa098-B24","doi-asserted-by":"crossref","first-page":"2310","DOI":"10.1377\/hlthaff.2010.1111","article-title":"Analysis & commentary: Clinical decision support systems could be modified to reduce \u201calert fatigue\u201d while still minimizing the risk of litigation","volume":"30","author":"Kesselheim","year":"2011","journal-title":"Health Aff (Millwood)"},{"issue":"6","key":"2020110613074369700_ocaa098-B25","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1016\/j.jelectrocard.2012.08.050","article-title":"Clinical alarm hazards: a top ten health technology safety concern","volume":"45","author":"Keller","year":"2012","journal-title":"J Electrocardiol"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/8\/1252\/34153299\/ocaa098.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/8\/1252\/34153299\/ocaa098.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T19:16:11Z","timestamp":1604690171000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/8\/1252\/5867236"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,4]]},"references-count":25,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,7,4]]},"published-print":{"date-parts":[[2020,8,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa098","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,8]]},"published":{"date-parts":[[2020,7,4]]}}}