{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T06:27:47Z","timestamp":1777357667341,"version":"3.51.4"},"reference-count":41,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2016,11,5]],"date-time":"2016-11-05T00:00:00Z","timestamp":1478304000000},"content-version":"vor","delay-in-days":222,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2016,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Objective To illustrate ways in which clinical decision support systems (CDSSs) malfunction and identify patterns of such malfunctions.<\/jats:p><jats:p>Materials and Methods We identified and investigated several CDSS malfunctions at Brigham and Women\u2019s Hospital and present them as a case series. We also conducted a preliminary survey of Chief Medical Information Officers to assess the frequency of such malfunctions.<\/jats:p><jats:p>Results We identified four CDSS malfunctions at Brigham and Women\u2019s Hospital: (1) an alert for monitoring thyroid function in patients receiving amiodarone stopped working when an internal identifier for amiodarone was changed in another system; (2) an alert for lead screening for children stopped working when the rule was inadvertently edited; (3) a software upgrade of the electronic health record software caused numerous spurious alerts to fire; and (4) a malfunction in an external drug classification system caused an alert to inappropriately suggest antiplatelet drugs, such as aspirin, for patients already taking one. We found that 93% of the Chief Medical Information Officers who responded to our survey had experienced at least one CDSS malfunction, and two-thirds experienced malfunctions at least annually.<\/jats:p><jats:p>Discussion CDSS malfunctions are widespread and often persist for long periods. The failure of alerts to fire is particularly difficult to detect. A range of causes, including changes in codes and fields, software upgrades, inadvertent disabling or editing of rules, and malfunctions of external systems commonly contribute to CDSS malfunctions, and current approaches for preventing and detecting such malfunctions are inadequate.<\/jats:p><jats:p>Conclusion CDSS malfunctions occur commonly and often go undetected. Better methods are needed to prevent and detect these malfunctions.<\/jats:p>","DOI":"10.1093\/jamia\/ocw005","type":"journal-article","created":{"date-parts":[[2016,3,30]],"date-time":"2016-03-30T00:18:52Z","timestamp":1459297132000},"page":"1068-1076","source":"Crossref","is-referenced-by-count":98,"title":["Analysis of clinical decision support system malfunctions: a case series and survey"],"prefix":"10.1093","volume":"23","author":[{"given":"Adam","family":"Wright","sequence":"first","affiliation":[{"name":"Brigham & Women\u2019s Hospital, Boston, MA, USA"},{"name":"Harvard Medical School, Boston, MA, USA"},{"name":"Partners HealthCare, Boston, MA, USA"}]},{"given":"Thu-Trang T","family":"Hickman","sequence":"additional","affiliation":[{"name":"Brigham & Women\u2019s Hospital, Boston, MA, USA"}]},{"given":"Dustin","family":"McEvoy","sequence":"additional","affiliation":[{"name":"Partners HealthCare, Boston, MA, USA"}]},{"given":"Skye","family":"Aaron","sequence":"additional","affiliation":[{"name":"Brigham & Women\u2019s Hospital, Boston, MA, USA"}]},{"given":"Angela","family":"Ai","sequence":"additional","affiliation":[{"name":"Brigham & Women\u2019s Hospital, Boston, MA, USA"}]},{"given":"Jan Marie","family":"Andersen","sequence":"additional","affiliation":[{"name":"Brigham & Women\u2019s Hospital, Boston, MA, USA"}]},{"given":"Salman","family":"Hussain","sequence":"additional","affiliation":[{"name":"Brigham & Women\u2019s Hospital, Boston, MA, USA"},{"name":"The Dartmouth Institute for Health Policy and Clinical Practice, Lebanon, NH, USA"}]},{"given":"Rachel","family":"Ramoni","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, MA, USA"},{"name":"Harvard School of Dental Medicine, Boston, MA, USA"}]},{"given":"Julie","family":"Fiskio","sequence":"additional","affiliation":[{"name":"Brigham & Women\u2019s Hospital, Boston, MA, USA"}]},{"given":"Dean F","family":"Sittig","sequence":"additional","affiliation":[{"name":"University of Texas Health Science Center, Houston, TX, USA"}]},{"given":"David W","family":"Bates","sequence":"additional","affiliation":[{"name":"Brigham & Women\u2019s Hospital, Boston, MA, USA"},{"name":"Harvard Medical School, Boston, MA, USA"},{"name":"Partners HealthCare, Boston, MA, USA"}]}],"member":"286","published-online":{"date-parts":[[2016,3,28]]},"reference":[{"issue":"3","key":"2020110612385136900_ocw005-B1","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1377\/hlthaff.2011.0178","article-title":"The benefits of health information technology: a review of the recent literature shows predominantly positive results","volume":"30","author":"Buntin","year":"2011","journal-title":"Health Affairs (Project Hope)."},{"issue":"5","key":"2020110612385136900_ocw005-B2","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1136\/jamia.2010.005561","article-title":"Clinical decision support: progress and opportunities","volume":"17","author":"Lyman","year":"2010","journal-title":"J Am Med Inform Assoc."},{"issue":"2","key":"2020110612385136900_ocw005-B3","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1197\/jamia.M2298","article-title":"Proposal for fulfilling strategic objectives of the U.S. Roadmap for national action on clinical decision support through a service-oriented architecture leveraging HL7 services","volume":"14","author":"Kawamoto","year":"2007","journal-title":"J Am Med Inform Assoc."},{"issue":"10","key":"2020110612385136900_ocw005-B4","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.1001\/jama.293.10.1223","article-title":"Effects of computerized clinical decision support systems on practitioner performance and patient outcomes","volume":"293","author":"Garg","year":"2005","journal-title":"JAMA."},{"issue":"7494","key":"2020110612385136900_ocw005-B5","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1136\/bmj.38398.500764.8F","article-title":"Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success","volume":"330","author":"Kawamoto","year":"2005","journal-title":"BMJ."},{"issue":"10","key":"2020110612385136900_ocw005-B6","doi-asserted-by":"crossref","first-page":"742","DOI":"10.7326\/0003-4819-144-10-200605160-00125","article-title":"Systematic review: impact of health information technology on quality, efficiency, and costs of medical care","volume":"144","author":"Chaudhry","year":"2006","journal-title":"Ann Int Med."},{"key":"2020110612385136900_ocw005-B7"},{"key":"2020110612385136900_ocw005-B8"},{"issue":"Pt 2","key":"2020110612385136900_ocw005-B9","first-page":"796","article-title":"Factors affecting physicians compliance with enrollment suggestions into a clinical reminders intervention","volume":"160","author":"Vashitz","year":"2010","journal-title":"Stud Health Technol Informatics."},{"key":"2020110612385136900_ocw005-B10","doi-asserted-by":"crossref","first-page":"i68","DOI":"10.1136\/qshc.2010.042085","article-title":"A new sociotechnical model for studying health information technology in complex adaptive healthcare systems","volume":"19","author":"Sittig","year":"2010","journal-title":"Qual Safety Health Care."},{"issue":"6","key":"2020110612385136900_ocw005-B11","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1111\/j.1365-2753.2009.01201.x","article-title":"The intended and unintended consequences of clinical guidelines","volume":"15","author":"Shackelton","year":"2009","journal-title":"J Eval Clin Pract."},{"issue":"3","key":"2020110612385136900_ocw005-B12","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1136\/amiajnl-2011-000185","article-title":"A framework for evaluating the appropriateness of clinical decision support alerts and responses","volume":"19","author":"McCoy","year":"2012","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612385136900_ocw005-B13","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1136\/amiajnl-2012-001494","article-title":"The hazard of software updates to clinical workstations: a natural experiment","volume":"20","author":"Landman","year":"2013","journal-title":"J Am Med Inform Assoc."},{"issue":"5","key":"2020110612385136900_ocw005-B14","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1197\/jamia.M2042","article-title":"Types of unintended consequences related to computerized provider order entry","volume":"13","author":"Campbell","year":"2006","journal-title":"J Am Med Inform Assoc."},{"issue":"1","key":"2020110612385136900_ocw005-B15","doi-asserted-by":"crossref","first-page":"29","DOI":"10.7326\/0003-4819-157-1-201207030-00450","article-title":"Effect of clinical decision-support systems: a systematic review","volume":"157","author":"Bright","year":"2012","journal-title":"Ann Int Med."},{"issue":"4","key":"2020110612385136900_ocw005-B16","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1197\/jamia.M2373","article-title":"The extent and importance of unintended consequences related to computerized provider order entry","volume":"14","author":"Ash","year":"2007","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612385136900_ocw005-B17","first-page":"26","article-title":"Some unintended consequences of clinical decision support systems","author":"Ash","year":"2007","journal-title":"AMIA Annual Symposium Proceedings\/AMIA Symposium AMIA Symposium."},{"key":"2020110612385136900_ocw005-B18","author":"Aleccia"},{"key":"2020110612385136900_ocw005-B19","author":"Associated Press"},{"key":"2020110612385136900_ocw005-B20","author":"Graham"},{"key":"2020110612385136900_ocw005-B21","author":"Schulte"},{"key":"2020110612385136900_ocw005-B22","author":"Silver"},{"key":"2020110612385136900_ocw005-B23"},{"key":"2020110612385136900_ocw005-B24"},{"issue":"5","key":"2020110612385136900_ocw005-B25","doi-asserted-by":"crossref","first-page":"1081","DOI":"10.1093\/jamia\/ocv073","article-title":"Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development","volume":"22","author":"McCoy","year":"2015","journal-title":"J Am Med Inform Assoc."},{"issue":"4","key":"2020110612385136900_ocw005-B26","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1197\/jamia.M2602","article-title":"A randomized trial of electronic clinical reminders to improve medication laboratory monitoring","volume":"15","author":"Matheny","year":"2008","journal-title":"J Am Med Inform Assoc."},{"issue":"5","key":"2020110612385136900_ocw005-B27","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1006\/cbmr.1993.1032","article-title":"Monitoring the monitor: automated statistical tracking of a clinical event monitor","volume":"26","author":"Hripcsak","year":"1993","journal-title":"Comp Biomed Res."},{"issue":"5","key":"2020110612385136900_ocw005-B28","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1016\/0010-4809(89)90040-2","article-title":"Implementation of a computerized patient advice system using the HELP clinical information system","volume":"22","author":"Sittig","year":"1989","journal-title":"Comp Biomed Res."},{"key":"2020110612385136900_ocw005-B29","author":"Office of the National Coordinator for Health Information Technology"},{"key":"2020110612385136900_ocw005-B30","article-title":"Committee on Patient Safety and Health Information Technology","volume-title":"Health IT and Patient Safety: Building Safer Systems for Better Care","author":"Institute of Medicine","year":"2012"},{"issue":"10","key":"2020110612385136900_ocw005-B31","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.1001\/jama.293.10.1223","article-title":"Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review","volume":"293","author":"Garg","year":"2005","journal-title":"JAMA."},{"issue":"4","key":"2020110612385136900_ocw005-B32","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1197\/jamia.M2364","article-title":"A description and functional taxonomy of rule-based decision support content at a large integrated delivery network","volume":"14","author":"Wright","year":"2007","journal-title":"J Am Med Inform Assoc."},{"issue":"2","key":"2020110612385136900_ocw005-B33","doi-asserted-by":"crossref","first-page":"304","DOI":"10.4338\/ACI-2013-02-CR-0014","article-title":"Lessons learned for collaborative clinical content development","volume":"4","author":"Collins","year":"2013","journal-title":"Appl Clin Informatics."},{"key":"2020110612385136900_ocw005-B34","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1186\/1472-6947-11-13","article-title":"Comparison of clinical knowledge management capabilities of commercially-available and leading internally-developed electronic health records","volume":"11","author":"Sittig","year":"2011","journal-title":"BMC Med Informatics Decis Mak."},{"issue":"1","key":"2020110612385136900_ocw005-B35","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1016\/j.ijmedinf.2009.09.003","article-title":"The state of the art in clinical knowledge management: an inventory of tools and techniques","volume":"79","author":"Sittig","year":"2010","journal-title":"Int J Med Informatics."},{"key":"2020110612385136900_ocw005-B36","first-page":"665","article-title":"Managing the investment in clinical decision support","volume-title":"Clinical Decision Support. The Road to Broad Adoption","author":"Hongsermeier","year":"2014","edition":"2nd edn"},{"issue":"2","key":"2020110612385136900_ocw005-B37","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1136\/amiajnl-2014-002988","article-title":"Patient safety goals for the proposed Federal Health Information Technology Safety Center","volume":"22","author":"Sittig","year":"2015","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612385136900_ocw005-B38","author":"US Department of Health and Human Services"},{"issue":"5","key":"2020110612385136900_ocw005-B39","first-page":"418","article-title":"The SAFER guides: empowering organizations to improve the safety and effectiveness of electronic health records","volume":"20","author":"Sittig","year":"2014","journal-title":"Am J Managed Care."},{"issue":"5","key":"2020110612385136900_ocw005-B40","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1001\/jama.2010.61","article-title":"Safe electronic health record use requires a comprehensive monitoring and evaluation framework","volume":"303","author":"Sittig","year":"2010","journal-title":"JAMA."},{"key":"2020110612385136900_ocw005-B41"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/23\/6\/1068\/34147881\/ocw005.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/23\/6\/1068\/34147881\/ocw005.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,16]],"date-time":"2022-06-16T15:58:30Z","timestamp":1655395110000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/23\/6\/1068\/2399245"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,3,28]]},"references-count":41,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2016,3,28]]},"published-print":{"date-parts":[[2016,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocw005","relation":{},"ISSN":["1527-974X","1067-5027"],"issn-type":[{"value":"1527-974X","type":"electronic"},{"value":"1067-5027","type":"print"}],"subject":[],"published-other":{"date-parts":[[2016,11]]},"published":{"date-parts":[[2016,3,28]]}}}