{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T03:23:02Z","timestamp":1775704982550,"version":"3.50.1"},"reference-count":16,"publisher":"Georg Thieme Verlag KG","issue":"03","funder":[{"name":"National Library of Medicine of the National Institutes of Health","award":["T15LM007088"],"award-info":[{"award-number":["T15LM007088"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2017,7]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>\n          Objectives: Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.<\/jats:p><jats:p>\n          Methods: We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.<\/jats:p><jats:p>\n          Results: Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.<\/jats:p><jats:p>\n          Discussion: CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.<\/jats:p><jats:p>\n          Conclusion: CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.<\/jats:p><jats:p>Citation: Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8: 910\u2013923 https:\/\/doi.org\/10.4338\/ACI-2017-01-RA-0006<\/jats:p>","DOI":"10.4338\/aci-2017-01-ra-0006","type":"journal-article","created":{"date-parts":[[2017,9,6]],"date-time":"2017-09-06T21:26:26Z","timestamp":1504733186000},"page":"910-923","source":"Crossref","is-referenced-by-count":28,"title":["Clinical decisions support malfunctions in a commercial electronic health record"],"prefix":"10.4338","volume":"08","author":[{"given":"Thomas R.","family":"Yackel","sequence":"first","affiliation":[]},{"given":"Paul N.","family":"Gorman","sequence":"first","affiliation":[]},{"given":"David A.","family":"Dorr","sequence":"first","affiliation":[]},{"given":"Steven Z.","family":"Kassakian","sequence":"additional","affiliation":[]}],"member":"194","published-online":{"date-parts":[[2017,12,20]]},"reference":[{"issue":"01","key":"ref1","doi-asserted-by":"crossref","first-page":"29","DOI":"10.7326\/0003-4819-157-1-201207030-00450","article-title":"Effect of clinical decision-support systems: a systematic review","volume":"157","author":"TJ Bright","year":"2012","journal-title":"Annals of internal medicine"},{"issue":"24","key":"ref3","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1056\/NEJM197612092952405","article-title":"Protocol-based computer reminders, the quality of care and the non-perfectability of man","volume":"295","author":"CJ McDonald","year":"1976","journal-title":"The New England journal of medicine"},{"issue":"12","key":"ref4","doi-asserted-by":"crossref","first-page":"e12","DOI":"10.2105\/AJPH.2014.302164","article-title":"Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis","volume":"104","author":"L Moja","year":"2014","journal-title":"Am J Public Health"},{"key":"ref6","first-page":"26","article-title":"Some unintended consequences of clinical decision support systems","volume":"2007","author":"JS Ash","journal-title":"AMIA Annu Symp Proc"},{"issue":"01","key":"ref7","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1197\/jamia.M1868","article-title":"Improving acceptance of computerized prescribing alerts in ambulatory care","volume":"13","author":"NR Shah","year":"2006","journal-title":"Journal of the American Medical Informatics Association : JAMIA"},{"key":"ref8","doi-asserted-by":"crossref","first-page":"e145","DOI":"10.1136\/amiajnl-2011-000743","article-title":"Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study","volume":"19","author":"PJ Embi","year":"2012","journal-title":"Journal of the American Medical Informatics Association JAMIA"},{"issue":"23","key":"ref9","doi-asserted-by":"crossref","first-page":"2098","DOI":"10.2146\/ajhp090181","article-title":"Alert fatigue","volume":"66","author":"JJ Cash","year":"2009","journal-title":"Am J Health Syst Pharm"},{"issue":"S 01","key":"ref11","doi-asserted-by":"crossref","first-page":"S69","DOI":"10.1016\/j.ijmedinf.2008.07.015","article-title":"The unintended consequences of computerized provider order entry: findings from a mixed methods exploration","volume":"78","author":"JS Ash","year":"2009","journal-title":"International journal of medical informatics"},{"key":"ref14","first-page":"806","article-title":"Identifying best practices for clinical decision support and knowledge management in the field","volume":"160","author":"JS Ash","year":"2010","journal-title":"Studies in Health Technology & Informatics"},{"issue":"03","key":"ref15","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1136\/amiajnl-2011-000113","article-title":"Development and evaluation of a comprehensive clinical decision support taxonomy: comparison of front-end tools in commercial and internally developed electronic health record systems","volume":"18","author":"A Wright","year":"2011","journal-title":"Journal of the American Medical Informatics Association : JAMIA"},{"issue":"05","key":"ref16","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1197\/jamia.M3111","article-title":"Clinical decision support capabilities of commercially-available clinical information systems","volume":"16","author":"A Wright","year":"2009","journal-title":"Journal of the American Medical Informatics Association JAMIA"},{"key":"ref17","volume-title":"Improving healthcare with control charts : basic and advanced SPC methods and case studies","author":"RG Carey","year":"2003"},{"issue":"22","key":"ref19","doi-asserted-by":"crossref","first-page":"2458","DOI":"10.1161\/CIR.0b013e318235eb4d","article-title":"AHA\/ACCF Secondary Prevention and Risk Reduction Therapy for Patients with Coronary and other Atherosclerotic Vascular Disease: 2011 update: a guideline from the American Heart Association and American College of Cardiology Foundation","volume":"124","author":"SC Smith Jr.,","year":"2011","journal-title":"Circulation"},{"issue":"01","key":"ref20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/2945.981847","article-title":"Information Visualization and Visual Data Mining","volume":"8","author":"DA Keim","year":"2002","journal-title":"IEEE Trans Vis Comput Graph"},{"issue":"05","key":"ref21","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1177\/0018720815576827","article-title":"Usability and Safety in Electronic Medical Records Interface Design: A Review of Recent Literature and Guideline Formulation","volume":"57","author":"M Zahabi","year":"2015","journal-title":"Hum Factors"},{"issue":"02","key":"ref23","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1197\/jamia.M1809","article-title":"Overriding of drug safety alerts in computerized physician order entry","volume":"13","author":"H van der Sijs","year":"2006","journal-title":"Journal of the American Medical Informatics Association JAMIA"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.4338\/ACI-2017-01-RA-0006.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,31]],"date-time":"2023-05-31T11:44:33Z","timestamp":1685533473000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.4338\/ACI-2017-01-RA-0006"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,7]]},"references-count":16,"journal-issue":{"issue":"03","published-online":{"date-parts":[[2017,12,20]]},"published-print":{"date-parts":[[2017,7]]}},"URL":"https:\/\/doi.org\/10.4338\/aci-2017-01-ra-0006","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,7]]}}}