{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T10:24:08Z","timestamp":1771237448172,"version":"3.50.1"},"reference-count":29,"publisher":"Georg Thieme Verlag KG","issue":"01","funder":[{"DOI":"10.13039\/100000936","name":"Gordon and Betty Moore Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100000936","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2021,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>\n          Background\u2003Substantial research has been performed about the impact of computerized physician order entry on medication safety in the inpatient setting; however, relatively little has been done in ambulatory care, where most medications are prescribed.<\/jats:p><jats:p>\n          Objective\u2003To outline the development and piloting process of the Ambulatory Electronic Health Record (EHR) Evaluation Tool and to report the quantitative and qualitative results from the pilot.<\/jats:p><jats:p>\n          Methods\u2003The Ambulatory EHR Evaluation Tool closely mirrors the inpatient version of the tool, which is administered by The Leapfrog Group. The tool was piloted with seven clinics in the United States, each using a different EHR. The tool consists of a medication safety test and a medication reconciliation module. For the medication test, clinics entered test patients and associated test orders into their EHR and recorded any decision support they received. An overall percentage score of unsafe orders detected, and order category scores were provided to clinics. For the medication reconciliation module, clinics demonstrated how their EHR electronically detected discrepancies between two medication lists.<\/jats:p><jats:p>\n          Results\u2003For the medication safety test, the clinics correctly alerted on 54.6% of unsafe medication orders. Clinics scored highest in the drug allergy (100%) and drug\u2013drug interaction (89.3%) categories. Lower scoring categories included drug age (39.3%) and therapeutic duplication (39.3%). None of the clinics alerted for the drug laboratory or drug monitoring orders. In the medication reconciliation module, three (42.8%) clinics had an EHR-based medication reconciliation function; however, only one of those clinics could demonstrate it during the pilot.<\/jats:p><jats:p>\n          Conclusion\u2003Clinics struggled in areas of advanced decision support such as drug age, drug laboratory, and drub monitoring. Most clinics did not have an EHR-based medication reconciliation function and this process was dependent on accessing patients' medication lists. Wider use of this tool could improve outpatient medication safety and can inform vendors about areas of improvement.<\/jats:p>","DOI":"10.1055\/s-0041-1722917","type":"journal-article","created":{"date-parts":[[2021,3,4]],"date-time":"2021-03-04T00:07:39Z","timestamp":1614816459000},"page":"153-163","source":"Crossref","is-referenced-by-count":10,"title":["The Development and Piloting of the Ambulatory Electronic Health Record Evaluation Tool: Lessons Learned"],"prefix":"10.1055","volume":"12","author":[{"given":"Zoe","family":"Co","sequence":"additional","affiliation":[{"name":"Division of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States"}]},{"given":"A. Jay","family":"Holmgren","sequence":"additional","affiliation":[{"name":"Harvard Business School, Boston, Massachusetts, United States"}]},{"given":"David C.","family":"Classen","sequence":"additional","affiliation":[{"name":"Department of Clinical Epidemiology, University of Utah, Salt Lake City, Utah, United States"}]},{"given":"Lisa P.","family":"Newmark","sequence":"additional","affiliation":[{"name":"Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, United States"}]},{"given":"Diane L.","family":"Seger","sequence":"additional","affiliation":[{"name":"Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, United States"}]},{"given":"Jessica M.","family":"Cole","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States"}]},{"given":"Barbara","family":"Pon","sequence":"additional","affiliation":[{"name":"Collaborative Healthcare Patient Safety Organization, Sacramento, California, United States"}]},{"given":"Karen P.","family":"Zimmer","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Thomas Jefferson University, Philadelphia, Pennsylvania, United States"}]},{"given":"David W.","family":"Bates","sequence":"additional","affiliation":[{"name":"Division of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States"},{"name":"Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, United States"},{"name":"Harvard Medical School, Boston, Massachusetts, United States"}]}],"member":"194","published-online":{"date-parts":[[2021,3,3]]},"reference":[{"issue":"05","key":"ref1","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1056\/NEJMp0912825","article-title":"Launching HITECH","volume":"362","author":"D Blumenthal","year":"2010","journal-title":"N Engl J Med"},{"issue":"01","key":"ref2","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1197\/jamia.M2170","article-title":"Medication-related clinical decision support in computerized provider order entry systems: a review","volume":"14","author":"G J Kuperman","year":"2007","journal-title":"J Am Med Inform Assoc"},{"issue":"04","key":"ref3","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1136\/jamia.1999.00660313","article-title":"The impact of computerized physician order entry on medication error prevention","volume":"6","author":"D W Bates","year":"1999","journal-title":"J Am Med Inform Assoc"},{"issue":"03","key":"ref4","doi-asserted-by":"crossref","first-page":"686","DOI":"10.4338\/ACI-2017-01-RA-0003","article-title":"Electronic health record alert-related workload as a predictor of burnout in primary care providers","volume":"8","author":"M E Gregory","year":"2017","journal-title":"Appl Clin Inform"},{"issue":"03","key":"ref5","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1136\/amiajnl-2012-001241","article-title":"Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems","volume":"20","author":"D C Radley","year":"2013","journal-title":"J Am Med Inform Assoc"},{"issue":"15","key":"ref6","doi-asserted-by":"crossref","first-page":"1311","DOI":"10.1001\/jama.280.15.1311","article-title":"Effect of computerized physician order entry and a team intervention on prevention of serious medication errors","volume":"280","author":"D W Bates","year":"1998","journal-title":"JAMA"},{"issue":"01","key":"ref7","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1136\/bmjqs-2019-009609","article-title":"Assessing the safety of electronic health records: a national longitudinal study of medication-related decision support","volume":"29","author":"A J Holmgren","year":"2020","journal-title":"BMJ Qual Saf"},{"issue":"05","key":"ref8","doi-asserted-by":"crossref","first-page":"e205547","DOI":"10.1001\/jamanetworkopen.2020.5547","article-title":"National trends in the safety performance of electronic health record systems from 2009 to 2018","volume":"3","author":"D C Classen","year":"2020","journal-title":"JAMA Netw Open"},{"issue":"04","key":"ref9","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1097\/PTS.0b013e3182a8c2b2","article-title":"Safe use of electronic health records and health information technology systems: trust but verify","volume":"9","author":"C R Denham","year":"2013","journal-title":"J Patient Saf"},{"issue":"06","key":"ref10","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1007\/s11606-009-1238-8","article-title":"Electronic prescribing improves medication safety in community-based office practices","volume":"25","author":"R Kaushal","year":"2010","journal-title":"J Gen Intern Med"},{"issue":"09","key":"ref11","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1111\/j.1525-1497.2005.0194.x","article-title":"Outpatient prescribing errors and the impact of computerized prescribing","volume":"20","author":"T K Gandhi","year":"2005","journal-title":"J Gen Intern Med"},{"issue":"08","key":"ref12","doi-asserted-by":"crossref","first-page":"702","DOI":"10.1001\/2013.jamainternmed.61","article-title":"Information overload and missed test results in EHR-based settings","volume":"173","author":"H Singh","year":"2013","journal-title":"JAMA Intern Med"},{"issue":"01","key":"ref13","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1186\/s12911-017-0430-8","article-title":"Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system","volume":"17","author":"J S Ancker","year":"2017","journal-title":"BMC Med Inform Decis Mak"},{"issue":"08","key":"ref14","doi-asserted-by":"crossref","first-page":"1252","DOI":"10.1093\/jamia\/ocaa098","article-title":"The tradeoffs between safety and alert fatigue: data from a national evaluation of hospital medication-related clinical decision support","volume":"27","author":"Z Co","year":"2020","journal-title":"J Am Med Inform Assoc"},{"issue":"02","key":"ref15","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1136\/qshc.2005.014969","article-title":"Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems","volume":"15","author":"P M Kilbridge","year":"2006","journal-title":"Qual Saf Health Care"},{"issue":"04","key":"ref16","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1377\/hlthaff.2010.0160","article-title":"Mixed results in the safety performance of computerized physician order entry","volume":"29","author":"J Metzger","year":"2010","journal-title":"Health Aff (Millwood)"},{"key":"ref17","doi-asserted-by":"crossref","first-page":"e85","DOI":"10.1136\/amiajnl-2012-001549","article-title":"Relationship between medication event rates and the Leapfrog computerized physician order entry evaluation tool","volume":"20","author":"A A Leung","year":"2013","journal-title":"J Am Med Inform Assoc"},{"issue":"16","key":"ref20","doi-asserted-by":"crossref","first-page":"1556","DOI":"10.1056\/NEJMsa020703","article-title":"Adverse drug events in ambulatory care","volume":"348","author":"T K Gandhi","year":"2003","journal-title":"N Engl J Med"},{"issue":"03","key":"ref21","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1136\/amiajnl-2012-001089","article-title":"Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records","volume":"20","author":"S Phansalkar","year":"2013","journal-title":"J Am Med Inform Assoc"},{"issue":"04","key":"ref23","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.4338\/ACI-2016-06-R-0105","article-title":"Mind the Gap. A systematic review to identify usability and safety challenges and practices during electronic health record implementation","volume":"7","author":"R Ratwani","year":"2016","journal-title":"Appl Clin Inform"},{"issue":"04","key":"ref24","doi-asserted-by":"crossref","first-page":"817","DOI":"10.1055\/s-0038-1675210","article-title":"An evidence-based tool for safe configuration of electronic health records: the eSafety checklist","volume":"9","author":"P Dhillon-Chattha","year":"2018","journal-title":"Appl Clin Inform"},{"issue":"01","key":"ref25","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1186\/1471-2296-8-32","article-title":"The frequency of missed test results and associated treatment delays in a highly computerized health system","volume":"8","author":"T L Wahls","year":"2007","journal-title":"BMC Fam Pract"},{"key":"ref26","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/1472-6947-11-22","article-title":"Understanding the management of electronic test result notifications in the outpatient setting","volume":"11","author":"S J Hysong","year":"2011","journal-title":"BMC Med Inform Decis Mak"},{"issue":"21","key":"ref27","doi-asserted-by":"crossref","first-page":"2625","DOI":"10.1001\/archinte.163.21.2625","article-title":"Physicians' decisions to override computerized drug alerts in primary care","volume":"163","author":"S N Weingart","year":"2003","journal-title":"Arch Intern Med"},{"issue":"02","key":"ref28","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1055\/s-0040-1709509","article-title":"The development and evaluation of an electronic health record efficiency workshop for providers","volume":"11","author":"K Scott","year":"2020","journal-title":"Appl Clin Inform"},{"key":"ref29","series-title":"The quality chasm series: implications for nursing","first-page":"1","volume-title":"Patient safety and quality: an evidence-based handbook for nurses","author":"M Wakefield","year":"2008"},{"issue":"03","key":"ref30","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1136\/jamia.1996.96310637","article-title":"The accuracy of medication data in an outpatient electronic medical record","volume":"3","author":"M M Wagner","year":"1996","journal-title":"J Am Med Inform Assoc"},{"issue":"09","key":"ref31","doi-asserted-by":"crossref","first-page":"e1910756","DOI":"10.1001\/jamanetworkopen.2019.10756","article-title":"Effect of an electronic medication reconciliation intervention on adverse drug events: a cluster randomized trial","volume":"2","author":"R Tamblyn","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"01","key":"ref32","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1186\/s12911-016-0353-9","article-title":"Impact of electronic medication reconciliation interventions on medication discrepancies at hospital transitions: a systematic review and meta-analysis","volume":"16","author":"A B Mekonnen","year":"2016","journal-title":"BMC Med Inform Decis Mak"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/s-0041-1722917.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,4]],"date-time":"2021-03-04T00:08:53Z","timestamp":1614816533000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0041-1722917"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1]]},"references-count":29,"journal-issue":{"issue":"01","published-online":{"date-parts":[[2021,1,6]]},"published-print":{"date-parts":[[2021,1]]}},"URL":"https:\/\/doi.org\/10.1055\/s-0041-1722917","relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1]]}}}