{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T02:39:20Z","timestamp":1780454360116,"version":"3.54.1"},"reference-count":33,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2017,11,24]],"date-time":"2017-11-24T00:00:00Z","timestamp":1511481600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/academic.oup.com\/journals\/pages\/about_us\/legal\/notices"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>The federal electronic health record (EHR) certification process was intended to ensure a baseline level of system quality and the ability to support meaningful use criteria. We sought to assess whether there was variation across EHR vendors in the degree to which hospitals using products from those vendors were able to achieve high levels of performance on meaningful use criteria.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>We created a cross-sectional national hospital sample from the Office of the National Coordinator for Health Information Technology EHR Products Used for Meaningful Use Attestation public use file and the Centers for Medicare &amp; Medicaid Services Medicare EHR Incentive Program Eligible Hospitals public use file. We used regression models to assess the relationship between vendor and hospital performance on 6 Stage 2 Meaningful Use criteria, controlling for hospital characteristics. We also calculated how much variation in performance is explained by vendor choice.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We found significant associations between specific vendor and level of hospital performance for all 6 meaningful use criteria. Epic was associated with significantly higher performance on 5 of the 6 criteria; relationships for other vendors were mixed, with some associated with significantly worse performance on multiple criteria. EHR vendor choice accounted for between 7% and 34% of performance variation across the 6 criteria.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>A nontrivial proportion of variation in hospital meaningful use performance is explained by vendor choice, and certain vendors are more often associated with better meaningful use performance than others. Our results suggest that policy-makers should improve the certification process by including more \u201creal-world\u201d scenario testing and provider feedback or ratings to reduce this variation. Hospitals can use these results to guide interactions with vendors.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Vendor choice accounts for a meaningful proportion of variation in hospital meaningful use performance, and specific vendors are consistently associated with higher or lower performance across criteria.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocx135","type":"journal-article","created":{"date-parts":[[2017,10,21]],"date-time":"2017-10-21T19:08:54Z","timestamp":1508612934000},"page":"654-660","source":"Crossref","is-referenced-by-count":49,"title":["Are all certified EHRs created equal? Assessing the relationship between EHR vendor and hospital meaningful use performance"],"prefix":"10.1093","volume":"25","author":[{"given":"A Jay","family":"Holmgren","sequence":"first","affiliation":[{"name":"Harvard Business School, Harvard University, Boston, MA 02163, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julia","family":"Adler-Milstein","sequence":"additional","affiliation":[{"name":"School of Medicine, University of California, San Francisco, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeffrey","family":"McCullough","sequence":"additional","affiliation":[{"name":"School of Public Health, University of Michigan, Ann Arbor, MI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2017,11,24]]},"reference":[{"issue":"6","key":"2020110612360832300_ocx135-B1","doi-asserted-by":"crossref","first-page":"1751","DOI":"10.1111\/1475-6773.12406","article-title":"EHR adoption and hospital performance: time-related effects","volume":"50","author":"Adler-Milstein","year":"2015","journal-title":"Health Services Res."},{"issue":"4","key":"2020110612360832300_ocx135-B2","doi-asserted-by":"crossref","first-page":"769","DOI":"10.1093\/jamia\/ocw185","article-title":"Are informed policies in place to promote safe and usable EHRs? A cross-industry comparison","volume":"24","author":"Savage","year":"2017","journal-title":"J Am Med Inform Assoc."},{"issue":"5","key":"2020110612360832300_ocx135-B3","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1056\/NEJMp0912825","article-title":"Launching HITECH","volume":"362","author":"Blumenthal","year":"2010","journal-title":"N Engl J Med."},{"issue":"12","key":"2020110612360832300_ocx135-B4","doi-asserted-by":"crossref","first-page":"2174","DOI":"10.1377\/hlthaff.2015.0992","article-title":"Electronic health record adoption in US hospitals: progress continues, but challenges persist","volume":"34","author":"Adler-Milstein","year":"2015","journal-title":"Health Aff (Millwood)."},{"key":"2020110612360832300_ocx135-B5","volume-title":"ONC Health IT Certification Program","author":"Office of the National Coordinator for Health Information","year":"2016"},{"issue":"4","key":"2020110612360832300_ocx135-B6","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1093\/jamia\/ocv013","article-title":"Graphical display of diagnostic test results in electronic health records: a comparison of 8 systems","volume":"22","author":"Sittig","year":"2015","journal-title":"J Am Med Inform Assoc."},{"issue":"5","key":"2020110612360832300_ocx135-B7","doi-asserted-by":"crossref","first-page":"1081","DOI":"10.1093\/jamia\/ocv073","article-title":"Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development","volume":"22","author":"McCoy","year":"2015","journal-title":"J Am Med Inform Assoc."},{"issue":"10","key":"2020110612360832300_ocx135-B8","doi-asserted-by":"crossref","first-page":"1070","DOI":"10.1001\/jama.2015.8372","article-title":"Electronic health record vendor adherence to usability certification requirements and testing standards","volume":"314","author":"Ratwani","year":"2015","journal-title":"JAMA."},{"issue":"6","key":"2020110612360832300_ocx135-B9","doi-asserted-by":"crossref","first-page":"1179","DOI":"10.1093\/jamia\/ocv050","article-title":"Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors","volume":"22","author":"Ratwani","year":"2015","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612360832300_ocx135-B10","article-title":"eClinicalWorks to pay $155 million to settle suit alleging it faked meaningful use certification","author":"Sullivan","year":"2017","journal-title":"Healthcare IT News."},{"key":"2020110612360832300_ocx135-B11","article-title":"Moving Past the EHR Interoperability Blame Game","author":"Adler-Milstein","year":"2017","journal-title":"NEJM Catalyst."},{"issue":"6","key":"2020110612360832300_ocx135-B12","doi-asserted-by":"crossref","first-page":"1506","DOI":"10.1542\/peds.2005-1287","article-title":"Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system","volume":"116","author":"Han","year":"2005","journal-title":"Pediatrics."},{"issue":"4","key":"2020110612360832300_ocx135-B13","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1377\/hlthaff.2010.0160","article-title":"Mixed results in the safety performance of computerized physician order entry","volume":"29","author":"Metzger","year":"2010","journal-title":"Health Aff."},{"issue":"5","key":"2020110612360832300_ocx135-B14","doi-asserted-by":"crossref","first-page":"S13","DOI":"10.1016\/j.pmrj.2017.01.009","article-title":"Electronic health record transition considerations","volume":"9","author":"Penrod","year":"2017","journal-title":"PM&R."},{"key":"2020110612360832300_ocx135-B15","author":"American Academy of Family Physicians","year":"2015"},{"key":"2020110612360832300_ocx135-B16","author":"Centers for Medicare & Medicaid Services","year":"2016"},{"key":"2020110612360832300_ocx135-B17","author":"Office of the National Coordinator for Health Information Technology"},{"key":"2020110612360832300_ocx135-B18","author":"Centers for Medicare & Medicaid Services","year":"2015"},{"issue":"e2","key":"2020110612360832300_ocx135-B19","doi-asserted-by":"crossref","first-page":"e257","DOI":"10.1136\/amiajnl-2013-002449","article-title":"Reliability and validity of the American Hospital Association's national longitudinal survey of health information technology adoption","volume":"21","author":"Everson","year":"2014","journal-title":"J Am Med Inform Assoc."},{"issue":"1","key":"2020110612360832300_ocx135-B20","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.hjdsi.2013.12.005","article-title":"Health information exchange among US hospitals: who's in, who's out, and why?","volume":"2","author":"Adler-Milstein","year":"2014","journal-title":"In Healthc (Amst)."},{"issue":"7","key":"2020110612360832300_ocx135-B21","doi-asserted-by":"crossref","first-page":"1286","DOI":"10.1377\/hlthaff.2015.1215","article-title":"Engagement in hospital health information exchange is associated with vendor marketplace dominance","volume":"35","author":"Everson","year":"2016","journal-title":"Health Affairs."},{"issue":"3","key":"2020110612360832300_ocx135-B22","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1136\/amiajnl-2012-001241","article-title":"Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems","volume":"20","author":"Radley","year":"2013","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612360832300_ocx135-B23","doi-asserted-by":"crossref","DOI":"10.1377\/hlthaff.W5.10","article-title":"The value of health care information exchange and interoperability","volume":"24","author":"Walker","year":"2005","journal-title":"Health Aff."},{"key":"2020110612360832300_ocx135-B24","first-page":"188","article-title":"Herfindahl-Hirschman index","volume":"79","author":"Rhoades","year":"1993","journal-title":"Fed Res Bull."},{"issue":"16","key":"2020110612360832300_ocx135-B25","doi-asserted-by":"crossref","first-page":"e32","DOI":"10.1056\/NEJMp1103602","article-title":"Launching accountable care organizations\u2014the proposed rule for the Medicare Shared Savings Program","volume":"364","author":"Berwick","year":"2011","journal-title":"New Engl J Med."},{"key":"2020110612360832300_ocx135-B26","author":"Patient-Centered Primary Care Collaborative","year":"2007"},{"issue":"1","key":"2020110612360832300_ocx135-B27","first-page":"w60","article-title":"The state of regional health information organizations: current activities and financing","volume":"27","author":"Adler-Milstein","year":"2008","journal-title":"Health Aff."},{"key":"2020110612360832300_ocx135-B28","article-title":"Physicians outline improvements to EHR certification process","author":"American Medical Association","year":"2015","journal-title":"AMA Wire."},{"issue":"2","key":"2020110612360832300_ocx135-B29","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1093\/jamia\/ocw134","article-title":"National trends in safety performance of electronic health record systems in children's hospitals","volume":"24","author":"Chaparro","year":"2017","journal-title":"J Am Med Inform Assoc."},{"issue":"1","key":"2020110612360832300_ocx135-B30","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1542\/peds.2006-0367","article-title":"Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit","volume":"118","author":"Del Beccaro","year":"2006","journal-title":"Pediatrics."},{"issue":"1","key":"2020110612360832300_ocx135-B31","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1542\/peds.2009-3271","article-title":"Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system","volume":"126","author":"Longhurst","year":"2010","journal-title":"Pediatrics."},{"issue":"e1","key":"2020110612360832300_ocx135-B32","doi-asserted-by":"crossref","first-page":"e9","DOI":"10.1136\/amiajnl-2013-001684","article-title":"Ten key considerations for the successful implementation and adoption of large-scale health information technology","volume":"20","author":"Cresswell","year":"2013","journal-title":"J Am Med Inform Assoc."},{"issue":"1","key":"2020110612360832300_ocx135-B33","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1093\/jamia\/ocw037","article-title":"Ten key considerations for the successful optimization of large-scale health information technology","volume":"2","author":"Cresswell","year":"2017","journal-title":"J Am Med Inform Assoc."}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/6\/654\/34150071\/ocx135.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/6\/654\/34150071\/ocx135.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,5]],"date-time":"2022-08-05T00:20:42Z","timestamp":1659658842000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/25\/6\/654\/4658771"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,24]]},"references-count":33,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2017,11,24]]},"published-print":{"date-parts":[[2018,6,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocx135","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,6]]},"published":{"date-parts":[[2017,11,24]]}}}