{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,30]],"date-time":"2026-03-30T14:07:37Z","timestamp":1774879657132,"version":"3.50.1"},"reference-count":25,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T00:00:00Z","timestamp":1718755200000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100005982","name":"Office of the National Coordinator for Health Information Technology","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100005982","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000016","name":"Department of Health and Human Services","doi-asserted-by":"publisher","award":["# 90AX0032\/01-02"],"award-info":[{"award-number":["# 90AX0032\/01-02"]}],"id":[{"id":"10.13039\/100000016","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>To identify impacts of different survey methodologies assessing primary care physicians' (PCPs\u2019) experiences with electronic health records (EHRs), we compared three surveys: the 2022 Continuous Certification Questionnaire (CCQ) from the American Board of Family Medicine, the 2022 University of California San Francisco (UCSF) Physician Health IT Survey, and the 2021 National Electronic Health Records Survey (NEHRS).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We evaluated differences between survey pairs using Rao-Scott corrected chi-square tests, which account for weighting.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>CCQ received 3991 responses from PCPs (100% response rate), UCSF received 1375 (3.6% response rate), and NEHRS received 858 (18.2% response rate). Substantial, statistically significant differences in demographics were detected across the surveys. CCQ respondents were younger and more likely to work in a health system; NEHRS respondents were more likely to work in private practice; and UCSF respondents disproportionately practiced in larger academic settings. Many EHR experience indicators were similar between CCQ and NEHRS, but CCQ respondents reported higher documentation burden.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>The UCSF approach is unlikely to supply reliable data. Significant demographic differences between CCQ and NEHRS raise response bias concerns, and while there were similarities in some reported EHR experiences, there were important, significant differences.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>Federal EHR policy monitoring and maintenance require reliable data. This test of existing and alternative sources suggest that diversified data sources are necessary to understand physicians\u2019 experiences with EHRs and interoperability. Comprehensive surveys administered by specialty boards have the potential to contribute to these efforts, since they are likely to be free of response bias.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae148","type":"journal-article","created":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T06:53:27Z","timestamp":1718780007000},"page":"1754-1762","source":"Crossref","is-referenced-by-count":6,"title":["Impact of response bias in three surveys on primary care providers\u2019 experiences with electronic health records"],"prefix":"10.1093","volume":"31","author":[{"given":"Nathaniel","family":"Hendrix","sequence":"first","affiliation":[{"name":"American Board of Family Medicine , Lexington, KY 40511, United States"},{"name":"Center for Professionalism and Value in Health Care , Washington, DC 20036, United States"}]},{"given":"Natalya","family":"Maisel","sequence":"additional","affiliation":[{"name":"Division of Clinical Informatics and Digital Transformation, University of California, San Francisco , San Francisco, CA 94117, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5355-6043","authenticated-orcid":false,"given":"Jordan","family":"Everson","sequence":"additional","affiliation":[{"name":"Department of Health and Human Services , Washington, DC 20201, United States"}]},{"given":"Vaishali","family":"Patel","sequence":"additional","affiliation":[{"name":"Department of Health and Human Services , Washington, DC 20201, United States"}]},{"given":"Andrew","family":"Bazemore","sequence":"additional","affiliation":[{"name":"American Board of Family Medicine , Lexington, KY 40511, United States"},{"name":"Center for Professionalism and Value in Health Care , Washington, DC 20036, United States"}]},{"given":"Lisa S","family":"Rotenstein","sequence":"additional","affiliation":[{"name":"Division of Clinical Informatics and Digital Transformation, University of California, San Francisco , San Francisco, CA 94117, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7939-6831","authenticated-orcid":false,"given":"A Jay","family":"Holmgren","sequence":"additional","affiliation":[{"name":"Division of Clinical Informatics and Digital Transformation, University of California, San Francisco , San Francisco, CA 94117, United States"}]},{"given":"Alex H","family":"Krist","sequence":"additional","affiliation":[{"name":"Family Medicine & Population Health, Virginia Commonwealth University , Richmond, VA 23219, United States"}]},{"given":"Julia","family":"Adler-Milstein","sequence":"additional","affiliation":[{"name":"Division of Clinical Informatics and Digital Transformation, University of California, San Francisco , San Francisco, CA 94117, United States"}]},{"given":"Robert L","family":"Phillips","sequence":"additional","affiliation":[{"name":"American Board of Family Medicine , Lexington, KY 40511, United States"},{"name":"Center for Professionalism and Value in Health Care , Washington, DC 20036, United States"}]}],"member":"286","published-online":{"date-parts":[[2024,6,18]]},"reference":[{"key":"2024071907520892300_ocae148-B1","year":"2022"},{"key":"2024071907520892300_ocae148-B2","author":"Pylypchuk","year":"2022"},{"key":"2024071907520892300_ocae148-B3","author":"Tripathi","year":"2023"},{"key":"2024071907520892300_ocae148-B4","year":"2022"},{"issue":"1","key":"2024071907520892300_ocae148-B5","doi-asserted-by":"crossref","first-page":"e100450","DOI":"10.1136\/bmjhci-2021-100450","article-title":"Exploring stakeholder attitudes towards AI in clinical practice","volume":"28","author":"Scott","year":"2021","journal-title":"BMJ Health Care Inform"},{"issue":"1","key":"2024071907520892300_ocae148-B6","doi-asserted-by":"crossref","first-page":"e75","DOI":"10.1097\/SLA.0000000000004078","article-title":"Global overview of response rates in patient and health care professional surveys in surgery","volume":"275","author":"Meyer","year":"2022","journal-title":"Ann Surg"},{"issue":"3","key":"2024071907520892300_ocae148-B7","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1177\/0163278713496425","article-title":"Enhancing surveys of health care professionals: a meta-analysis of techniques to improve response","volume":"36","author":"Cho","year":"2013","journal-title":"Eval Health Prof"},{"issue":"6","key":"2024071907520892300_ocae148-B8","doi-asserted-by":"crossref","first-page":"863","DOI":"10.1001\/jamainternmed.2021.0256","article-title":"Differences in total and after-hours electronic health record time across ambulatory specialties","volume":"181","author":"Rotenstein","year":"2021","journal-title":"JAMA Intern Med"},{"issue":"5","key":"2024071907520892300_ocae148-B9","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1370\/afm.2121","article-title":"Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations","volume":"15","author":"Arndt","year":"2017","journal-title":"Ann Fam Med"},{"issue":"4","key":"2024071907520892300_ocae148-B10","doi-asserted-by":"crossref","first-page":"479","DOI":"10.4300\/JGME-D-16-00123.1","article-title":"Electronic health record effects on work-life balance and burnout within the i3 population collaborative","volume":"9","author":"Robertson","year":"2017","journal-title":"J Grad Med Educ"},{"issue":"5","key":"2024071907520892300_ocae148-B11","doi-asserted-by":"crossref","first-page":"733","DOI":"10.3122\/jabfm.2018.05.170418","article-title":"How is family medicine engaging patients at the practice-level? A national sample of family physicians","volume":"31","author":"Sharma","year":"2018","journal-title":"J Am Board Fam Med"},{"issue":"2","key":"2024071907520892300_ocae148-B12","doi-asserted-by":"crossref","first-page":"577","DOI":"10.2307\/2532311","article-title":"A simple method for the analysis of clustered binary data","volume":"48","author":"Rao","year":"1992","journal-title":"Biometrics"},{"key":"2024071907520892300_ocae148-B13","author":"R Core Team","year":"2021"},{"issue":"8","key":"2024071907520892300_ocae148-B14","first-page":"1","article-title":"Analysis of complex survey samples","volume":"9","author":"Lumley","year":"2004","journal-title":"J Stat Soft [Internet]"},{"issue":"13","key":"2024071907520892300_ocae148-B15","doi-asserted-by":"crossref","first-page":"2980","DOI":"10.1007\/s11606-023-08169-5","article-title":"Variation in family physicians\u2019 experiences across different electronic health record platforms: a descriptive study","volume":"38","author":"Hendrix","year":"2023","journal-title":"J Gen Intern Med"},{"key":"2024071907520892300_ocae148-B16","author":"Avalere Health","year":"2022"},{"issue":"2","key":"2024071907520892300_ocae148-B17","first-page":"193","article-title":"Using variation in response rates of demographic subgroups as evidence of nonresponse bias in survey estimates","volume":"25","author":"Peytcheva","year":"2009","journal-title":"J Off Stat"},{"issue":"1","key":"2024071907520892300_ocae148-B18","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1186\/1472-6947-11-69","article-title":"Quality of human-computer interaction\u2014results of a national usability survey of hospital-IT in Germany","volume":"11","author":"Bundschuh","year":"2011","journal-title":"BMC Med Inform Decis Mak"},{"issue":"2","key":"2024071907520892300_ocae148-B19","doi-asserted-by":"crossref","first-page":"e5238","DOI":"10.2196\/medinform.5238","article-title":"Electronic health record-related safety concerns: a cross-sectional survey of electronic health record users","volume":"4","author":"Palojoki","year":"2016","journal-title":"JMIR Med Inform"},{"key":"2024071907520892300_ocae148-B20","doi-asserted-by":"crossref","first-page":"103938","DOI":"10.1016\/j.ijmedinf.2019.07.017","article-title":"A clinician survey of using speech recognition for clinical documentation in the electronic health record","volume":"130","author":"Goss","year":"2019","journal-title":"Int J Med Inform"},{"issue":"3","key":"2024071907520892300_ocae148-B21","doi-asserted-by":"crossref","first-page":"476","DOI":"10.1016\/j.mayocp.2019.09.024","article-title":"The association between perceived electronic health record usability and professional burnout among US physicians","volume":"95","author":"Melnick","year":"2020","journal-title":"Mayo Clin Proc"},{"key":"2024071907520892300_ocae148-B22"},{"issue":"1","key":"2024071907520892300_ocae148-B23","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1097\/ACM.0000000000001889","article-title":"Primary care residents in teaching health centers: their intentions to practice in underserved settings after residency training","volume":"93","author":"Talib","year":"2018","journal-title":"Acad Med"},{"issue":"1","key":"2024071907520892300_ocae148-B24","doi-asserted-by":"crossref","first-page":"556","DOI":"10.1353\/hpu.2018.0036","article-title":"The racial and ethnic composition and distribution of primary care physicians","volume":"29","author":"Xierali","year":"2018","journal-title":"J Health Care Poor Underserved"},{"issue":"7","key":"2024071907520892300_ocae148-B25","first-page":"640","article-title":"Diversity in the family medicine workforce","volume":"53","author":"Jabbarpour","year":"2021","journal-title":"Fam Med"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/8\/1754\/58591205\/ocae148.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/8\/1754\/58591205\/ocae148.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,19]],"date-time":"2024-07-19T07:52:32Z","timestamp":1721375552000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/8\/1754\/7695999"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,18]]},"references-count":25,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,6,18]]},"published-print":{"date-parts":[[2024,8,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae148","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,8]]},"published":{"date-parts":[[2024,6,18]]}}}