{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,31]],"date-time":"2026-01-31T04:46:46Z","timestamp":1769834806111,"version":"3.49.0"},"reference-count":41,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2020,7,27]],"date-time":"2020-07-27T00:00:00Z","timestamp":1595808000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100007747","name":"University of Utah","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007747","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Health Studies Fund"},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R38 HL 143605"],"award-info":[{"award-number":["R38 HL 143605"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000133","name":"Agency for Healthcare Research and Quality","doi-asserted-by":"publisher","award":["R18 HS26198"],"award-info":[{"award-number":["R18 HS26198"]}],"id":[{"id":"10.13039\/100000133","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>The study sought to evaluate a novel electronic health record (EHR) add-on application for chronic disease management that uses an integrated display to decrease user cognitive load, improve efficiency, and support clinical decision making.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We designed a chronic disease management application using the technology framework known as SMART on FHIR (Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources). We used mixed methods to obtain user feedback on a prototype to support ambulatory providers managing chronic obstructive pulmonary disease. Each participant managed 2 patient scenarios using the regular EHR with and without access to our prototype in block-randomized order. The primary outcome was the percentage of expert-recommended ideal care tasks completed. Timing, keyboard and mouse use, and participant surveys were also collected. User experiences were captured using a retrospective think-aloud interview analyzed by concept coding.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>With our prototype, the 13 participants completed more recommended care (81% vs 48%; P\u2009&amp;lt;\u2009.001) and recommended tasks per minute (0.8 vs 0.6; P\u2009=\u2009.03) over longer sessions (7.0 minutes vs 5.4 minutes; P\u2009=\u2009.006). Keystrokes per task were lower with the prototype (6 vs 18; P\u2009&amp;lt;\u2009.001). Qualitative themes elicited included the desire for reliable presentation of information which matches participants\u2019 mental models of disease and for intuitive navigation in order to decrease cognitive load.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Participants completed more recommended care by taking more time when using our prototype. Interviews identified a tension between using the inefficient but familiar EHR vs learning to use our novel prototype. Concept coding of user feedback generated actionable insights.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Mixed methods can support the design and evaluation of SMART on FHIR EHR add-on applications by enhancing understanding of the user experience.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaa099","type":"journal-article","created":{"date-parts":[[2020,5,12]],"date-time":"2020-05-12T12:59:04Z","timestamp":1589288344000},"page":"1225-1234","source":"Crossref","is-referenced-by-count":36,"title":["Integrated displays to improve chronic disease management in ambulatory care: A SMART on FHIR application informed by mixed-methods user testing"],"prefix":"10.1093","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5337-8590","authenticated-orcid":false,"given":"Rebecca L","family":"Curran","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5576-1486","authenticated-orcid":false,"given":"Polina V","family":"Kukhareva","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Teresa","family":"Taft","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Charlene R","family":"Weir","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1081-1670","authenticated-orcid":false,"given":"Thomas J","family":"Reese","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claude","family":"Nanjo","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Salvador","family":"Rodriguez-Loya","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Douglas K","family":"Martin","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Phillip B","family":"Warner","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David E","family":"Shields","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael C","family":"Flynn","sequence":"additional","affiliation":[{"name":"Community Physicians Group, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan P","family":"Boltax","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kensaku","family":"Kawamoto","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2020,7,27]]},"reference":[{"key":"2020110613074949400_ocaa099-B1","first-page":"677","article-title":"Implementing computers in ambulatory care: implications of physician practice patterns for system design","author":"Aydin","year":"1997","journal-title":"Proc AMIA Annu Fall Symp"},{"key":"2020110613074949400_ocaa099-B2","first-page":"8","article-title":"Broadening our approach to evaluating medical information systems","author":"Forsythe","year":"1991","journal-title":"Proc Annu Symp Comput Appl Med Care"},{"issue":"4","key":"2020110613074949400_ocaa099-B3","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1093\/jamia\/ocv022","article-title":"Leveraging health information technology to achieve the \u201ctriple aim\u201d of healthcare reform","volume":"22","author":"Sheikh","year":"2015","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2020110613074949400_ocaa099-B4","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/j.ophtha.2018.09.023","article-title":"Realizing the promise of electronic health records: moving beyond \u201cPaper on a Screen.\u201d","volume":"126","author":"Liebman","year":"2019","journal-title":"Ophthalmology"},{"issue":"6","key":"2020110613074949400_ocaa099-B5","doi-asserted-by":"crossref","first-page":"618","DOI":"10.1093\/jamia\/ocx094","article-title":"Implementing electronic health records (EHRs): health care provider perceptions before and after transition from a local basic EHR to a commercial comprehensive EHR","volume":"25","author":"Krousel-Wood","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"11","key":"2020110613074949400_ocaa099-B6","doi-asserted-by":"crossref","first-page":"1556","DOI":"10.1093\/jamia\/ocy112","article-title":"Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review","volume":"25","author":"Powers","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"2020110613074949400_ocaa099-B7","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1093\/jamia\/ocz220","article-title":"Electronic health records and burnout: Time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians","volume":"27","author":"Adler-Milstein","year":"2020","journal-title":"J Am Med Inform Assoc"},{"issue":"8","key":"2020110613074949400_ocaa099-B8","doi-asserted-by":"crossref","first-page":"e199609","DOI":"10.1001\/jamanetworkopen.2019.9609","article-title":"Association of electronic health record design and use factors with clinician stress and burnout","volume":"2","author":"Kroth","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"20","key":"2020110613074949400_ocaa099-B9","doi-asserted-by":"crossref","first-page":"1866","DOI":"10.1001\/archinternmed.2009.341","article-title":"Primary care visit duration and quality: does good care take longer?","volume":"169","author":"Chen","year":"2009","journal-title":"Arch Intern Med"},{"issue":"1","key":"2020110613074949400_ocaa099-B10","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1093\/fampra\/cmw108","article-title":"Time and feasibility of prevention in primary care","volume":"34","author":"Bucher","year":"2017","journal-title":"Fam Pract"},{"issue":"2","key":"2020110613074949400_ocaa099-B11","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1016\/j.amepre.2011.10.008","article-title":"Prioritization of evidence-based preventive health services during periodic health examinations","volume":"42","author":"Shires","year":"2012","journal-title":"Am J Prev Med"},{"issue":"4","key":"2020110613074949400_ocaa099-B12","doi-asserted-by":"crossref","first-page":"635","DOI":"10.2105\/AJPH.93.4.635","article-title":"Primary care: is there enough time for prevention?","volume":"93","author":"Yarnall","year":"2003","journal-title":"Am J Public Health"},{"issue":"2","key":"2020110613074949400_ocaa099-B13","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1037\/h0043158","article-title":"The magical number seven plus or minus two: some limits on our capacity for processing information","volume":"63","author":"Miller","year":"1956","journal-title":"Psychol Rev"},{"issue":"2","key":"2020110613074949400_ocaa099-B14","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1080\/14639220210123806","article-title":"Multiple resources and performance prediction","volume":"3","author":"Wickens","year":"2002","journal-title":"Theor Issues Ergon Sci"},{"issue":"7","key":"2020110613074949400_ocaa099-B15","doi-asserted-by":"crossref","first-page":"1008","DOI":"10.1177\/0018720818781224","article-title":"The effect of cognitive load and task complexity on automation bias in electronic prescribing","volume":"60","author":"Lyell","year":"2018","journal-title":"Hum Factors"},{"key":"2020110613074949400_ocaa099-B16","first-page":"2","article-title":"Support for information management in critical care: a new approach to identify needs","author":"Rosenal","year":"1995","journal-title":"Proc Annu Symp Comput Appl Med Care"},{"issue":"3","key":"2020110613074949400_ocaa099-B17","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1197\/jamia.M1008","article-title":"Providing concept-oriented views for clinical data using a knowledge-based system: an evaluation","volume":"9","author":"Zeng","year":"2002","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110613074949400_ocaa099-B18","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1177\/001872089003200105","article-title":"Proximity compatibility and information display: effects of color, space, and object display on information integration","volume":"32","author":"Wickens","year":"1990","journal-title":"Hum Factors"},{"key":"2020110613074949400_ocaa099-B19","first-page":"377","article-title":"An object oriented computer-based patient record reference model","author":"Dore","year":"1995","journal-title":"Proc Annu Symp Comput Appl Med Care"},{"issue":"1","key":"2020110613074949400_ocaa099-B20","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1055\/s-0037-1619454","article-title":"Determining primary care physician information needs to inform ambulatory visit note display","volume":"5","author":"Clarke","year":"2014","journal-title":"Appl Clin Inform"},{"key":"2020110613074949400_ocaa099-B21","doi-asserted-by":"crossref","first-page":"171","DOI":"10.2147\/COPD.S16396","article-title":"Barriers to adherence to chronic obstructive pulmonary disease guidelines by primary care physicians","volume":"6","author":"Salinas","year":"2011","journal-title":"Int J Chron Obstruct Pulmon Dis"},{"issue":"11","key":"2020110613074949400_ocaa099-B22","first-page":"1586","article-title":"Implementing clinical guidelines for chronic obstructive pulmonary disease: barriers and solutions","volume":"6","author":"Overington","year":"2014","journal-title":"J Thorac Dis"},{"issue":"2","key":"2020110613074949400_ocaa099-B23","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1016\/j.amepre.2019.03.015","article-title":"Lung cancer screening utilization: a behavioral risk factor surveillance system analysis","volume":"57","author":"Zahnd","year":"2019","journal-title":"Am J Prev Med"},{"issue":"5","key":"2020110613074949400_ocaa099-B24","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1093\/jamia\/ocv189","article-title":"SMART on FHIR: a standards-based, interoperable apps platform for electronic health records","volume":"23","author":"Mandel","year":"2016","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2020110613074949400_ocaa099-B25","doi-asserted-by":"crossref","first-page":"e12902","DOI":"10.2196\/12902","article-title":"Beyond one-off integrations: a commercial, substitutable, reusable, standards-based, electronic health record-connected app","volume":"21","author":"Mandl","year":"2019","journal-title":"J Med Internet Res"},{"issue":"11","key":"2020110613074949400_ocaa099-B26","doi-asserted-by":"crossref","first-page":"e1915343","DOI":"10.1001\/jamanetworkopen.2019.15343","article-title":"Association of an electronic health record add-on app for neonatal bilirubin management with physician efficiency and care quality","volume":"2","author":"Kawamoto","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"10","key":"2020110613074949400_ocaa099-B27","doi-asserted-by":"crossref","first-page":"1070","DOI":"10.1001\/jama.2015.8372","article-title":"Electronic health record vendor adherence to usability certification requirements and testing standards","volume":"314","author":"Ratwani","year":"2015","journal-title":"JAMA"},{"issue":"4","key":"2020110613074949400_ocaa099-B28","doi-asserted-by":"crossref","first-page":"e191709","DOI":"10.1001\/jamanetworkopen.2019.1709","article-title":"Association of the usability of electronic health records with cognitive workload and performance levels among physicians","volume":"2","author":"Mazur","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"5","key":"2020110613074949400_ocaa099-B29","doi-asserted-by":"crossref","first-page":"1900164","DOI":"10.1183\/13993003.00164-2019","article-title":"Global strategy for the diagnosis, management, and prevention of chronic obstructive lung disease: the GOLD science committee report 2019","volume":"53","author":"Singh","year":"2019","journal-title":"Eur Respir J"},{"key":"2020110613074949400_ocaa099-B30","first-page":"373","author":"Nielsen","year":"1992"},{"key":"2020110613074949400_ocaa099-B31","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/S0166-4115(08)62386-9","article-title":"Development of NASA-TLX (Task Load Index): results of empirical and theoretical research","volume":"52","author":"Hart","year":"1988","journal-title":"Adv Psychol"},{"issue":"1","key":"2020110613074949400_ocaa099-B32","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1037\/0033-295X.94.1.3","article-title":"What do people think they\u2019re doing? Action identification and human behavior","volume":"94","author":"Vallacher","year":"1987","journal-title":"Psychol Rev"},{"issue":"1","key":"2020110613074949400_ocaa099-B33","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1186\/1471-2288-13-117","article-title":"Using the framework method for the analysis of qualitative data in multi-disciplinary health research","volume":"13","author":"Gale","year":"2013","journal-title":"BMC Med Res Methodol"},{"key":"2020110613074949400_ocaa099-B34","volume-title":"Qualitative Research & Evaluation Methods","author":"Patton","year":"2015","edition":"4th ed"},{"issue":"1","key":"2020110613074949400_ocaa099-B35","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"issue":"2","key":"2020110613074949400_ocaa099-B36","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1177\/160940690200100202","article-title":"Verification strategies for establishing reliability and validity in qualitative research","volume":"1","author":"Morse","year":"2002","journal-title":"Int J Qual Methods"},{"issue":"2","key":"2020110613074949400_ocaa099-B37","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1097\/01.NPR.0000476377.35114.d7","volume":"41","author":"Pearce","year":"2016","journal-title":"Nurse Pract"},{"issue":"9","key":"2020110613074949400_ocaa099-B38","doi-asserted-by":"crossref","first-page":"e0184032","DOI":"10.1371\/journal.pone.0184032","article-title":"Overview of preventive practices provided by primary care physicians: a cross-sectional study in Switzerland and France","volume":"12","author":"Sebo","year":"2017","journal-title":"PLoS One"},{"key":"2020110613074949400_ocaa099-B39","first-page":"2333392816650344","article-title":"A case study of visit-driven preventive care screening using clinical decision support: the need to redesign preventive care screening","volume":"3","author":"Garrison","year":"2016","journal-title":"Health Serv Res Manag Epidemiol"},{"key":"2020110613074949400_ocaa099-B40","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.ijmedinf.2016.01.011","article-title":"User perception and experience of the introduction of a novel critical care patient viewer in the ICU setting","volume":"88","author":"Dziadzko","year":"2016","journal-title":"Int J Med Inform"},{"key":"2020110613074949400_ocaa099-B41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ijmedinf.2016.12.005","article-title":"Opening the Duke electronic health record to apps: implementing SMART on FHIR","volume":"99","author":"Bloomfield","year":"2017","journal-title":"Int J Med Inform"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/8\/1225\/34153503\/ocaa099.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/8\/1225\/34153503\/ocaa099.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T19:17:24Z","timestamp":1604690244000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/8\/1225\/5876974"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,27]]},"references-count":41,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,7,27]]},"published-print":{"date-parts":[[2020,8,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa099","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,8]]},"published":{"date-parts":[[2020,7,27]]}}}