{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T22:34:06Z","timestamp":1774391646778,"version":"3.50.1"},"reference-count":33,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2021,2,12]],"date-time":"2021-02-12T00:00:00Z","timestamp":1613088000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"National Library of Medicine of the National Institutes of Health","award":["R01LM012854"],"award-info":[{"award-number":["R01LM012854"]}]},{"name":"National Library of Medicine of the National Institutes of Health","award":["T32LM012412"],"award-info":[{"award-number":["T32LM012412"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,6,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>The characteristics of clinician activities while interacting with electronic health record (EHR) systems can influence the time spent in EHRs and workload. This study aims to characterize EHR activities as tasks and define novel, data-driven metrics.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs. We developed metrics characterizing the prevalence of unique events and event repetition and applied them to categorize tasks into 4 complexity profiles. Between these profiles, Mann-Whitney U tests were applied to measure the differences in performance time, event type, and clinician prevalence, or the number of unique clinicians who were observed performing these tasks. In addition, we apply process mining frameworks paired with clinical annotations to support the validity of a sample of our identified tasks. We apply our approaches to learn tasks performed by nurses in the Vanderbilt University Medical Center neonatal intensive care unit.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We examined EHR audit logs generated by 33 neonatal intensive care unit nurses resulting in 57\u00a0234 sessions and 81 tasks. Our results indicated significant differences in performance time for each observed task complexity profile. There were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types between tasks of different complexities. We presented a sample of expert-reviewed, annotated task workflows supporting the interpretation of their clinical meaningfulness.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>The use of the audit log provides an opportunity to assist hospitals in further investigating clinician activities to optimize EHR workflows.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaa338","type":"journal-article","created":{"date-parts":[[2020,12,18]],"date-time":"2020-12-18T16:55:05Z","timestamp":1608310505000},"page":"1168-1177","source":"Crossref","is-referenced-by-count":32,"title":["Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning"],"prefix":"10.1093","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1508-1758","authenticated-orcid":false,"given":"Bob","family":"Chen","sequence":"first","affiliation":[{"name":"Epithelial Biology Center, Vanderbilt University Medical Center, Nashville, Tennessee, USA"},{"name":"Program in Chemical and Physical Biology, Vanderbilt University, Nashville, Tennessee, USA"}]},{"given":"Wael","family":"Alrifai","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"},{"name":"Department of Pediatrics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"}]},{"given":"Cheng","family":"Gao","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"}]},{"given":"Barrett","family":"Jones","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0415-4301","authenticated-orcid":false,"given":"Laurie","family":"Novak","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"}]},{"given":"Nancy","family":"Lorenzi","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"}]},{"given":"Daniel","family":"France","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology, Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, Tennessee, USA"}]},{"given":"Bradley","family":"Malin","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"},{"name":"Department of Biostatistics, School of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, USA"},{"name":"Department of Electrical Engineering and Computer Science, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA"}]},{"given":"You","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"},{"name":"Department of Electrical Engineering and Computer Science, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA"}]}],"member":"286","published-online":{"date-parts":[[2021,2,12]]},"reference":[{"issue":"8","key":"2021061318593539400_ocaa338-B1","doi-asserted-by":"crossref","first-page":"e199609","DOI":"10.1001\/jamanetworkopen.2019.9609","article-title":"Association of electronic health record design and use factors with clinician stress and burnout","volume":"2","author":"Kroth","year":"2019","journal-title":"JAMA Netw Open"},{"key":"2021061318593539400_ocaa338-B2","first-page":"SP345","article-title":"The impact of electronic health record use on physician productivity","volume":"19","author":"Adler-Milstein","year":"2013","journal-title":"Am J Manag Care"},{"issue":"1","key":"2021061318593539400_ocaa338-B3","doi-asserted-by":"crossref","first-page":"39","DOI":"10.4300\/JGME-D-15-00240.1","article-title":"Racing against the clock: internal medicine residents' time spent on electronic health records","volume":"8","author":"Chen","year":"2016","journal-title":"J Grad Med Educ"},{"issue":"e1","key":"2021061318593539400_ocaa338-B4","doi-asserted-by":"crossref","first-page":"e100","DOI":"10.1136\/amiajnl-2013-001875","article-title":"Electronic medical records and physician stress in primary care: results from the MEMO Study","volume":"21","author":"Babbott","year":"2014","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2021061318593539400_ocaa338-B5","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1370\/afm.2223","article-title":"Physician burnout: resilience training is only part of the solution","volume":"16","author":"Card","year":"2018","journal-title":"Ann Fam Med"},{"issue":"4","key":"2021061318593539400_ocaa338-B6","doi-asserted-by":"crossref","first-page":"479","DOI":"10.4300\/JGME-D-16-00123.1","article-title":"Electronic health record effects on work-life balance and burnout within the I3 population collaborative","volume":"9","author":"Robertson","year":"2017","journal-title":"J Grad Med Educ"},{"issue":"3","key":"2021061318593539400_ocaa338-B7","doi-asserted-by":"crossref","first-page":"169","DOI":"10.7326\/M18-3684","article-title":"Physician time spent using the electronic health record during outpatient encounters: a descriptive study","volume":"172","author":"Overhage","year":"2020","journal-title":"Ann Intern Med"},{"issue":"3","key":"2021061318593539400_ocaa338-B8","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1093\/jamia\/ocz196","article-title":"Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods","volume":"27","author":"Rule","year":"2020","journal-title":"J Am Med Inform Assoc"},{"issue":"11","key":"2021061318593539400_ocaa338-B9","doi-asserted-by":"crossref","first-page":"2355","DOI":"10.1007\/s11606-019-05025-3","article-title":"Variation in physicians' electronic health record documentation and potential patient harm from that variation","volume":"34","author":"Cohen","year":"2019","journal-title":"J Gen Intern Med"},{"key":"2021061318593539400_ocaa338-B10","first-page":"S253","article-title":"Measuring provider efficiency in Epic: a preliminary mixed-methods exploration of the provider efficiency profile","volume":"33","author":"Monsen","year":"2018","journal-title":"J Gen Intern Med"},{"issue":"5","key":"2021061318593539400_ocaa338-B11","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1370\/afm.2121","article-title":"Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations","volume":"15","author":"Arndt","year":"2017","journal-title":"Ann Fam Med"},{"issue":"4","key":"2021061318593539400_ocaa338-B12","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1093\/jamia\/ocz223","article-title":"Metrics for assessing physician activity using electronic health record log data","volume":"27","author":"Sinsky","year":"2020","journal-title":"J Am Med Inform Assoc"},{"key":"2021061318593539400_ocaa338-B13","doi-asserted-by":"crossref","first-page":"103343","DOI":"10.1016\/j.jbi.2019.103343","article-title":"EHR audit logs: A new goldmine for health services research?","volume":"101","author":"Adler-Milstein","year":"2020","journal-title":"J Biomed Inform"},{"key":"2021061318593539400_ocaa338-B14","first-page":"416","article-title":"Inferring clinical workflow efficiency via electronic medical record utilization","volume":"2015","author":"Chen","year":"2015","journal-title":"AMIA Annu Symp Proc"},{"issue":"7","key":"2021061318593539400_ocaa338-B15","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1016\/j.ijmedinf.2014.04.006","article-title":"We work with them? healthcare workers interpretation of organizational relations mined from electronic health records","volume":"83","author":"Chen","year":"2014","journal-title":"Int J Med Inform"},{"issue":"7","key":"2021061318593539400_ocaa338-B16","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1093\/jamia\/ocy009","article-title":"Interaction patterns of trauma providers are associated with length of stay","volume":"25","author":"Chen","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"11","key":"2021061318593539400_ocaa338-B17","doi-asserted-by":"crossref","first-page":"1606","DOI":"10.1164\/rccm.202008-3114LE","article-title":"Network analysis subtleties in ICU structures and outcomes","volume":"202","author":"Chen","year":"2020","journal-title":"Am J Respir Crit Care Med"},{"issue":"4\/5","key":"2021061318593539400_ocaa338-B18","first-page":"109","article-title":"Modeling care team structures in the neonatal intensive care unit through network analysis of EHR audit logs","volume":"58","author":"Chen","year":"2019","journal-title":"Methods Inf Med"},{"issue":"2","key":"2021061318593539400_ocaa338-B19","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1145\/362883.362920","article-title":"Characterizing web user sessions","volume":"28","author":"Arlitt","year":"2000","journal-title":"SIGMETRICS Perform Eval Rev"},{"key":"2021061318593539400_ocaa338-B20","first-page":"166","author":"Satop\u00e4\u00e4","year":"2011"},{"key":"2021061318593539400_ocaa338-B21","first-page":"133","author":"Ramos","year":"2003"},{"issue":"2","key":"2021061318593539400_ocaa338-B22","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/97.991133","article-title":"Face recognition using kernel principal component analysis","volume":"9","author":"Kim","year":"2002","journal-title":"IEEE Signal Process Lett"},{"issue":"1","key":"2021061318593539400_ocaa338-B23","doi-asserted-by":"crossref","first-page":"5415","DOI":"10.1038\/s41467-019-13055-y","article-title":"Automated optimized parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large datasets","volume":"10","author":"Belkina","year":"2019","journal-title":"Nat Commun"},{"issue":"1","key":"2021061318593539400_ocaa338-B24","doi-asserted-by":"crossref","first-page":"5233","DOI":"10.1038\/s41598-019-41695-z","article-title":"From Louvain to Leiden: guaranteeing well-connected communities","volume":"9","author":"Traag","year":"2019","journal-title":"Sci Rep"},{"key":"2021061318593539400_ocaa338-B25","first-page":"420","author":"Rozinat","year":"2006"},{"issue":"4","key":"2021061318593539400_ocaa338-B26","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1093\/beheco\/ark016","article-title":"The unequal variance t-test is an underused alternative to Student's t-test and the Mann\u2013Whitney U test","volume":"17","author":"Ruxton","year":"2006","journal-title":"Behav Ecol"},{"issue":"1","key":"2021061318593539400_ocaa338-B27","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s13059-017-1382-0","article-title":"SCANPY: large-scale single-cell gene expression data analysis","volume":"19","author":"Wolf","year":"2018","journal-title":"Genome Biol"},{"key":"2021061318593539400_ocaa338-B28","first-page":"136","article-title":"Burnout and EHR use among academic primary care physicians with varied clinical workloads","volume":"2019","author":"Tran","year":"2019","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"issue":"1","key":"2021061318593539400_ocaa338-B29","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1186\/s13643-019-0971-2","article-title":"Evaluation of interventions to improve electronic health record documentation within the inpatient setting: a protocol for a systematic review","volume":"8","author":"Varela","year":"2019","journal-title":"Syst. Rev"},{"issue":"9","key":"2021061318593539400_ocaa338-B30","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1097\/CIN.0000000000000329","article-title":"Use of electronic health records by nurses for symptom management in inpatient settings: a systematic review","volume":"35","author":"Ozkaynak","year":"2017","journal-title":"Comput Inform Nurs"},{"issue":"1","key":"2021061318593539400_ocaa338-B31","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1186\/1472-6947-13-84","article-title":"Analysis of the quality of hospital information systems audit trails","volume":"13","author":"Cruz-Correia","year":"2013","journal-title":"BMC Med Inform Decis Mak"},{"key":"2021061318593539400_ocaa338-B32","year":"2018"},{"key":"2021061318593539400_ocaa338-B33","first-page":"1008","article-title":"Breadcrumbs: assessing the feasibility of automating provider documentation using electronic health record activity","volume":"2018","author":"Tang","year":"2018","journal-title":"AMIA Annu Symp Proc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/6\/1168\/38615440\/ocaa338.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/6\/1168\/38615440\/ocaa338.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,13]],"date-time":"2021-06-13T19:00:56Z","timestamp":1623610856000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/6\/1168\/6133907"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,12]]},"references-count":33,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,2,12]]},"published-print":{"date-parts":[[2021,6,12]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa338","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,6,1]]},"published":{"date-parts":[[2021,2,12]]}}}