{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T12:11:24Z","timestamp":1774959084091,"version":"3.50.1"},"reference-count":35,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2025,9,23]],"date-time":"2025-09-23T00:00:00Z","timestamp":1758585600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Automation of clinical orders in electronic health records (EHRs) has the potential to reduce clinician burden and enhance patient safety. However, determining which orders are appropriate for automation requires a structured framework to ensure clinical validity, transparency, and safety.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>To develop and validate a framework of desiderata for assessing the appropriateness of automating clinical orders in EHRs and to demonstrate its operational value in a live health system dataset.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>The study comprised 4 phases to move from concept generation to real-world demonstration. First, we conducted focus group analyses using ground theory to identify themes and developed desiderata informed by these themes and existing literature. We validated the desiderata by surveying clinicians at a single institution, presenting 10 use cases to and assessing perceived appropriateness, cognitive support, and patient safety using a 4-point Likert scale. Survey results were compared to a priori appropriateness designations using t-tests. To evaluate operational impact, we analyzed one year of order-based alerts and orders (1.4 million firings alert and 44.1 million orders, respectively) using filtering rules and association rule mining to identify candidate orders for automation and their impact.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We identified 8 desiderata for automated order appropriateness: logical consistency, data provenance, order transparency, context permanence, monitoring plans, trigger consistency, care team empowerment, and system accountability. Use cases deemed appropriate based on these criteria received significantly higher scores for appropriateness (3.13\u2009\u00b1\u20090.84 vs 2.30\u2009\u00b1\u20090.99), cognitive support (3.08\u2009\u00b1\u20090.82 vs 2.25\u2009\u00b1\u20090.94), and patient safety (3.08\u2009\u00b1\u20090.86 vs 2.21\u2009\u00b1\u20090.98) (all P\u2009&amp;lt;\u2009.001) compared to those considered inappropriate. Operational analysis revealed an alert firing 19\u2009109 times annually, with a 96% signed order rate, where automation could save an estimated 26.5 provider hours per year. Additionally, an association rule with 16\u2009628 occurrences (68.4% confidence) suggested automation could save 15.8 hours annually and yield 8000 additional appropriate orders.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>The desiderata align with clinician perceptions and provide a structured approach for evaluating automated orders. Our findings highlight the potential for automation of certain clinical orders to improve cognitive support while maintaining patient safety.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Healthcare systems should use these desiderata, coupled with data mining techniques, to systematically identify and govern appropriate automated orders. Further research is needed to validate operational scalability.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocaf152","type":"journal-article","created":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T11:48:33Z","timestamp":1756900113000},"page":"1899-1907","source":"Crossref","is-referenced-by-count":2,"title":["Development and application of desiderata for automated clinical ordering"],"prefix":"10.1093","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5959-1659","authenticated-orcid":false,"given":"Sameh N","family":"Saleh","sequence":"first","affiliation":[{"name":"Department of Biomedical and Health Informatics, Children\u2019s Hospital of Philadelphia , Philadelphia, PA 19146,","place":["United States"]},{"name":"Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania , Philadelphia, PA 19146,","place":["United States"]},{"name":"Department of Health Informatics, Inova Health System , Fairfax, VA 22042,","place":["United States"]}]},{"given":"Kevin B","family":"Johnson","sequence":"additional","affiliation":[{"name":"Department of Biomedical and Health Informatics, Children\u2019s Hospital of Philadelphia , Philadelphia, PA 19146,","place":["United States"]},{"name":"Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania , Philadelphia, PA 19146,","place":["United States"]}]}],"member":"286","published-online":{"date-parts":[[2025,9,23]]},"reference":[{"key":"2025112514465719100_ocaf152-B1","doi-asserted-by":"publisher","first-page":"50","DOI":"10.7326\/M18-0139","article-title":"Physician burnout in the electronic health record era: are we ignoring the real cause?","volume":"169","author":"Downing","year":"2018","journal-title":"Ann Intern Med"},{"key":"2025112514465719100_ocaf152-B2","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1056\/NEJMp1006114","article-title":"The \u201cmeaningful use\u201d regulation for electronic health records","volume":"363","author":"Blumenthal","year":"2010","journal-title":"N Engl J Med"},{"key":"2025112514465719100_ocaf152-B3","doi-asserted-by":"publisher","first-page":"476","DOI":"10.1016\/j.mayocp.2019.09.024","article-title":"The association between perceived electronic health record usability and professional burnout among US physicians","volume":"95","author":"Melnick","year":"2020","journal-title":"Mayo Clin Proc"},{"key":"2025112514465719100_ocaf152-B4","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1001\/jama.2022.12243","article-title":"Making electronic health records both SAFER and SMARTER","volume":"328","author":"Johnson","year":"2022","journal-title":"JAMA"},{"key":"2025112514465719100_ocaf152-B5","doi-asserted-by":"publisher","first-page":"1632","DOI":"10.1093\/jamia\/ocab059","article-title":"The association between perceived electronic health record usability and professional burnout among US nurses","volume":"28","author":"Melnick","year":"2021","journal-title":"J Am Med Inform Assoc"},{"key":"2025112514465719100_ocaf152-B6","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1370\/afm.2121","article-title":"Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations","volume":"15","author":"Arndt","year":"2017","journal-title":"Ann Fam Med"},{"key":"2025112514465719100_ocaf152-B7","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1197\/jamia.M2231","article-title":"A cognitive task analysis of information management strategies in a computerized provider order entry environment","volume":"14","author":"Weir","year":"2007","journal-title":"J Am Med Inf Assoc"},{"key":"2025112514465719100_ocaf152-B8","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1016\/S1532-0464(03)00062-5","article-title":"A framework for analyzing the cognitive complexity of computer-assisted clinical ordering","volume":"36","author":"Horsky","year":"2003","journal-title":"J Biomed Inform"},{"key":"2025112514465719100_ocaf152-B9","doi-asserted-by":"publisher","first-page":"1409","DOI":"10.1001\/archinte.163.12.1409","article-title":"Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review","volume":"163","author":"Kaushal","year":"2003","journal-title":"Arch Intern Med"},{"key":"2025112514465719100_ocaf152-B10","year":"2022"},{"key":"2025112514465719100_ocaf152-B11","doi-asserted-by":"publisher","first-page":"907","DOI":"10.14236\/jhi.v24i3.907","article-title":"Effectively reducing amylase testing using computer order entry in the emergency department: quality improvement without eliminating physician choice","volume":"24","author":"Jaeger","year":"2017","journal-title":"J Innov Health Inform"},{"key":"2025112514465719100_ocaf152-B12","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/j.ijmedinf.2017.10.018","article-title":"Longitudinal study of user experiences of a CPOE system in a pediatric hospital","volume":"109","author":"Baysari","year":"2018","journal-title":"Int J Med Inform"},{"key":"2025112514465719100_ocaf152-B13","first-page":"1105","article-title":"A comparison of medication administrations errors using CPOE orders vs handwritten orders for pediatric continuous drug infusions","volume":"2006","author":"Sowan","year":"2006","journal-title":"AMIA Annu Symp Proc"},{"key":"2025112514465719100_ocaf152-B14","doi-asserted-by":"publisher","first-page":"31","DOI":"10.7326\/0003-4819-139-1-200307010-00010","article-title":"Computer physician order entry: benefits, costs, and issues","volume":"139","author":"Kuperman","year":"2003","journal-title":"Ann Intern Med"},{"key":"2025112514465719100_ocaf152-B15","first-page":"26","article-title":"Some unintended consequences of clinical decision support systems","volume":"2007","author":"Ash","year":"2007","journal-title":"AMIA Annu Symp Proc"},{"key":"2025112514465719100_ocaf152-B16","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1197\/jamia.M2042","article-title":"Types of unintended consequences related to computerized provider order entry","volume":"13","author":"Campbell","year":"2006","journal-title":"J Am Med Inform Assoc"},{"key":"2025112514465719100_ocaf152-B17","volume-title":"The Design of Everyday Things. 1st Basic Paperback","author":"Norman","year":"2002"},{"key":"2025112514465719100_ocaf152-B18","article-title":"Cognition and human computer interaction in health and biomedicine","author":"Themes","year":"2023","journal-title":"Basicmedical Key"},{"key":"2025112514465719100_ocaf152-B19","first-page":"71","article-title":"Human factors considerations relevant to CPOE implementations","volume":"19","author":"Saathoff","year":"2005","journal-title":"J Healthc Inf Manag"},{"key":"2025112514465719100_ocaf152-B20","doi-asserted-by":"publisher","first-page":"e547","DOI":"10.1542\/peds.2011-3408","article-title":"Effects of CPOE on provider cognitive workload: a randomized crossover trial","volume":"130","author":"Avansino","year":"2012","journal-title":"Pediatrics"},{"key":"2025112514465719100_ocaf152-B21","doi-asserted-by":"publisher","first-page":"604977","DOI":"10.3389\/fpsyg.2021.604977","article-title":"Measurement of trust in automation: a narrative review and reference guide","volume":"12","author":"Kohn","year":"2021","journal-title":"Front Psychol"},{"key":"2025112514465719100_ocaf152-B22","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1177\/21695067231192208","article-title":"Evaluating responsivity and resilience in trust in healthcare systems: a scoping literature review","author":"Nare","year":"2023"},{"key":"2025112514465719100_ocaf152-B23","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1197\/jamia.M3092","article-title":"A \u201cFundamental Theorem\u201d of biomedical informatics","volume":"16","author":"Friedman","year":"2009","journal-title":"J Am Med Inf Assoc"},{"key":"2025112514465719100_ocaf152-B24","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1055\/s-0041-1731744","article-title":"Identifying opportunities for workflow automation in health care: lessons learned from other industries","volume":"12","author":"Zayas-Cab\u00e1n","year":"2021","journal-title":"Appl Clin Inform"},{"key":"2025112514465719100_ocaf152-B25","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1093\/jamia\/ocac197","article-title":"Priorities to accelerate workflow automation in health care","volume":"30","author":"Zayas-Cab\u00e1n","year":"2022","journal-title":"J Am Med Inform Assoc"},{"key":"2025112514465719100_ocaf152-B26","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1055\/s-0038-1634558","article-title":"Desiderata for controlled medical vocabularies in the twenty-first century","volume":"37","author":"Cimino","year":"1998","journal-title":"Methods Inf Med"},{"key":"2025112514465719100_ocaf152-B27","doi-asserted-by":"publisher","first-page":"299","DOI":"10.1016\/j.jbi.2005.11.008","article-title":"In defense of the desiderata","volume":"39","author":"Cimino","year":"2006","journal-title":"J Biomed Inform"},{"key":"2025112514465719100_ocaf152-B28","first-page":"270","volume-title":"Basics of Qualitative Research: Grounded Theory Procedures and Techniques","author":"Strauss","year":"1990"},{"key":"2025112514465719100_ocaf152-B29","doi-asserted-by":"publisher","first-page":"012048","DOI":"10.1088\/1742-6596\/705\/1\/012048","article-title":"Formal logic and flowchart for diagnosis validity verification and inclusion in clinical decision support systems","volume":"705","author":"Sosa","year":"2016","journal-title":"J Phys: Conf Ser"},{"key":"2025112514465719100_ocaf152-B30","doi-asserted-by":"publisher","first-page":"1068","DOI":"10.1093\/jamia\/ocw005","article-title":"Analysis of clinical decision support system malfunctions: a case series and survey","volume":"23","author":"Wright","year":"2016","journal-title":"J Am Med Inform Assoc"},{"key":"2025112514465719100_ocaf152-B31","doi-asserted-by":"publisher","first-page":"496","DOI":"10.1093\/jamia\/ocx106","article-title":"Clinical decision support alert malfunctions: analysis and empirically derived taxonomy","volume":"25","author":"Wright","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2025112514465719100_ocaf152-B32","doi-asserted-by":"publisher","first-page":"1552","DOI":"10.1093\/jamia\/ocy091","article-title":"Smashing the strict hierarchy: three cases of clinical decision support malfunctions involving carvedilol","volume":"25","author":"Wright","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2025112514465719100_ocaf152-B33","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1055\/s-0041-1722918","article-title":"Algorithmic detection of Boolean logic errors in clinical decision support statements","volume":"12","author":"Wright","year":"2021","journal-title":"Appl Clin Inform"},{"key":"2025112514465719100_ocaf152-B34","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1055\/s-0041-1741480","article-title":"Principles for designing and developing a workflow monitoring tool to enable and enhance clinical workflow automation","volume":"13","author":"Wu","year":"2022","journal-title":"Appl Clin Inform"},{"key":"2025112514465719100_ocaf152-B35","doi-asserted-by":"publisher","first-page":"2567","DOI":"10.1002\/mp.13552","article-title":"A risk assessment of automated treatment planning and recommendations for clinical deployment","volume":"46","author":"Kisling","year":"2019","journal-title":"Med Phys"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/12\/1899\/64352580\/ocaf152.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/12\/1899\/64352580\/ocaf152.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,25]],"date-time":"2025-11-25T19:47:06Z","timestamp":1764100026000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/32\/12\/1899\/8262150"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,23]]},"references-count":35,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2025,9,23]]},"published-print":{"date-parts":[[2025,12,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaf152","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,12]]},"published":{"date-parts":[[2025,9,23]]}}}