{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T10:43:38Z","timestamp":1770893018807,"version":"3.50.1"},"reference-count":41,"publisher":"Oxford University Press (OUP)","issue":"11","license":[{"start":{"date-parts":[[2021,8,23]],"date-time":"2021-08-23T00:00:00Z","timestamp":1629676800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"name":"Stanford Health Care"},{"name":"The Department of Medicine"},{"name":"Stanford School of Medicine"},{"name":"Debra and Mark Leslie"},{"name":"AI in Healthcare"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,10,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Artificial intelligence (AI) and machine learning (ML) enabled healthcare is now feasible for many health systems, yet little is known about effective strategies of system architecture and governance mechanisms for implementation. Our objective was to identify the different computational and organizational setups that early-adopter health systems have utilized to integrate AI\/ML clinical decision support (AI-CDS) and scrutinize their trade-offs.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We conducted structured interviews with health systems with AI deployment experience about their organizational and computational setups for deploying AI-CDS at point of care.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We contacted 34 health systems and interviewed 20 healthcare sites (58% response rate). Twelve (60%) sites used the native electronic health record vendor configuration for model development and deployment, making it the most common shared infrastructure. Nine (45%) sites used alternative computational configurations which varied significantly. Organizational configurations for managing AI-CDS were distinguished by how they identified model needs, built and implemented models, and were separable into 3 major types: Decentralized translation (n\u2009=\u200910, 50%), IT Department led (n\u2009=\u20092, 10%), and AI in Healthcare (AIHC) Team (n\u2009=\u20098, 40%).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>No singular computational configuration enables all current use cases for AI-CDS. Health systems need to consider their desired applications for AI-CDS and whether investment in extending the off-the-shelf infrastructure is needed. Each organizational setup confers trade-offs for health systems planning strategies to implement AI-CDS.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>Health systems will be able to use this framework to understand strengths and weaknesses of alternative organizational and computational setups when designing their strategy for artificial intelligence.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocab154","type":"journal-article","created":{"date-parts":[[2021,7,12]],"date-time":"2021-07-12T11:08:44Z","timestamp":1626088124000},"page":"2445-2450","source":"Crossref","is-referenced-by-count":27,"title":["A survey of extant organizational and computational setups for deploying predictive models in health systems"],"prefix":"10.1093","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0276-2183","authenticated-orcid":false,"given":"Sehj","family":"Kashyap","sequence":"first","affiliation":[{"name":"Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, California, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8307-1642","authenticated-orcid":false,"given":"Keith E","family":"Morse","sequence":"additional","affiliation":[{"name":"Division of Pediatric Hospital Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7673-6143","authenticated-orcid":false,"given":"Birju","family":"Patel","sequence":"additional","affiliation":[{"name":"Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, California, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9385-7158","authenticated-orcid":false,"given":"Nigam H","family":"Shah","sequence":"additional","affiliation":[{"name":"Stanford Center for Biomedical Informatics Research, Stanford University, Stanford, California, USA"}]}],"member":"286","published-online":{"date-parts":[[2021,8,23]]},"reference":[{"issue":"6","key":"2021101218240910900_ocab154-B1","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1002\/msj.21351","article-title":"Future of electronic health records: implications for decision support","volume":"79","author":"Rothman","year":"2012","journal-title":"Mt Sinai J Med"},{"issue":"13","key":"2021101218240910900_ocab154-B2","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1056\/NEJMp1606181","article-title":"Predicting the future - big data, machine learning, and clinical medicine","volume":"375","author":"Obermeyer","year":"2016","journal-title":"N Engl J Med"},{"issue":"13","key":"2021101218240910900_ocab154-B3","doi-asserted-by":"crossref","first-page":"1317","DOI":"10.1001\/jama.2017.18391","article-title":"Big data and machine learning in health care","volume":"319","author":"Beam","year":"2018","journal-title":"JAMA"},{"issue":"20","key":"2021101218240910900_ocab154-B4","doi-asserted-by":"crossref","first-page":"1951","DOI":"10.1056\/NEJMsa2001090","article-title":"Automated identification of adults at risk for in-hospital clinical deterioration","volume":"383","author":"Escobar","year":"2020","journal-title":"N Engl J Med"},{"issue":"23","key":"2021101218240910900_ocab154-B5","doi-asserted-by":"crossref","first-page":"2281","DOI":"10.1001\/jama.2019.4914","article-title":"Artificial intelligence in health care: will the value match the hype?","volume":"321","author":"Emanuel","year":"2019","journal-title":"JAMA"},{"key":"2021101218240910900_ocab154-B6","year":"2020"},{"key":"2021101218240910900_ocab154-B7","year":"2020"},{"issue":"1","key":"2021101218240910900_ocab154-B8","first-page":"1","article-title":"Machine learning in health care: a critical appraisal of challenges and opportunities","volume":"7","author":"Sendak","year":"2019","journal-title":"EGEMS (Wash DC)"},{"issue":"4","key":"2021101218240910900_ocab154-B9","doi-asserted-by":"crossref","first-page":"e16848","DOI":"10.2196\/16848","article-title":"Development, implementation, and evaluation of a personalized machine learning algorithm for clinical decision support: case study with shingles vaccination","volume":"22","author":"Chen","year":"2020","journal-title":"J Med Internet Res"},{"key":"2021101218240910900_ocab154-B10","author":"Drysdale"},{"issue":"7","key":"2021101218240910900_ocab154-B11","doi-asserted-by":"crossref","first-page":"e15182","DOI":"10.2196\/15182","article-title":"Real-world integration of a sepsis deep learning technology into routine clinical care: implementation study","volume":"8","author":"Sendak","year":"2020","journal-title":"JMIR Med Inform"},{"key":"2021101218240910900_ocab154-B12","year":"2019"},{"key":"2021101218240910900_ocab154-B13","year":"2020"},{"key":"2021101218240910900_ocab154-B14","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1038\/s41746-020-00318-y","article-title":"Developing a delivery science for artificial intelligence in healthcare","volume":"3","author":"Li","year":"2020","journal-title":"NPJ Digit Med"},{"key":"2021101218240910900_ocab154-B15","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1145\/3351095.3372827","volume-title":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","author":"Sendak","year":"2020"},{"key":"2021101218240910900_ocab154-B16","author":"Timsina","year":"2019"},{"key":"2021101218240910900_ocab154-B17","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1093\/jamia\/ocaa319","article-title":"Recommendations for the safe, effective use of adaptive CDS in the US healthcare system: an AMIA position paper","author":"Petersen","year":"2021","journal-title":"J Am Med Inform Assoc"},{"key":"2021101218240910900_ocab154-B18","year":"2019"},{"key":"2021101218240910900_ocab154-B19","year":"2021"},{"issue":"5","key":"2021101218240910900_ocab154-B20","doi-asserted-by":"crossref","first-page":"810","DOI":"10.1055\/s-0039-1697905","article-title":"Towards a maturity model for clinical decision support operations","volume":"10","author":"Orenstein","year":"2019","journal-title":"Appl Clin Inform"},{"issue":"1","key":"2021101218240910900_ocab154-B21","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1108\/JHOM-03-2020-0074","article-title":"Organizational readiness for artificial intelligence in health care: insights for decision-making and practice","volume":"35","author":"Alami","year":"2020","journal-title":"J Health Organ Manag"},{"issue":"suppl 31","key":"2021101218240910900_ocab154-B22","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1200\/JCO.2019.37.31_suppl.131","article-title":"Derivation and implementation of a machine learning approach to prompt serious illness conversations among outpatients with cancer","volume":"37","author":"Parikh","year":"2019","journal-title":"JCO"},{"issue":"2","key":"2021101218240910900_ocab154-B23","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1093\/jamiaopen\/ooz046","article-title":"Overcoming barriers to the adoption and implementation of predictive modeling and machine learning in clinical care: what can we learn from US academic medical centers?","volume":"3","author":"Watson","year":"2020","journal-title":"JAMIA Open"},{"key":"2021101218240910900_ocab154-B24","author":"Sendak"},{"key":"2021101218240910900_ocab154-B25","first-page":"A7016","volume-title":"D15. Critical Care: Do We Have a Crystal Ball? Predicting Clinical Deterioration and Outcome in Critically Ill Patients","author":"Henry","year":"2017"},{"issue":"11","key":"2021101218240910900_ocab154-B26","doi-asserted-by":"crossref","first-page":"e22421","DOI":"10.2196\/22421","article-title":"Integrating a machine learning system into clinical workflows: qualitative study","volume":"22","author":"Sandhu","year":"2020","journal-title":"J Med Internet Res"},{"issue":"1","key":"2021101218240910900_ocab154-B27","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1038\/s41591-018-0307-0","article-title":"The practical implementation of artificial intelligence technologies in medicine","volume":"25","author":"He","year":"2019","journal-title":"Nat Med"},{"issue":"9","key":"2021101218240910900_ocab154-B28","doi-asserted-by":"crossref","first-page":"1337","DOI":"10.1038\/s41591-019-0548-6","article-title":"Do no harm: a roadmap for responsible machine learning for health care","volume":"25","author":"Wiens","year":"2019","journal-title":"Nat Med"},{"issue":"1","key":"2021101218240910900_ocab154-B29","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1186\/s12916-019-1426-2","article-title":"Key challenges for delivering clinical impact with artificial intelligence","volume":"17","author":"Kelly","year":"2019","journal-title":"BMC Med"},{"key":"2021101218240910900_ocab154-B30","author":"Paleyes","year":"2020"},{"key":"2021101218240910900_ocab154-B31"},{"issue":"1","key":"2021101218240910900_ocab154-B32","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1214\/aoms\/1177705148","article-title":"Snowball sampling","volume":"32","author":"Goodman","year":"1961","journal-title":"Ann Math Statist"},{"issue":"10","key":"2021101218240910900_ocab154-B33","doi-asserted-by":"crossref","first-page":"e1915997","DOI":"10.1001\/jamanetworkopen.2019.15997","article-title":"Machine learning approaches to predict 6-month mortality among patients with cancer","volume":"2","author":"Parikh","year":"2019","journal-title":"JAMA Netw Open"},{"key":"2021101218240910900_ocab154-B34","author":"Sculley","year":"2014"},{"issue":"2","key":"2021101218240910900_ocab154-B35","first-page":"345","article-title":"From development to deployment: dataset shift, causality, and shift-stable models in health AI","volume":"21","author":"Subbaswamy","year":"2020","journal-title":"Biostatistics"},{"issue":"12","key":"2021101218240910900_ocab154-B36","doi-asserted-by":"crossref","first-page":"1645","DOI":"10.1093\/jamia\/ocz145","article-title":"Prognostic models will be victims of their own success, unless\u2026","volume":"26","author":"Lenert","year":"2019","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2021101218240910900_ocab154-B37","doi-asserted-by":"crossref","first-page":"e100183","DOI":"10.1136\/bmjhci-2020-100183","article-title":"The clinical artificial intelligence department: a prerequisite for success","volume":"27","author":"Cosgriff","year":"2020","journal-title":"BMJ Health Care Inform"},{"key":"2021101218240910900_ocab154-B38","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-38319-4","volume-title":"Clinical Decision Support Systems: Theory and Practice","author":"Berner","year":"2007"},{"issue":"4","key":"2021101218240910900_ocab154-B39","first-page":"240","article-title":"The diffusion of decision support systems in healthcare: are we there yet?","volume":"45","author":"Wong","year":"2000","journal-title":"J Healthc Manag"},{"issue":"7","key":"2021101218240910900_ocab154-B40","doi-asserted-by":"crossref","first-page":"316","DOI":"10.2307\/30146474","article-title":"Improved perioperative antibiotic use and reduced surgical wound infections through use of computer decision analysis","volume":"10","author":"Larsen","year":"1989","journal-title":"Infect Control Hosp Epidemiol"},{"key":"2021101218240910900_ocab154-B41","article-title":"Google strikes deal with hospital chain to develop healthcare algorithms","author":"Evans","year":"2021"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/11\/2445\/40576517\/ocab154.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/11\/2445\/40576517\/ocab154.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,17]],"date-time":"2021-10-17T22:48:57Z","timestamp":1634510937000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/11\/2445\/6356337"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,23]]},"references-count":41,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,8,23]]},"published-print":{"date-parts":[[2021,10,12]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab154","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,11,1]]},"published":{"date-parts":[[2021,8,23]]}}}