{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T07:21:06Z","timestamp":1781335266224,"version":"3.54.1"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T00:00:00Z","timestamp":1712620800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T00:00:00Z","timestamp":1712620800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000936","name":"Gordon and Betty Moore Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000936","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>When integrating AI tools in healthcare settings, complex interactions between technologies and primary users are not always fully understood or visible. This deficient and ambiguous understanding hampers attempts by healthcare organizations to adopt AI\/ML, and it also creates new challenges for researchers to identify opportunities for simplifying adoption and developing best practices for the use of AI-based solutions. Our study fills this gap by documenting the process of designing, building, and maintaining an AI solution called SepsisWatch at Duke University Health System. We conducted 20 interviews with the team of engineers and scientists that led the multi-year effort to build the tool, integrate it into practice, and maintain the solution. This \u201cAlgorithm Journey Map\u201d enumerates all social and technical activities throughout the AI solution\u2019s procurement, development, integration, and full lifecycle management. In addition to mapping the \u201cwho?\u201d and \u201cwhat?\u201d of the adoption of the AI tool, we also show several \u2018lessons learned\u2019 throughout the algorithm journey maps including modeling assumptions, stakeholder inclusion, and organizational structure. In doing so, we identify generalizable insights about how to recognize and navigate barriers to AI\/ML adoption in healthcare settings. We expect that this effort will further the development of best practices for operationalizing and sustaining ethical principles\u2014in algorithmic systems.<\/jats:p>","DOI":"10.1038\/s41746-024-01061-4","type":"journal-article","created":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T14:01:37Z","timestamp":1712671297000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["The algorithm journey map: a tangible approach to implementing AI solutions in healthcare"],"prefix":"10.1038","volume":"7","author":[{"given":"William","family":"Boag","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-1361-3744","authenticated-orcid":false,"given":"Alifia","family":"Hasan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4850-9757","authenticated-orcid":false,"given":"Jee Young","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mike","family":"Revoir","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0969-1982","authenticated-orcid":false,"given":"Marshall","family":"Nichols","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4015-3805","authenticated-orcid":false,"given":"William","family":"Ratliff","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Gao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7564-727X","authenticated-orcid":false,"given":"Shira","family":"Zilberstein","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zainab","family":"Samad","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zahra","family":"Hoodbhoy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mushyada","family":"Ali","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nida Saddaf","family":"Khan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manesh","family":"Patel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4929-9130","authenticated-orcid":false,"given":"Suresh","family":"Balu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5828-4497","authenticated-orcid":false,"given":"Mark","family":"Sendak","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,4,9]]},"reference":[{"key":"1061_CR1","doi-asserted-by":"publisher","first-page":"94","DOI":"10.7861\/futurehosp.6-2-94","volume":"6","author":"T Davenport","year":"2019","unstructured":"Davenport, T. & Kalakota, R. The potential for artificial intelligence in healthcare. Future Healthc. J. 6, 94\u201398 (2019).","journal-title":"Future Healthc. J."},{"key":"1061_CR2","first-page":"2","volume":"65","author":"E Park","year":"2023","unstructured":"Park, E. The AI Bill of Rights: a step in the right direction. Orange Cty. Lawyer Mag. 65, 2 (2023).","journal-title":"Orange Cty. Lawyer Mag."},{"key":"1061_CR3","unstructured":"Boag, W. Evidence-based AI Ethics. Dissertation, Massachusetts Institute of Technology (2022)."},{"key":"1061_CR4","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1038\/s41746-021-00524-2","volume":"4","author":"Q Zhou","year":"2021","unstructured":"Zhou, Q., Chen, Z.-h, Cao, Y.-h & Peng, S. Clinical impact and quality of randomized controlled trials involving interventions evaluating artificial intelligence prediction tools: a systematic review. NPJ Digit. Med. 4, 154 (2021).","journal-title":"NPJ Digit. Med."},{"key":"1061_CR5","doi-asserted-by":"publisher","first-page":"410","DOI":"10.1038\/s41563-019-0345-0","volume":"18","author":"P-HC Chen","year":"2019","unstructured":"Chen, P.-H. C., Liu, Y. & Peng, L. How to develop machine learning models for healthcare. Nat. Mater. 18, 410\u2013414 (2019).","journal-title":"Nat. Mater."},{"key":"1061_CR6","doi-asserted-by":"publisher","first-page":"1337","DOI":"10.1038\/s41591-019-0548-6","volume":"25","author":"J Wiens","year":"2019","unstructured":"Wiens, J. et al. Do no harm: a roadmap for responsible machine learning for health care. Nat. Med. 25, 1337\u20131340 (2019).","journal-title":"Nat. Med."},{"key":"1061_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1136\/bmjhci-2021-100444","volume":"28","author":"S Reddy","year":"2021","unstructured":"Reddy, S. et al. Evaluation framework to guide implementation of AI systems into healthcare settings. BMJ Health Care Inform. 28, 1 (2021).","journal-title":"BMJ Health Care Inform."},{"key":"1061_CR8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1136\/bmjhci-2021-100495","volume":"29","author":"D van de Sande","year":"2022","unstructured":"van de Sande, D. et al. Developing, implementing and governing artificial intelligence in medicine: a step-by-step approach to prevent an artificial intelligence winter. BMJ Health Care Inform. 29, 1 (2022).","journal-title":"BMJ Health Care Inform."},{"key":"1061_CR9","unstructured":"Tonekaboni, S. et al. How to validate machine learning models prior to deployment: silent trial protocol for evaluation of real-time models at ICU. In Proc. Conference on Health, Inference, and Learning, PMLR. Vol. 174 (eds Gerardo F., George H C., Tom P., Joyce C H., & Tristan N.) 169\u2013182 (Proceedings of Machine Learning Research (PMLR), 2022)."},{"key":"1061_CR10","doi-asserted-by":"publisher","first-page":"e2227779","DOI":"10.1001\/jamanetworkopen.2022.27779","volume":"5","author":"JH Lu","year":"2022","unstructured":"Lu, J. H. et al. Assessment of adherence to reporting guidelines by commonly used clinical prediction models from a single vendor: a systematic review. JAMA Netw. Open 5, e2227779 (2022).","journal-title":"JAMA Netw. Open"},{"key":"1061_CR11","doi-asserted-by":"publisher","first-page":"S25","DOI":"10.1002\/jhm.2661","volume":"11","author":"B Dummett","year":"2016","unstructured":"Dummett, B. et al. Incorporating an early detection system into routine clinical practice in two community hospitals. J. Hosp. Med. 11, S25\u2013S31 (2016).","journal-title":"J. Hosp. Med."},{"key":"1061_CR12","doi-asserted-by":"publisher","first-page":"e15182","DOI":"10.2196\/15182","volume":"8","author":"MP Sendak","year":"2020","unstructured":"Sendak, M. P. et al. Real-world integration of a sepsis deep learning technology into routine clinical care: implementation study. JMIR Med. Inform. 8, e15182 (2020b).","journal-title":"JMIR Med. Inform."},{"key":"1061_CR13","first-page":"18","volume":"1","author":"P Salber","year":"2015","unstructured":"Salber, P. Wachter\u2019s digital doctor explores the dark side of health IT. Mhealth 1, 18 (2015).","journal-title":"Mhealth"},{"key":"1061_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1287\/orsc.2018.1277","volume":"30","author":"R Huising","year":"2019","unstructured":"Huising, R. Moving off the map: how knowledge of organizational operations empowers and alienates. Organ. Sci. 30, 1 (2019).","journal-title":"Organ. Sci."},{"key":"1061_CR15","doi-asserted-by":"publisher","first-page":"81","DOI":"10.2307\/2928741","volume":"40","author":"L Daston","year":"1992","unstructured":"Daston, L. & Galison, P. The image of objectivity. Representations 40, 81\u2013128 (1992).","journal-title":"Representations"},{"key":"1061_CR16","doi-asserted-by":"crossref","unstructured":"D\u2019Ignazio, C. & Klein, L. F. Data Feminism (MIT Press, 2020).","DOI":"10.7551\/mitpress\/11805.001.0001"},{"key":"1061_CR17","doi-asserted-by":"crossref","unstructured":"Kim, J. Y. et al. Organizational governance of emerging technologies: AI adoption in healthcare. In 2023 ACM Conference on Fairness, Accountability, Transparency, FAccT \u201923, Chicago, IL, USA (ACM, 2023).","DOI":"10.1145\/3593013.3594089"},{"key":"1061_CR18","doi-asserted-by":"publisher","first-page":"887","DOI":"10.1002\/cncy.21946","volume":"125","author":"YK Heher","year":"2017","unstructured":"Heher, Y. K. & Chen, Y. Process mapping: a cornerstone of quality improvement. Cancer Cytopathol. 125, 887\u2013890 (2017).","journal-title":"Cancer Cytopathol."},{"key":"1061_CR19","unstructured":"Sendak, M. et al. Development and validation of ML-DQA\u2013a machine learning data quality assurance framework for healthcare. Machine Learning for Healthcare Conference. PMLR, 741\u2013759 (2022)."},{"key":"1061_CR20","doi-asserted-by":"publisher","first-page":"e2138823","DOI":"10.1001\/jamanetworkopen.2021.38823","volume":"4","author":"F Gesten","year":"2021","unstructured":"Gesten, F. & Evans, L. SEP-1\u2014taking the measure of a measure. JAMA Netw. Open 4, e2138823 (2021).","journal-title":"JAMA Netw. Open"},{"key":"1061_CR21","unstructured":"Haraway, D. Simians, Cyborgs, and Women: The Reinvention of Nature (Routledge, 1990)."},{"key":"1061_CR22","doi-asserted-by":"publisher","unstructured":"Moser, I. & Law, J. Fluids or flows? Information and qualculation in medical practice. Inf. Technol. People 19, (2006). https:\/\/doi.org\/10.1108\/09593840610649961.","DOI":"10.1108\/09593840610649961"},{"key":"1061_CR23","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1177\/0306312717706110","volume":"47","author":"S Pfotenhauer","year":"2017","unstructured":"Pfotenhauer, S. & Jasanoff, S. Panacea or diagnosis? Imaginaries of innovation and the \u2018MIT model\u2019 in three political cultures. Soc. Stud. Sci. 47, 783\u2013810 (2017).","journal-title":"Soc. Stud. Sci."},{"key":"1061_CR24","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1038\/s41746-020-0253-3","volume":"3","author":"MP Sendak","year":"2020","unstructured":"Sendak, M. P., Gao, M., Brajer, N. & Balu, S. Presenting machine learning model information to clinical end users with model facts labels. NPJ Digit. Med. 3, 41 (2020).","journal-title":"NPJ Digit. Med."},{"key":"1061_CR25","first-page":"2","volume":"15","author":"A Richardson","year":"2010","unstructured":"Richardson, A. Using customer journey maps to improve customer experience. Harv. Bus. Rev. 15, 2\u20135 (2010).","journal-title":"Harv. Bus. Rev."},{"key":"1061_CR26","unstructured":"Damelio, R. The Basics of Process Mapping (CRC Press, 2011)."},{"key":"1061_CR27","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-021-06254-1","volume":"21","author":"G Antonacci","year":"2021","unstructured":"Antonacci, G., Lennox, L., Barlow, J., Evans, L. & Reed, J. Process mapping in healthcare: a systematic review. BMC Health Serv. Res. 21, 342 (2021).","journal-title":"BMC Health Serv. Res."},{"key":"1061_CR28","first-page":"39","volume":"19","author":"AH Van de Ven","year":"2017","unstructured":"Van de Ven, A. H. The innovation journey: you can\u2019t control it, but you can learn to maneuver it. Innov.-Eur. J. Soc. Sci. Res. 19, 39\u201342 (2017).","journal-title":"Innov.-Eur. J. Soc. Sci. Res."},{"key":"1061_CR29","doi-asserted-by":"crossref","unstructured":"Kellogg, K. C. Challenging Operations: Medical Reform and Resistance in Surgery (University of Chicago Press, 2011).","DOI":"10.7208\/chicago\/9780226430010.001.0001"},{"key":"1061_CR30","unstructured":"Elish, M. C. & Watkins, E. A. Repairing innovation: a study of integrating AI in clinical care. New York: Data & Society Research Institute, 1\u201362 (2020). https:\/\/datasociety.net\/library\/repairing-innovation\/."},{"key":"1061_CR31","unstructured":"Duke University Institutional Review Board. Quality Improvement or Research Checklist. (Duke University Institutional Review Board, accessed 16 February 2024). https:\/\/irb.duhs.duke.edu\/sites\/default\/files\/2022-04\/qi_policy_and_checklist_4-14-2021.pdf."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01061-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01061-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01061-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T14:06:50Z","timestamp":1712671610000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01061-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,9]]},"references-count":31,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["1061"],"URL":"https:\/\/doi.org\/10.1038\/s41746-024-01061-4","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,9]]},"assertion":[{"value":"21 July 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 April 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"M.S., S.B., W.R., M.N., M.R. and M.G. reported co-inventing software at Duke University licensed by Duke University to external commercial entities including Clinetic, Cohere Med, Kela Health, and Fullsteam Health. M.S., S.B., M.N., M.R. and M.G. own equity in Clinetic. No other disclosures were reported.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"87"}}