{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T03:17:23Z","timestamp":1781839043951,"version":"3.54.5"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,1,27]],"date-time":"2022-01-27T00:00:00Z","timestamp":1643241600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,27]],"date-time":"2022-01-27T00:00:00Z","timestamp":1643241600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100007229","name":"Harvard University","doi-asserted-by":"publisher","award":["Biomedical Informatics and Data Science Research Training (BIRT) Program"],"award-info":[{"award-number":["Biomedical Informatics and Data Science Research Training (BIRT) Program"]}],"id":[{"id":"10.13039\/100007229","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"U.S. Department of Health & Human Services | NIH | U.S. National Library of Medicine","doi-asserted-by":"publisher","award":["T15LM007092"],"award-info":[{"award-number":["T15LM007092"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In recent years, the number of digital health tools with the potential to significantly improve delivery of healthcare services has grown tremendously. However, the use of these tools in large, complex health systems remains comparatively limited. The adoption and implementation of digital health tools at an enterprise level is a challenge; few strategies exist to help tools cross the chasm from clinical validation to integration within the workflows of a large health system. Many previously proposed frameworks for digital health implementation are difficult to operationalize in these dynamic organizations. In this piece, we put forth nine dimensions along which clinically validated digital health tools should be examined by health systems prior to adoption, and propose strategies for selecting digital health tools and planning for implementation in this setting. By evaluating prospective tools along these dimensions, health systems can evaluate which existing digital health solutions are worthy of adoption, ensure they have sufficient resources for deployment and long-term use, and devise a strategic plan for implementation.<\/jats:p>","DOI":"10.1038\/s41746-022-00557-1","type":"journal-article","created":{"date-parts":[[2022,1,27]],"date-time":"2022-01-27T11:03:15Z","timestamp":1643281395000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":210,"title":["Deploying digital health tools within large, complex health systems: key considerations for adoption and implementation"],"prefix":"10.1038","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3833-7448","authenticated-orcid":false,"given":"Jayson S.","family":"Marwaha","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2166-0521","authenticated-orcid":false,"given":"Adam B.","family":"Landman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gabriel A.","family":"Brat","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Todd","family":"Dunn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William J.","family":"Gordon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,1,27]]},"reference":[{"key":"557_CR1","unstructured":"RockHealth. Q1 2021 Funding Report: digital health is all grown up. https:\/\/rockhealth.com\/reports\/q1-2021-funding-report-digital-health-is-all-grown-up\/ (2021)."},{"key":"557_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-020-00370-8","volume":"3","author":"NA Patel","year":"2020","unstructured":"Patel, N. A. & Butte, A. J. Characteristics and challenges of the clinical pipeline of digital therapeutics. npj Digital Med. 3, 1\u20135 (2020).","journal-title":"npj Digital Med."},{"key":"557_CR3","doi-asserted-by":"publisher","first-page":"1851","DOI":"10.1001\/jama.2014.2564","volume":"311","author":"AC Powell","year":"2014","unstructured":"Powell, A. C., Landman, A. B. & Bates, D. W. In search of a few good apps. JAMA 311, 1851\u20131852 (2014).","journal-title":"JAMA"},{"key":"557_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-019-0212-z","volume":"3","author":"WJ Gordon","year":"2020","unstructured":"Gordon, W. J., Landman, A., Zhang, H. & Bates, D. W. Beyond validation: getting health apps into clinical practice. npj Digital Med. 3, 1\u20136 (2020).","journal-title":"npj Digital Med."},{"key":"557_CR5","unstructured":"Abraham, T. Hospitals look to venture capital as R&D extension. https:\/\/www.healthcaredive.com\/news\/hospitals-look-to-venture-capital-as-rd-extension\/549854\/ (2019)."},{"key":"557_CR6","unstructured":"PWC. PwC US Health Top Health Issues 2021. https:\/\/www.pwc.com\/us\/en\/industries\/health-industries\/assets\/pwc-us-health-top-health-issues.pdf (2021)."},{"key":"557_CR7","doi-asserted-by":"publisher","unstructured":"Flannery, D. & Jarrin, R. Building a regulatory and payment framework flexible enough to withstand technological progress. Health Aff. https:\/\/doi.org\/10.1377\/hlthaff.2018.05151 (2018).","DOI":"10.1377\/hlthaff.2018.05151"},{"key":"557_CR8","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1038\/s41591-019-0651-8","volume":"26","author":"KE Morse","year":"2020","unstructured":"Morse, K. E., Bagley, S. C. & Shah, N. H. Estimate the hidden deployment cost of predictive models to improve patient care. Nat. Med. 26, 18\u201319 (2020).","journal-title":"Nat. Med."},{"key":"557_CR9","doi-asserted-by":"crossref","unstructured":"Henson, P., David, G., Albright, K. & Torous, J. Deriving a practical framework for the evaluation of health apps. Lancet Digit. Health 1, e52\u2013e54 (2019).","DOI":"10.1016\/S2589-7500(19)30013-5"},{"key":"557_CR10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-018-0076-7","volume":"2","author":"SC Mathews","year":"2019","unstructured":"Mathews, S. C. et al. Digital health: a path to validation. npj Digital Med. 2, 1\u20139 (2019).","journal-title":"npj Digital Med."},{"key":"557_CR11","doi-asserted-by":"publisher","first-page":"e022969","DOI":"10.1136\/bmjopen-2018-022969","volume":"8","author":"MH Van Velthoven","year":"2018","unstructured":"Van Velthoven, M. H., Smith, J., Wells, G. & Brindley, D. Digital health app development standards: a systematic review protocol. BMJ Open 8, e022969 (2018).","journal-title":"BMJ Open"},{"key":"557_CR12","doi-asserted-by":"publisher","first-page":"e73","DOI":"10.1016\/j.ijmedinf.2012.10.007","volume":"82","author":"K Cresswell","year":"2013","unstructured":"Cresswell, K. & Sheikh, A. Organizational issues in the implementation and adoption of health information technology innovations: an interpretative review. Int. J. Med. Inform. 82, e73\u2013e86 (2013).","journal-title":"Int. J. Med. Inform."},{"key":"557_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-020-00318-y","volume":"3","author":"RC Li","year":"2020","unstructured":"Li, R. C., Asch, S. M. & Shah, N. H. Developing a delivery science for artificial intelligence in healthcare. npj Digital Med. 3, 1\u20133 (2020).","journal-title":"npj Digital Med."},{"key":"557_CR14","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1016\/j.hjdsi.2017.09.003","volume":"6","author":"J Tseng","year":"2018","unstructured":"Tseng, J., Samagh, S., Fraser, D. & Landman, A. B. Catalyzing healthcare transformation with digital health: Performance indicators and lessons learned from a Digital Health Innovation Group. Healthcare 6, 150\u2013155 (2018).","journal-title":"Healthcare"},{"key":"557_CR15","doi-asserted-by":"publisher","first-page":"E1351","DOI":"10.1503\/cmaj.202434","volume":"193","author":"AA Verma","year":"2021","unstructured":"Verma, A. A. et al. Implementing machine learning in medicine. CMAJ 193, E1351\u2013E1357 (2021).","journal-title":"CMAJ"},{"key":"557_CR16","doi-asserted-by":"crossref","unstructured":"Reeves, J. J. et al. Rapid response to COVID-19: health informatics support for outbreak management in an academic health system. J. Am. Med. Inform. Assoc. 27, 853\u2013859 (2020).","DOI":"10.1093\/jamia\/ocaa037"},{"key":"557_CR17","unstructured":"Brodwin, E., Aguilar, M., STAT staff, Ross, C. & Palmer, K. \u2018It\u2019s really on them to learn\u2019: How the rapid rollout of AI tools has fueled frustration among clinicians - STAT. https:\/\/www.statnews.com\/2020\/12\/17\/artificial-intelligence-doctors-nurses-frustrated-ai-hospitals\/ (2020)."},{"key":"557_CR18","unstructured":"Adams, K. Which digital health tools helped hospitals weather the pandemic? 4 CIOs discuss. https:\/\/www.beckershospitalreview.com\/digital-transformation\/which-digital-health-tools-helped-hospitals-weather-the-pandemic-4-cios-discuss.html."},{"key":"557_CR19","unstructured":"Massgeneral. Mass General Provides Telemedicine in Response to COVID-19. https:\/\/advances.massgeneral.org\/research-and-innovation\/article.aspx?id=1201 (2020)."},{"key":"557_CR20","doi-asserted-by":"publisher","first-page":"e19533","DOI":"10.2196\/19533","volume":"4","author":"H Zhang","year":"2020","unstructured":"Zhang, H. et al. A web-based, mobile-responsive application to screen health care workers for COVID-19 symptoms: rapid design, deployment, and usage. JMIR Form. Res. 4, e19533 (2020).","journal-title":"JMIR Form. Res."},{"key":"557_CR21","doi-asserted-by":"publisher","first-page":"e210339","DOI":"10.1001\/jamahealthforum.2021.0339","volume":"2","author":"RN Shah","year":"2021","unstructured":"Shah, R. N., Nghiem, J. & Ranney, M. L. The rise of digital health and innovation centers at academic medical centers: time for a new industry relationship paradigm. JAMA Health Forum 2, e210339\u2013e210339 (2021).","journal-title":"JAMA Health Forum"},{"key":"557_CR22","doi-asserted-by":"crossref","unstructured":"Perakslis, E. & Ginsburg, G. S. Digital health-the need to assess benefits, risks, and value. JAMA 325, 127\u2013128 (2021).","DOI":"10.1001\/jama.2020.22919"},{"key":"557_CR23","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1370\/afm.1713","volume":"12","author":"T Bodenheimer","year":"2014","unstructured":"Bodenheimer, T. & Sinsky, C. From triple to quadruple aim: care of the patient requires care of the provider. Ann. Fam. Med. 12, 573\u2013576 (2014).","journal-title":"Ann. Fam. Med."},{"key":"557_CR24","doi-asserted-by":"publisher","unstructured":"Kim, E. et al. COVID-19 screening system utilizing daily symptom attestation helps identify hospital employees who should be tested to protect patients and co-workers. Infect. Control Hosp. Epidemiol. https:\/\/doi.org\/10.1017\/ice.2021.461 (2021).","DOI":"10.1017\/ice.2021.461"},{"key":"557_CR25","doi-asserted-by":"publisher","first-page":"1149","DOI":"10.1093\/jamia\/ocaa318","volume":"28","author":"K Jung","year":"2020","unstructured":"Jung, K. et al. A framework for making predictive models useful in practice. J. Am. Med. Inform. Assoc. 28, 1149\u20131158 (2020).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"557_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-021-00463-y","volume":"4","author":"SB Golas","year":"2021","unstructured":"Golas, S. B. et al. Predictive analytics and tailored interventions improve clinical outcomes in older adults: a randomized controlled trial. npj Digital Med. 4, 1\u201310 (2021).","journal-title":"npj Digital Med."},{"key":"557_CR27","doi-asserted-by":"crossref","unstructured":"Barker, W. & Johnson, C. The ecosystem of apps and software integrated with certified health information technology. J. Am. Med. Inform. Assoc. 28, 2379\u20132384 (2021).","DOI":"10.1093\/jamia\/ocab171"},{"key":"557_CR28","unstructured":"Ross, C., Palmer, K., Feuerstein, A. & Sheridan, K. Hospitals launch new venture to build a better digital health marketplace. https:\/\/www.statnews.com\/2021\/10\/05\/graphite-health-digital-marketplace-civicarx\/ (2021)."},{"key":"557_CR29","unstructured":"Department of Health and Human Services. Practices Guide. https:\/\/www.hhs.gov\/sites\/default\/files\/ocio\/eplc\/EPLC%20Archive%20Documents\/55-Data%20Use%20Agreement%20%28DUA%29\/eplc_dua_practices_guide.pdf (2012)."},{"key":"557_CR30","doi-asserted-by":"publisher","unstructured":"Kappelman & Lean, M. User engagement in the development, implementation, and use of information technologies. in Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences HICSS-94 (IEEE, 1994). https:\/\/doi.org\/10.1109\/hicss.1994.323467.","DOI":"10.1109\/hicss.1994.323467"},{"key":"557_CR31","doi-asserted-by":"publisher","first-page":"1928","DOI":"10.1111\/1475-6773.12581","volume":"52","author":"KM Cresswell","year":"2017","unstructured":"Cresswell, K. M. et al. Sustained user engagement in health information technology: the long road from implementation to system optimization of computerized physician order entry and clinical decision support systems for prescribing in hospitals in England. Health Serv. Res. 52, 1928 (2017).","journal-title":"Health Serv. Res."},{"key":"557_CR32","doi-asserted-by":"publisher","first-page":"e9","DOI":"10.1136\/amiajnl-2013-001684","volume":"20","author":"KM Cresswell","year":"2013","unstructured":"Cresswell, K. M., Bates, D. W. & Sheikh, A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J. Am. Med. Inform. Assoc. 20, e9 (2013).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"557_CR33","doi-asserted-by":"crossref","unstructured":"Cresswell, K., Morrison, Z., Crowe, S., Robertson, A. & Sheikh, A. Anything but engaged: user involvement in the context of a national electronic health record implementation. Inform. Prim. Care 19, 191\u2013206 (2011).","DOI":"10.14236\/jhi.v19i4.814"},{"key":"557_CR34","unstructured":"Dagroso, D. et al. Implementation of an obstetrics EMR module: overcoming user dissatisfaction. J. Healthc. Inf. Manag. 21, 87\u201394 (2007)."},{"key":"557_CR35","unstructured":"SAFER. Safer System Configuration. https:\/\/www.healthit.gov\/sites\/default\/files\/safer\/guides\/safer_system_configuration.pdf (2016)."},{"key":"557_CR36","unstructured":"SAFER. General instructions for the SAFER self-assessment guides. https:\/\/www.healthit.gov\/sites\/default\/files\/safer\/guides\/safer_system_interfaces.pdf (2016)."},{"key":"557_CR37","unstructured":"NEJM Catalyst. How to Avoid Common Pitfalls of Health IT Implementation. https:\/\/catalyst.nejm.org\/doi\/full\/10.1056\/CAT.20.0048 (2020)."},{"key":"557_CR38","unstructured":"Brigham Digital Innovation Hub. Can a Simple Checklist Help Effectively Launch Pilots for Digital Health? \u2013 Brigham Digital Innovation Hub. https:\/\/bwhihub.org\/news\/2018-3-5-can-a-simple-checklist-help-effectively-launch-pilots-for-digital-health\/ (2018)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00557-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00557-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00557-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,25]],"date-time":"2022-11-25T08:21:27Z","timestamp":1669364487000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00557-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,27]]},"references-count":38,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["557"],"URL":"https:\/\/doi.org\/10.1038\/s41746-022-00557-1","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,27]]},"assertion":[{"value":"4 August 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 December 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 January 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"WJG reports consulting income from the Office of the National Coordinator for Health IT and Novocardia Inc., both outside the scope of this work. ABL was previously a member of the Abbott Medical Device Cybersecurity Council. JSM, GAB, and TD declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"13"}}