{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T11:38:24Z","timestamp":1777030704497,"version":"3.51.4"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T00:00:00Z","timestamp":1557705600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T00:00:00Z","timestamp":1557705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Digital health solutions continue to grow in both number and capabilities. Despite these advances, the confidence of the various stakeholders \u2014 from patients and clinicians to payers, industry and regulators \u2014 in medicine remains quite low. As a result, there is a need for objective, transparent, and standards-based evaluation of digital health products that can bring greater clarity to the digital health marketplace. We believe an approach that is guided by end-user requirements and formal assessment across technical, clinical, usability, and cost domains is one possible solution. For digital health solutions to have greater impact, quality and value must be easier to distinguish. To that end, we review the existing landscape and gaps, highlight the evolving responses and approaches, and detail one pragmatic framework that addresses the current limitations in the marketplace with a path toward implementation.<\/jats:p>","DOI":"10.1038\/s41746-019-0111-3","type":"journal-article","created":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T22:03:33Z","timestamp":1557785013000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":411,"title":["Digital health: a path to validation"],"prefix":"10.1038","volume":"2","author":[{"given":"Simon C.","family":"Mathews","sequence":"first","affiliation":[]},{"given":"Michael J.","family":"McShea","sequence":"additional","affiliation":[]},{"given":"Casey L.","family":"Hanley","sequence":"additional","affiliation":[]},{"given":"Alan","family":"Ravitz","sequence":"additional","affiliation":[]},{"given":"Alain B.","family":"Labrique","sequence":"additional","affiliation":[]},{"given":"Adam B.","family":"Cohen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,5,13]]},"reference":[{"key":"111_CR1","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1097\/00004479-200004000-00003","volume":"23","author":"SR Frank","year":"2000","unstructured":"Frank, S. R. Digital health care\u2014the convergence of health care and the Internet. J. Ambul. care Manag. 23, 8\u201317 (2000).","journal-title":"J. Ambul. care Manag."},{"key":"111_CR2","unstructured":"Rock Health. 2016 Year End Funding Report: A Reality Check for Digital Health. (San Francisco, CA, Rock Health, 2017). https:\/\/rockhealth.com\/reports\/2016-year-end-funding-report-a-reality-check-for-digital-health\/."},{"key":"111_CR3","unstructured":"World Health Organization. Classification of digital health interventions v1. 0: a shared language to describe the uses of digital technology for health. World Health Organization (2018)."},{"key":"111_CR4","unstructured":"Rock Health. 2017 Year End Funding Report: The End of the Beginning of Digital Health. https:\/\/rockhealth.com\/reports\/2017-year-end-funding-report-the-end-of-the-beginning-of-digital-health\/ (2018)."},{"key":"111_CR5","unstructured":"IQVIA. IQVIA Institute for Human Data Science Study: Impact of Digital Health Grows as Innovation, Evidence and Adoption of Mobile Health Apps Accelerate. https:\/\/www.iqvia.com\/newsroom\/2017\/11\/impact-of-digital-health-grows-as-innovation-evidence-and-adoption-of-mobile-health-apps-accelerate\/ (2017)."},{"key":"111_CR6","doi-asserted-by":"publisher","first-page":"704","DOI":"10.1001\/jamainternmed.2016.0326","volume":"176","author":"AJ Schoenfeld","year":"2016","unstructured":"Schoenfeld, A. J., Sehgal, N. J. & Auerbach, A. The challenges of mobile health regulation. JAMA Intern Med. 176, 704\u2013705 (2016).","journal-title":"JAMA Intern Med."},{"key":"111_CR7","doi-asserted-by":"publisher","first-page":"9","DOI":"10.2196\/jmir.2673","volume":"16","author":"TL Lewis","year":"2014","unstructured":"Lewis, T. L. & Wyatt, J. C. mHealth and mobile medical apps: a framework to assess risk and promote safer use. J. Med. Internet Res. 16, 9 (2014).","journal-title":"J. Med. Internet Res."},{"key":"111_CR8","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1089\/dia.2008.0283","volume":"10","author":"CC Quinn","year":"2008","unstructured":"Quinn, C. C. et al. WellDoc\u2122 mobile diabetes management randomized controlled trial: change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol. Ther. 10, 160\u2013168 (2008).","journal-title":"Diabetes Technol. Ther."},{"key":"111_CR9","doi-asserted-by":"publisher","first-page":"1062","DOI":"10.1111\/bjd.16667","volume":"179","author":"MT Svendsen","year":"2018","unstructured":"Svendsen, M. T. et al. A smartphone application supporting patients with psoriasis improves adherence to topical treatment: a randomized controlled trial. Br. J. Dermatol. 179, 1062\u20131071 (2018).","journal-title":"Br. J. Dermatol."},{"key":"111_CR10","doi-asserted-by":"publisher","first-page":"700","DOI":"10.1001\/jamainternmed.2016.0157","volume":"176","author":"TB Plante","year":"2016","unstructured":"Plante, T. B. et al. Validation of the instant blood pressure smartphone app. JAMA Intern. Med. 176, 700\u2013702 (2016).","journal-title":"JAMA Intern. Med."},{"key":"111_CR11","unstructured":"Veazie, S. et al. Mobile Applications for Self-Management of Diabetes. Technical Brief No. 31. (Prepared by the Scientific Resource Center under Contract Nos. 290-2012-0004-C and 290-2017-00003-C.) AHRQ Publication No. 18-EHC010-EF (Agency for Healthcare Research and Quality, Rockville, MD, 2018)."},{"key":"111_CR12","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1177\/0091270004273680","volume":"45","author":"CM Smith","year":"2015","unstructured":"Smith, C. M. Origin and uses of primum non nocere\u2014above all, do no harm! J. Clin. Pharmacol. 45, 371\u2013377 (2015).","journal-title":"J. Clin. Pharmacol."},{"key":"111_CR13","unstructured":"21st Century Cures Act, H.R. 34, 114th Cong. https:\/\/www.congress.gov\/114\/plaws\/publ255\/PLAW-114publ255.pdf (2015)."},{"key":"111_CR14","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1001\/jama.2018.8832","volume":"320","author":"J Shuren","year":"2018","unstructured":"Shuren, J., Patel, B. & Gottlieb, S. FDA regulation of mobile medical apps. JAMA 320, 337\u2013338 (2018).","journal-title":"JAMA"},{"key":"111_CR15","unstructured":"Federal Trade Commission. Mobile Health Apps Interactive Tool. https:\/\/www.ftc.gov\/tips-advice\/business-center\/guidance\/mobile-health-apps-interactive-tool (2016)."},{"key":"111_CR16","unstructured":"AI in Healthcare. Happtique Moving Forward with app Certification Program. https:\/\/www.aiin.healthcare\/topics\/connected-care\/happtique-moving-forward-app-certification-program (2012)."},{"key":"111_CR17","unstructured":"HIStalk. The Rise and Fall of Happtique, mHealth\u2019s First App Prescribing Platform. http:\/\/histalkmobile.com\/the-rise-and-fall-of-happtique-mhealths-first-app-prescribing-platform\/ (2014)."},{"key":"111_CR18","unstructured":"Personal Connected Health Alliance. Personal Connected Health. http:\/\/www.pchalliance.org\/ (2017)."},{"key":"111_CR19","unstructured":"Xcertia. mHealth App Guidelines. http:\/\/xcertia.org\/ (2017)."},{"key":"111_CR20","unstructured":"NHS. NHS Apps Library. https:\/\/apps.beta.nhs.uk\/ (2018)."},{"key":"111_CR21","unstructured":"Ranked Health. Curated Health Apps and Devices With a Focus on Clinical Relevance, Safety, and Efficacy. http:\/\/www.rankedhealth.com\/ (2018)."},{"key":"111_CR22","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1007\/s13142-014-0293-9","volume":"4","author":"ED Boudreaux","year":"2014","unstructured":"Boudreaux, E. D. et al. Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations. Transl. Behav. Med. 4, 363\u2013371 (2014).","journal-title":"Transl. Behav. Med."},{"key":"111_CR23","first-page":"229","volume":"5","author":"MN Boulos","year":"2014","unstructured":"Boulos, M. N., Brewer, A. C., Karimkhani, C., Buller, D. B. & Dellavalle, R. P. Mobile medical and health apps: state of the art, concerns, regulatory control and certification. Online J. public health Inform. 5, 229 (2014).","journal-title":"Online J. public health Inform."},{"key":"111_CR24","unstructured":"UL. About UL. https:\/\/www.ul.com\/aboutul\/ (2018)."},{"key":"111_CR25","unstructured":"Consumer Reports. About Us. https:\/\/www.consumerreports.org\/cro\/about-us\/what-we-do\/index.htm (2018)."},{"key":"111_CR26","doi-asserted-by":"publisher","first-page":"2805","DOI":"10.1093\/eurheartj\/ehw632","volume":"38","author":"JE Sharman","year":"2017","unstructured":"Sharman, J. E. et al. Validation of non-invasive central blood pressure devices: ARTERY Society task force consensus statement on protocol standardization. Eur. Heart J. 38, 2805\u20132812 (2017).","journal-title":"Eur. Heart J."},{"key":"111_CR27","doi-asserted-by":"crossref","unstructured":"Safavi, K., Mathews, S. C., Bates, D. W., Dorsey, E. R. & Cohen, A. B. Top-funded digital health companies and their impact on high-burden, high-cost conditions. Health Affairs. 38,115\u201323 (2019).","DOI":"10.1377\/hlthaff.2018.05081"},{"key":"111_CR28","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1097\/PHH.0000000000000665","volume":"24","author":"E Hatef","year":"2018","unstructured":"Hatef, E., Sharfstein, J. M. & Labrique, A. B. Innovation and entrepreneurship: harnessing the public health skill set in a new era of health reforms and investment. J. Public Health Manag. Pract. 24, 99\u2013101 (2018).","journal-title":"J. Public Health Manag. Pract."},{"key":"111_CR29","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1000094","volume":"6","author":"BP Kavanagh","year":"2009","unstructured":"Kavanagh, B. P. The GRADE system for rating clinical guidelines. PLoS Med. 6, e1000094 (2009).","journal-title":"PLoS Med."},{"issue":"Supplement 1","key":"111_CR30","doi-asserted-by":"publisher","first-page":"S13","DOI":"10.2337\/dc19-S002","volume":"42","author":"American Diabetes Association.","year":"2019","unstructured":"American Diabetes Association. 2. Classification and diagnosis of diabetes: standards of medical care in diabetes\u20142019. Diabetes Care. 42(Supplement 1), S13\u2013S28 (2019).","journal-title":"Diabetes Care."},{"key":"111_CR31","unstructured":"International Consortium for Health Outcomes Measurement. ICHOM Diabetes Standard Set Released On World Diabetes Day 2018. https:\/\/www.ichom.org\/news\/ichom-diabetes-standard-set-released-on-world-diabetes-day-2018\/ (2018)."},{"key":"111_CR32","unstructured":"U.S. Food & Drug Administration. Postmarket Requirements (Devices). https:\/\/www.fda.gov\/medicaldevices\/deviceregulationandguidance\/postmarketrequirements\/default.htm (2018)."},{"key":"111_CR33","unstructured":"U.S. Food & Drug Administration. Postmarket Surveillance Under Section 522 of the Federal Food, Drug, and Cosmetic Act. https:\/\/www.fda.gov\/downloads\/MedicalDevices\/DeviceRegulationandGuidance\/GuidanceDocuments\/UCM268141.pdf (2016)."},{"key":"111_CR34","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-018-0039-z","volume":"1","author":"TB Plante","year":"2018","unstructured":"Plante, T. B. et al. User experience of instant blood pressure: exploring reasons for the popularity of an inaccurate mobile health app. npj Digit. Med. 1, 31 (2018).","journal-title":"npj Digit. Med."},{"key":"111_CR35","unstructured":"World Wide Web Consortium. Notes on User Centered Design Process. http:\/\/www.w3.org\/WAI\/redesign\/ucd (2008)."},{"key":"111_CR36","unstructured":"International Organization for Standardization. ISO 9241-210:2010 Ergonomics of human-system interaction \u2013 Part 210: Human-centred design for interactive systems. https:\/\/www.iso.org\/standard\/52075.html (2010)."},{"key":"111_CR37","unstructured":"UXChallenge. UX Evaluation Criteria. http:\/\/uxchallenge.org\/user-experience-ux-evaluation-criteria (2018)."},{"key":"111_CR38","unstructured":"Nielsen Norman Group. 10 Usability Heuristics for User Interface Design. https:\/\/www.nngroup.com\/articles\/ten-usability-heuristics\/ (1995)."},{"key":"111_CR39","first-page":"70","volume":"4","author":"JB Smelcer","year":"2009","unstructured":"Smelcer, J. B., Miller-Jacobs, H. & Kantrovich, L. Usability of Electronic Medical Records. J. Usability Stud. 4, 70\u201384 (2009).","journal-title":"J. Usability Stud."},{"key":"111_CR40","doi-asserted-by":"crossref","unstructured":"Iribarren, S. J., Cato, K., Falzon, L. & Stone, P. W. What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions. PLoS ONE 12, e0170581 (2017).","DOI":"10.1371\/journal.pone.0170581"},{"key":"111_CR41","unstructured":"Forbes. Credit Scores Vs. Credit Reports: What\u2019s the Difference? https:\/\/www.forbes.com\/sites\/learnvest\/2017\/02\/17\/credit-scores-vs-credit-reports-whats-the-difference\/#64cc151b35ed (2017)."},{"key":"111_CR42","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1377\/hlthaff.2012.0693","volume":"32","author":"AL Kellermann","year":"2013","unstructured":"Kellermann, A. L. & Jones, S. S. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff. 32, 63\u201368 (2013).","journal-title":"Health Aff."},{"key":"111_CR43","unstructured":"CNBC. CVS creates new health-care giant as $69 billion merger with Aetna official closes. https:\/\/www.cnbc.com\/2018\/11\/28\/cvs-creates-new-health-care-giant-as-69-billion-aetna-merger-closes.html (2018)."},{"key":"111_CR44","unstructured":"USA Today. How Amazon, JPMorgan, Berkshire could transform American health care. https:\/\/www.usatoday.com\/story\/money\/2018\/02\/25\/amazon-jpmorgan-berkshire-health-care\/350625002\/ (2018)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0111-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0111-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0111-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,17]],"date-time":"2022-12-17T18:27:22Z","timestamp":1671301642000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0111-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,13]]},"references-count":44,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["111"],"URL":"https:\/\/doi.org\/10.1038\/s41746-019-0111-3","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,13]]},"assertion":[{"value":"4 January 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 April 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 May 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"38"}}