{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T11:14:02Z","timestamp":1776165242006,"version":"3.50.1"},"reference-count":46,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2019,6,13]],"date-time":"2019-06-13T00:00:00Z","timestamp":1560384000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Predictive analytics in health care has generated increasing enthusiasm recently, as reflected in a rapidly growing body of predictive models reported in literature and in real-time embedded models using electronic health record data. However, estimating the benefit of applying any single model to a specific clinical problem remains challenging today. Developing a shared framework for estimating model value is therefore critical to facilitate the effective, safe, and sustainable use of predictive tools into the future. We highlight key concepts within the prediction-action dyad that together are expected to impact model benefit. These include factors relevant to model prediction (including the number needed to screen) as well as those relevant to the subsequent action (number needed to treat). In the simplest terms, a number needed to benefit contextualizes the numbers needed to screen and treat, offering an opportunity to estimate the value of a clinical predictive model in action.<\/jats:p>","DOI":"10.1093\/jamia\/ocz088","type":"journal-article","created":{"date-parts":[[2019,5,18]],"date-time":"2019-05-18T11:08:43Z","timestamp":1558177723000},"page":"1655-1659","source":"Crossref","is-referenced-by-count":51,"title":["The number needed to benefit: estimating the value of predictive analytics in healthcare"],"prefix":"10.1093","volume":"26","author":[{"given":"Vincent X","family":"Liu","sequence":"first","affiliation":[{"name":"Division of Research, Kaiser Permanente, Oakland, California, USA"}]},{"given":"David W","family":"Bates","sequence":"additional","affiliation":[{"name":"Division of General Internal Medicine, Brigham and Women\u2019s Hospital, and Harvard Medical School, Boston, Massachusetts, USA"}]},{"given":"Jenna","family":"Wiens","sequence":"additional","affiliation":[{"name":"Division of Computer Science and Engineering, College of Engineering, University of Michigan, Ann Arbor, Michigan, USA"}]},{"given":"Nigam H","family":"Shah","sequence":"additional","affiliation":[{"name":"Division of Biomedical Informatics Research, Stanford University, Stanford, California, USA"}]}],"member":"286","published-online":{"date-parts":[[2019,6,13]]},"reference":[{"issue":"7","key":"2020110612453857400_ocz088-B1","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1001\/jama.2015.19417","article-title":"Integrating predictive analytics into high-value care: the dawn of precision delivery","volume":"315","author":"Parikh","year":"2016","journal-title":"JAMA"},{"issue":"17","key":"2020110612453857400_ocz088-B2","doi-asserted-by":"crossref","first-page":"1609","DOI":"10.1056\/NEJMp1613224","article-title":"Beyond genes and molecules\u2014a precision delivery initiative for precision medicine","volume":"376","author":"Parikh","year":"2017","journal-title":"N Engl J Med"},{"issue":"23","key":"2020110612453857400_ocz088-B3","doi-asserted-by":"crossref","first-page":"2161","DOI":"10.1056\/NEJMp1401111","article-title":"Learning from big health care data","volume":"370","author":"Schneeweiss","year":"2014","journal-title":"N Engl J Med"},{"issue":"13","key":"2020110612453857400_ocz088-B4","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1001\/jama.2013.393","article-title":"The inevitable application of big data to health care","volume":"309","author":"Murdoch","year":"2013","journal-title":"JAMA"},{"issue":"13","key":"2020110612453857400_ocz088-B5","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1056\/NEJMp1606181","article-title":"Predicting the future\u2014big data, machine learning, and clinical medicine","volume":"375","author":"Obermeyer","year":"2016","journal-title":"N Engl J Med"},{"issue":"26","key":"2020110612453857400_ocz088-B6","doi-asserted-by":"crossref","first-page":"2507","DOI":"10.1056\/NEJMp1702071","article-title":"Machine learning and prediction in medicine\u2014beyond the peak of inflated expectations","volume":"376","author":"Chen","year":"2017","journal-title":"N Engl J Med"},{"issue":"1","key":"2020110612453857400_ocz088-B7","doi-asserted-by":"crossref","first-page":"1.","DOI":"10.1093\/jamia\/ocw163","article-title":"Using health information technology for clinical decision support and predictive analytics","volume":"24","author":"Ohno-Machado","year":"2017","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110612453857400_ocz088-B8","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1093\/jamia\/ocw042","article-title":"Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review","volume":"24","author":"Goldstein","year":"2017","journal-title":"J Am Med Inform Assoc"},{"issue":"8","key":"2020110612453857400_ocz088-B9","doi-asserted-by":"crossref","first-page":"969","DOI":"10.1093\/jamia\/ocy032","article-title":"Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare data","volume":"25","author":"Reps","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"7","key":"2020110612453857400_ocz088-B10","doi-asserted-by":"crossref","first-page":"1530","DOI":"10.1093\/aje\/kwy018","article-title":"Landmark models for optimizing the use of repeated measurements of risk factors in electronic health records to predict future disease risk","volume":"187","author":"Paige","year":"2018","journal-title":"Am J Epidemiol"},{"issue":"7","key":"2020110612453857400_ocz088-B11","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.1377\/hlthaff.2014.0041","article-title":"Big data in health care: using analytics to identify and manage high-risk and high-cost patients","volume":"33","author":"Bates","year":"2014","journal-title":"Health Aff (Millwood)"},{"issue":"21","key":"2020110612453857400_ocz088-B12","doi-asserted-by":"crossref","first-page":"2345","DOI":"10.1001\/jama.2009.1757","article-title":"Assessment of claims of improved prediction beyond the Framingham risk score","volume":"302","author":"Tzoulaki","year":"2009","journal-title":"JAMA"},{"issue":"10","key":"2020110612453857400_ocz088-B13","doi-asserted-by":"crossref","first-page":"1063","DOI":"10.1001\/jama.2015.11082","article-title":"Evaluating discrimination of risk prediction models: the C statistic","volume":"314","author":"Pencina","year":"2015","journal-title":"JAMA"},{"issue":"4","key":"2020110612453857400_ocz088-B14","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4","article-title":"Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors","volume":"15","author":"Harrell","year":"1996","journal-title":"Stat Med"},{"key":"2020110612453857400_ocz088-B15","doi-asserted-by":"crossref","first-page":"S25","DOI":"10.1002\/jhm.2661","article-title":"Incorporating an early detection system into routine clinical practice in two community hospitals","volume":"11 (Suppl 1)","author":"Dummett","year":"2016","journal-title":"J Hosp Med"},{"issue":"6","key":"2020110612453857400_ocz088-B16","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1197\/jamia.M1370","article-title":"Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality","volume":"10","author":"Bates","year":"2003","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110612453857400_ocz088-B17","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1093\/jamia\/ocx118","article-title":"Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support","volume":"25","author":"Miller","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2020110612453857400_ocz088-B18","first-page":"1189","article-title":"Automating guidelines for clinical decision support: knowledge engineering and implementation","volume":"2016","author":"Tso","year":"2016","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110612453857400_ocz088-B19","first-page":"S103\u201316.","article-title":"Clinical decision support: a 25 year retrospective and a 25 year vision","volume":"Suppl 1","author":"Middleton","year":"2016","journal-title":"Yearb Med Inform"},{"issue":"2","key":"2020110612453857400_ocz088-B20","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1016\/j.jbi.2007.09.003","article-title":"Grand challenges in clinical decision support","volume":"41","author":"Sittig","year":"2008","journal-title":"J Biomed Inform"},{"issue":"7494","key":"2020110612453857400_ocz088-B21","doi-asserted-by":"crossref","first-page":"765.","DOI":"10.1136\/bmj.38398.500764.8F","article-title":"Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success","volume":"330","author":"Kawamoto","year":"2005","journal-title":"BMJ"},{"key":"2020110612453857400_ocz088-B22","doi-asserted-by":"crossref","first-page":"i6.","DOI":"10.1136\/bmj.i6","article-title":"Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests","volume":"352","author":"Vickers","year":"2016","journal-title":"BMJ"},{"issue":"6","key":"2020110612453857400_ocz088-B23","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision curve analysis: a novel method for evaluating prediction models","volume":"26","author":"Vickers","year":"2006","journal-title":"Med Decis Making"},{"key":"2020110612453857400_ocz088-B24","doi-asserted-by":"crossref","first-page":"g7594.","DOI":"10.1136\/bmj.g7594","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement","volume":"350","author":"Collins","year":"2015","journal-title":"BMJ"},{"key":"2020110612453857400_ocz088-B25","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.jbi.2016.09.013","article-title":"Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU","volume":"64","author":"Kipnis","year":"2016","journal-title":"J Biomed Inform"},{"issue":"26","key":"2020110612453857400_ocz088-B26","doi-asserted-by":"crossref","first-page":"1728","DOI":"10.1056\/NEJM198806303182605","article-title":"An assessment of clinically useful measures of the consequences of treatment","volume":"318","author":"Laupacis","year":"1988","journal-title":"N Engl J Med"},{"issue":"21","key":"2020110612453857400_ocz088-B27","doi-asserted-by":"crossref","first-page":"2813","DOI":"10.1001\/jama.287.21.2813","article-title":"Reporting number needed to treat and absolute risk reduction in randomized controlled trials","volume":"287","author":"Nuovo","year":"2002","journal-title":"JAMA"},{"issue":"9170","key":"2020110612453857400_ocz088-B28","doi-asserted-by":"crossref","first-page":"2105","DOI":"10.1016\/S0140-6736(98)11415-0","article-title":"Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists' Collaborative Group","volume":"353","author":"Rothwell","year":"1999","journal-title":"Lancet"},{"issue":"12","key":"2020110612453857400_ocz088-B29","doi-asserted-by":"crossref","first-page":"1467","DOI":"10.1002\/(SICI)1097-0258(19990630)18:12<1467::AID-SIM149>3.0.CO;2-H","article-title":"Selection bias and treatment heterogeneity in clinical trials","volume":"18","author":"Longford","year":"1999","journal-title":"Stat Med"},{"issue":"10","key":"2020110612453857400_ocz088-B30","doi-asserted-by":"crossref","first-page":"1209\u201312.","DOI":"10.1001\/jama.298.10.1209","article-title":"Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification","volume":"298","author":"Kent","year":"2007","journal-title":"JAMA"},{"issue":"14","key":"2020110612453857400_ocz088-B31","doi-asserted-by":"crossref","first-page":"1377","DOI":"10.1001\/jama.2017.12126","article-title":"Discrimination and calibration of clinical prediction models: users' guides to the medical literature","volume":"318","author":"Alba","year":"2017","journal-title":"JAMA"},{"key":"2020110612453857400_ocz088-B32","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/j.jclinepi.2015.12.005","article-title":"A calibration hierarchy for risk models was defined: from utopia to empirical data","volume":"74","author":"Van Calster","year":"2016","journal-title":"J Clin Epidemiol"},{"issue":"3","key":"2020110612453857400_ocz088-B33","doi-asserted-by":"crossref","first-page":"e009957.","DOI":"10.1136\/bmjopen-2015-009957","article-title":"Impact analysis studies of clinical prediction rules relevant to primary care: a systematic review","volume":"6","author":"Wallace","year":"2016","journal-title":"BMJ Open"},{"issue":"1","key":"2020110612453857400_ocz088-B34","doi-asserted-by":"crossref","first-page":"e000234.","DOI":"10.1136\/bmjresp-2017-000234","article-title":"Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial","volume":"4","author":"Shimabukuro","year":"2017","journal-title":"BMJ Open Resp Res"},{"issue":"8","key":"2020110612453857400_ocz088-B35","doi-asserted-by":"crossref","first-page":"1595","DOI":"10.1097\/CCM.0000000000001020","article-title":"An electronic tool for the evaluation and treatment of sepsis in the ICU: a randomized controlled trial","volume":"43","author":"Semler","year":"2015","journal-title":"Crit Care Med"},{"key":"2020110612453857400_ocz088-B36","article-title":"Electronic health record-based clinical decision support alert for severe sepsis: a randomised evaluation","author":"Downing","journal-title":"BMJ Qual Saf"},{"issue":"4","key":"2020110612453857400_ocz088-B37","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1017\/ice.2018.16","article-title":"A generalizable, data-driven approach to predict daily risk of clostridium difficile infection at two large academic health centers","volume":"39","author":"Oh","year":"2018","journal-title":"Infect Control Hosp Epidemiol"},{"issue":"8","key":"2020110612453857400_ocz088-B38","doi-asserted-by":"crossref","first-page":"e185097.","DOI":"10.1001\/jamanetworkopen.2018.5097","article-title":"Validation of prediction models for critical care outcomes using natural language processing of electronic health record data","volume":"1","author":"Marafino","year":"2018","journal-title":"JAMA Netw Open"},{"key":"2020110612453857400_ocz088-B39","doi-asserted-by":"crossref","first-page":"18.","DOI":"10.1038\/s41746-018-0029-1","article-title":"Scalable and accurate deep learning with electronic health records","volume":"1","author":"Rajkomar","year":"2018","journal-title":"Digit Med"},{"issue":"11","key":"2020110612453857400_ocz088-B40","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1161\/CIRCULATIONAHA.115.018791","article-title":"Prevalence of Familial Hypercholesterolemia in the 1999 to 2012 United States National Health and Nutrition Examination Surveys (NHANES)","volume":"133","author":"de Ferranti","year":"2016","journal-title":"Circulation"},{"issue":"4","key":"2020110612453857400_ocz088-B41","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1001\/jama.2017.8543","article-title":"Cascade screening for familial hypercholesterolemia and the use of genetic testing","volume":"318","author":"Knowles","year":"2017","journal-title":"JAMA"},{"issue":"1","key":"2020110612453857400_ocz088-B42","doi-asserted-by":"crossref","first-page":"23.","DOI":"10.1038\/s41746-019-0101-5","article-title":"Finding missed cases of familial hypercholesterolemia in health systems using machine learning","volume":"2","author":"Banda","year":"2019","journal-title":"NPJ Digit Med"},{"issue":"1","key":"2020110612453857400_ocz088-B43","first-page":"104","article-title":"Indications of PCSK9 inhibitors for patients at high and very high cardiovascular risk","volume":"111","author":"Behr","year":"2018","journal-title":"Arq Bras Cardiol"},{"issue":"18","key":"2020110612453857400_ocz088-B44","doi-asserted-by":"crossref","first-page":"2389","DOI":"10.1001\/jama.1990.03450180053028","article-title":"Admission source to the medical intensive care unit predicts hospital death independent of APACHE II score","volume":"264","author":"Escarce","year":"1990","journal-title":"JAMA"},{"issue":"2","key":"2020110612453857400_ocz088-B45","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1002\/jhm.817","article-title":"Intra-hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS)","volume":"6","author":"Escobar","year":"2011","journal-title":"J Hosp Med"},{"issue":"4","key":"2020110612453857400_ocz088-B46","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1177\/0272989X8800800409","article-title":"Focusing technology assessment using medical decision theory","volume":"8","author":"Phelps","year":"1988","journal-title":"Med Decis Making"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/12\/1655\/34151894\/ocz088.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/12\/1655\/34151894\/ocz088.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T18:30:47Z","timestamp":1604687447000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/26\/12\/1655\/5516459"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,13]]},"references-count":46,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2019,6,13]]},"published-print":{"date-parts":[[2019,12,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz088","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,12]]},"published":{"date-parts":[[2019,6,13]]}}}