{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T23:12:45Z","timestamp":1781651565322,"version":"3.54.5"},"reference-count":49,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T00:00:00Z","timestamp":1679875200000},"content-version":"vor","delay-in-days":2,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Digital Health Cooperative Research Centre"},{"name":"Commonwealth\u2019s Cooperative Research Centres"},{"DOI":"10.13039\/501100000925","name":"NHMRC","doi-asserted-by":"publisher","award":["#1181138"],"award-info":[{"award-number":["#1181138"]}],"id":[{"id":"10.13039\/501100000925","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000925","name":"NHMRC","doi-asserted-by":"publisher","award":["#2008313"],"award-info":[{"award-number":["#2008313"]}],"id":[{"id":"10.13039\/501100000925","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,5,19]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>Clinical prediction models providing binary categorizations for clinical decision support require the selection of a probability threshold, or \u201ccutpoint,\u201d to classify individuals. Existing cutpoint selection approaches typically optimize test-specific metrics, including sensitivity and specificity, but overlook the consequences of correct or incorrect classification. We introduce a new cutpoint selection approach considering downstream consequences using net monetary benefit (NMB) and through simulations compared it with alternative approaches in 2 use-cases: (i) preventing intensive care unit readmission and (ii) preventing inpatient falls.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and methods<\/jats:title><jats:p>Parameter estimates for costs and effectiveness from prior studies were included in Monte Carlo simulations. For each use-case, we simulated the expected NMB resulting from the model-guided decision using a range of cutpoint selection approaches, including our new value-optimizing approach. Sensitivity analyses applied alternative event rates, model discrimination, and calibration performance.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The proposed approach that considered expected downstream consequences was frequently NMB-maximizing compared with other methods. Sensitivity analysis demonstrated that it was or closely tracked the optimal strategy under a range of scenarios. Under scenarios of relatively low event rates and discrimination that may be considered realistic for intensive care (prevalence\u2009=\u20090.025, area under the receiver operating characteristic curve [AUC]\u2009=\u20090.70) and falls (prevalence\u2009=\u20090.036, AUC\u2009=\u20090.70), our proposed cutpoint method was either the best or similar to the best of the compared methods regarding NMB, and was robust to model miscalibration.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>Our results highlight the potential value of conditioning cutpoints on the implementation setting, particularly for rare and costly events, which are often the target of prediction model development research.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>This study proposes a cutpoint selection method that may optimize clinical decision support systems toward value-based care.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocad042","type":"journal-article","created":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T09:38:11Z","timestamp":1679909891000},"page":"1103-1113","source":"Crossref","is-referenced-by-count":10,"title":["Integrating economic considerations into cutpoint selection may help align clinical decision support toward value-based healthcare"],"prefix":"10.1093","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6053-8174","authenticated-orcid":false,"given":"Rex","family":"Parsons","sequence":"first","affiliation":[{"name":"Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology , Kelvin Grove, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3643-4332","authenticated-orcid":false,"given":"Robin","family":"Blythe","sequence":"additional","affiliation":[{"name":"Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology , Kelvin Grove, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Susanna M","family":"Cramb","sequence":"additional","affiliation":[{"name":"Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology , Kelvin Grove, Australia"},{"name":"Jamieson Trauma Institute, Royal Brisbane and Women\u2019s Hospital, Metro North Health , Herston, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steven M","family":"McPhail","sequence":"additional","affiliation":[{"name":"Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology , Kelvin Grove, Australia"},{"name":"Digital Health and Informatics, Metro South Health , Woolloongabba, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,3,25]]},"reference":[{"issue":"3","key":"2023051917301972200_ocad042-B1","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1001\/jama.2020.1230","article-title":"Validation and utility testing of clinical prediction models: Time to change the approach","volume":"324","author":"Adibi","year":"2020","journal-title":"JAMA"},{"issue":"2","key":"2023051917301972200_ocad042-B2","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1016\/S0003-4975(01)02870-3","article-title":"Receiver operating characteristic curve analysis of clinical risk models","volume":"72","author":"Grunkemeier","year":"2001","journal-title":"Ann Thorac Surg"},{"issue":"4","key":"2023051917301972200_ocad042-B3","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1002\/bimj.200410135","article-title":"Estimation of the Youden index and its associated cutoff point","volume":"47","author":"Fluss","year":"2005","journal-title":"Biom J"},{"key":"2023051917301972200_ocad042-B4","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1186\/s12874-015-0009-y","article-title":"Optimal cut-point definition in biomarkers: The case of censored failure time outcome","volume":"15","author":"Rota","year":"2015","journal-title":"BMC Med Res Methodol"},{"issue":"23","key":"2023051917301972200_ocad042-B5","doi-asserted-by":"crossref","first-page":"2676","DOI":"10.1002\/sim.4509","article-title":"Classification accuracy and cut point selection","volume":"31","author":"Liu","year":"2012","journal-title":"Stat Med"},{"key":"2023051917301972200_ocad042-B6","doi-asserted-by":"crossref","first-page":"3762651","DOI":"10.1155\/2017\/3762651","article-title":"Defining an optimal cut-point value in ROC analysis: An alternative approach","volume":"2017","author":"Unal","year":"2017","journal-title":"Comput Math Methods Med"},{"issue":"6","key":"2023051917301972200_ocad042-B7","doi-asserted-by":"crossref","first-page":"670","DOI":"10.1016\/S1076-6332(03)80087-9","article-title":"Sensitivity and specificity should be de-emphasized in diagnostic accuracy studies","volume":"10","author":"Moons","year":"2003","journal-title":"Acad Radiol"},{"issue":"1","key":"2023051917301972200_ocad042-B8","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1097\/00001648-199701000-00002","article-title":"Limitations of sensitivity, specificity, likelihood ratio, and Bayes\u2019 theorem in assessing diagnostic probabilities: A clinical example","volume":"8","author":"Moons","year":"1997","journal-title":"Epidemiology"},{"issue":"1","key":"2023051917301972200_ocad042-B9","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1186\/s13054-015-0999-1","article-title":"Why the C-statistic is not informative to evaluate early warning scores and what metrics to use","volume":"19","author":"Romero-Brufau","year":"2015","journal-title":"Crit Care"},{"key":"2023051917301972200_ocad042-B10","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1186\/1742-5573-3-11","article-title":"New patient-oriented summary measure of net total gain in certainty for dichotomous diagnostic tests","volume":"3","author":"Linn","year":"2006","journal-title":"Epidemiol Perspect Innov"},{"key":"2023051917301972200_ocad042-B11","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1186\/s41512-019-0064-7","article-title":"A simple, step-by-step guide to interpreting decision curve analysis","volume":"3","author":"Vickers","year":"2019","journal-title":"Diagn Progn Res"},{"issue":"4","key":"2023051917301972200_ocad042-B12","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1001\/jama.2015.37","article-title":"Decision curve analysis","volume":"313","author":"Fitzgerald","year":"2015","journal-title":"JAMA"},{"issue":"1","key":"2023051917301972200_ocad042-B13","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1053\/j.seminoncol.2009.12.004","article-title":"Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: Towards a decision analytic framework","volume":"37","author":"Vickers","year":"2010","journal-title":"Semin Oncol"},{"issue":"6","key":"2023051917301972200_ocad042-B14","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision curve analysis: A novel method for evaluating prediction models","volume":"26","author":"Vickers","year":"2006","journal-title":"Med Decis Making"},{"issue":"6","key":"2023051917301972200_ocad042-B15","doi-asserted-by":"crossref","first-page":"e436","DOI":"10.1016\/S2589-7500(22)00042-5","article-title":"Cost-effectiveness of artificial intelligence for screening colonoscopy: A modelling study","volume":"4","author":"Areia","year":"2022","journal-title":"Lancet Digit Health"},{"issue":"4","key":"2023051917301972200_ocad042-B16","doi-asserted-by":"crossref","first-page":"718","DOI":"10.1016\/j.jval.2017.01.001","article-title":"Systematic review of health economic impact evaluations of risk prediction models: Stop developing, start evaluating","volume":"20","author":"van Giessen","year":"2017","journal-title":"Value Health"},{"key":"2023051917301972200_ocad042-B17","first-page":"14","article-title":"Inpatient fall prediction models: A scoping review","author":"Parsons","year":"2022","journal-title":"Gerontology"},{"key":"2023051917301972200_ocad042-B18","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/1472-6947-12-19","article-title":"Mining geriatric assessment data for in-patient fall prediction models and high-risk subgroups","volume":"12","author":"Marschollek","year":"2012","journal-title":"BMC Med Inform Decis Making"},{"issue":"1","key":"2023051917301972200_ocad042-B19","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1158\/1940-6207.CAPR-10-0262","article-title":"Health benefits and cost-effectiveness of primary genetic screening for lynch syndrome in the general population health benefits and cost-effectiveness of primary genetic screening","volume":"4","author":"Dinh","year":"2011","journal-title":"Cancer Prev Res"},{"issue":"11","key":"2023051917301972200_ocad042-B20","doi-asserted-by":"crossref","first-page":"2579","DOI":"10.1007\/s00198-011-1869-6","article-title":"Cost-effective intervention thresholds against osteoporotic fractures based on FRAX\u00ae in Switzerland","volume":"23","author":"Lippuner","year":"2012","journal-title":"Osteoporos Int"},{"issue":"4","key":"2023051917301972200_ocad042-B21","doi-asserted-by":"crossref","first-page":"1491","DOI":"10.1007\/s00198-012-2115-6","article-title":"Intervention thresholds for Denosumab in the UK using a FRAX\u00ae-based cost-effectiveness analysis","volume":"24","author":"Str\u00f6m","year":"2013","journal-title":"Osteoporos Int"},{"issue":"1","key":"2023051917301972200_ocad042-B22","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1186\/s12916-019-1425-3","article-title":"Three myths about risk thresholds for prediction models","volume":"17","author":"Wynants","year":"2019","journal-title":"BMC Med"},{"issue":"6","key":"2023051917301972200_ocad042-B23","doi-asserted-by":"crossref","first-page":"1132","DOI":"10.1111\/jth.13687","article-title":"A decision model to estimate a risk threshold for venous thromboembolism prophylaxis in hospitalized medical patients","volume":"15","author":"Le","year":"2017","journal-title":"J Thromb Haemost"},{"issue":"1","key":"2023051917301972200_ocad042-B24","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1186\/1471-2288-14-40","article-title":"External validation of multivariable prediction models: A systematic review of methodological conduct and reporting","volume":"14","author":"Collins","year":"2014","journal-title":"BMC Med Res Methodol"},{"key":"2023051917301972200_ocad042-B25","doi-asserted-by":"crossref","first-page":"S68","DOI":"10.1177\/0272989X98018002S09","article-title":"Net health benefits: A new framework for the analysis of uncertainty in cost-effectiveness analysis","volume":"18","author":"Stinnett","year":"1998","journal-title":"Med Decis Making"},{"issue":"4","key":"2023051917301972200_ocad042-B26","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/j.aucc.2019.05.005","article-title":"Risk prediction models for intensive care unit readmission: A systematic review of methodology and applicability","volume":"33","author":"Markazi-Moghaddam","year":"2020","journal-title":"Aust Crit Care"},{"key":"2023051917301972200_ocad042-B27","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1186\/1741-7015-11-135","article-title":"Cost effectiveness of patient education for the prevention of falls in hospital: Economic evaluation from a randomized controlled trial","volume":"11","author":"Haines","year":"2013","journal-title":"BMC Med"},{"issue":"9","key":"2023051917301972200_ocad042-B28","doi-asserted-by":"crossref","first-page":"367","DOI":"10.5694\/mja15.00296","article-title":"The extra resource burden of in-hospital falls: A cost of falls study","volume":"203","author":"Morello","year":"2015","journal-title":"Med J Aust"},{"issue":"11","key":"2023051917301972200_ocad042-B29","doi-asserted-by":"crossref","first-page":"e48758","DOI":"10.1371\/journal.pone.0048758","article-title":"Readmissions and death after ICU discharge: Development and validation of two predictive models","volume":"7","author":"Badawi","year":"2012","journal-title":"PLoS ONE"},{"key":"2023051917301972200_ocad042-B30","volume-title":"Methods for the Economic Evaluation of Health Care Programmes","author":"Drummond","year":"2015"},{"issue":"2","key":"2023051917301972200_ocad042-B31","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/s40273-017-0585-2","article-title":"Estimating the reference incremental cost-effectiveness ratio for the Australian Health System","volume":"36","author":"Edney","year":"2018","journal-title":"Pharmacoeconomics"},{"key":"2023051917301972200_ocad042-B32","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780195108248.001.0001","volume-title":"Cost-Effectiveness in Health and Medicine","author":"Gold","year":"1996"},{"issue":"4","key":"2023051917301972200_ocad042-B33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v064.i04","article-title":"fitdistrplus: An R package for fitting distributions","volume":"64","author":"Delignette-Muller","year":"2015","journal-title":"J Stat Softw"},{"issue":"3","key":"2023051917301972200_ocad042-B34","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1016\/j.jhealeco.2004.09.011","article-title":"Generalized modeling approaches to risk adjustment of skewed outcomes data","volume":"24","author":"Manning","year":"2005","journal-title":"J Health Econ"},{"issue":"11","key":"2023051917301972200_ocad042-B35","doi-asserted-by":"crossref","first-page":"1834","DOI":"10.1097\/00003246-199811000-00025","article-title":"Patients readmitted to the intensive care unit during the same hospitalization: Clinical features and outcomes","volume":"26","author":"Chen","year":"1998","journal-title":"Crit Care Med"},{"issue":"1","key":"2023051917301972200_ocad042-B36","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1186\/s12913-017-2079-5","article-title":"What is a hospital bed day worth? A contingent valuation study of hospital Chief Executive Officers","volume":"17","author":"Page","year":"2017","journal-title":"BMC Health Serv Res"},{"issue":"3","key":"2023051917301972200_ocad042-B37","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1016\/j.jval.2021.06.018","article-title":"The potential cost-effectiveness of a machine learning tool that can prevent untimely intensive care unit discharge","volume":"25","author":"de Vos","year":"2022","journal-title":"Value Health"},{"issue":"7","key":"2023051917301972200_ocad042-B38","doi-asserted-by":"crossref","first-page":"324","DOI":"10.5694\/mja2.50309","article-title":"The financial cost of intensive care in Australia: A multicentre registry study","volume":"211","author":"Hicks","year":"2019","journal-title":"Med J Aust"},{"issue":"9987","key":"2023051917301972200_ocad042-B39","doi-asserted-by":"crossref","first-page":"2592","DOI":"10.1016\/S0140-6736(14)61945-0","article-title":"Fall rates in hospital rehabilitation units after individualised patient and staff education programmes: A pragmatic, stepped-wedge, cluster-randomised controlled trial","volume":"385","author":"Hill","year":"2015","journal-title":"Lancet"},{"issue":"9","key":"2023051917301972200_ocad042-B40","doi-asserted-by":"crossref","first-page":"367","DOI":"10.5694\/mja15.00296","article-title":"The extra resource burden of in-hospital falls: A cost of falls study","volume":"203","author":"Morello","year":"2015","journal-title":"Med J Aust"},{"issue":"6","key":"2023051917301972200_ocad042-B41","doi-asserted-by":"crossref","first-page":"516","DOI":"10.1001\/archinternmed.2010.444","article-title":"Patient education to prevent falls among older hospital inpatients: A randomized controlled trial","volume":"171","author":"Haines","year":"2011","journal-title":"Arch Intern Med"},{"issue":"5","key":"2023051917301972200_ocad042-B42","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1093\/ageing\/aft076","article-title":"Cost\u2013utility analysis of a shock-absorbing floor intervention to prevent injuries from falls in hospital wards for older people","volume":"42","author":"Latimer","year":"2013","journal-title":"Age Ageing"},{"issue":"1","key":"2023051917301972200_ocad042-B43","doi-asserted-by":"crossref","first-page":"35","DOI":"10.5093\/ejpalc2018a5","article-title":"Transforming the area under the normal curve (AUC) into Cohen\u2019s d, Pearson\u2019s rpb, odds-ratio, and natural log odds-ratio: Two conversion tables","volume":"10","author":"Salgado","year":"2018","journal-title":"Eur J Psychol Appl Legal Context"},{"key":"2023051917301972200_ocad042-B44","author":"Ensor","year":"2022"},{"issue":"11","key":"2023051917301972200_ocad042-B45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v098.i11","article-title":"cutpointr: Improved estimation and validation of optimal cutpoints in R","volume":"98","author":"Thiele","year":"2021","journal-title":"J Stat Softw"},{"issue":"5","key":"2023051917301972200_ocad042-B46","doi-asserted-by":"crossref","first-page":"479","DOI":"10.2165\/00019053-200017050-00006","article-title":"Handling uncertainty in cost-effectiveness models","volume":"17","author":"Briggs","year":"2000","journal-title":"Pharmacoeconomics"},{"key":"2023051917301972200_ocad042-B47","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1111\/jth.12262","article-title":"Diagnostic and prognostic prediction models","volume":"11 (Suppl 1)","author":"Hendriksen","year":"2013","journal-title":"J Thromb Haemost"},{"key":"2023051917301972200_ocad042-B48","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1186\/s41512-018-0033-6","article-title":"Evaluating the impact of prediction models: Lessons learned, challenges, and recommendations","volume":"2","author":"Kappen","year":"2018","journal-title":"Diagn Progn Res"},{"issue":"3","key":"2023051917301972200_ocad042-B49","doi-asserted-by":"crossref","first-page":"138","DOI":"10.12788\/jhm.3066","article-title":"Beyond reporting early warning score sensitivity: The temporal relationship and clinical relevance of \u201ctrue positive\u201d alerts that precede critical deterioration","volume":"14","author":"Winter","year":"2019","journal-title":"J Hosp Med"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/6\/1103\/50374837\/ocad042.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/6\/1103\/50374837\/ocad042.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,8]],"date-time":"2023-12-08T21:29:25Z","timestamp":1702070965000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/30\/6\/1103\/7086046"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,25]]},"references-count":49,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,3,25]]},"published-print":{"date-parts":[[2023,5,19]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocad042","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,6,1]]},"published":{"date-parts":[[2023,3,25]]}}}