{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,4]],"date-time":"2026-09-04T15:14:55Z","timestamp":1788534895968,"version":"build-2803163510"},"reference-count":50,"publisher":"SAGE Publications","issue":"8","license":[{"start":{"date-parts":[[2018,10,14]],"date-time":"2018-10-14T00:00:00Z","timestamp":1539475200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/100000050","name":"National Heart, Lung, and Blood Institute","doi-asserted-by":"publisher","award":["R01-HL072966"],"award-info":[{"award-number":["R01-HL072966"]}],"id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["UL1TR000423"],"award-info":[{"award-number":["UL1TR000423"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100001797","name":"Pharmaceutical Research and Manufacturers of America Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100001797","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Med Decis Making"],"published-print":{"date-parts":[[2018,11]]},"abstract":"<jats:p>Many medical decisions involve the use of dynamic information collected on individual patients toward predicting likely transitions in their future health status. If accurate predictions are developed, then a prognostic model can identify patients at greatest risk for future adverse events and may be used clinically to define populations appropriate for targeted intervention. In practice, a prognostic model is often used to guide decisions at multiple time points over the course of disease, and classification performance (i.e., sensitivity and specificity) for distinguishing high-risk v. low-risk individuals may vary over time as an individual\u2019s disease status and prognostic information change. In this tutorial, we detail contemporary statistical methods that can characterize the time-varying accuracy of prognostic survival models when used for dynamic decision making. Although statistical methods for evaluating prognostic models with simple binary outcomes are well established, methods appropriate for survival outcomes are less well known and require time-dependent extensions of sensitivity and specificity to fully characterize longitudinal biomarkers or models. The methods we review are particularly important in that they allow for appropriate handling of censored outcomes commonly encountered with event time data. We highlight the importance of determining whether clinical interest is in predicting cumulative (or prevalent) cases over a fixed future time interval v. predicting incident cases over a range of follow-up times and whether patient information is static or updated over time. We discuss implementation of time-dependent receiver operating characteristic approaches using relevant R statistical software packages. The statistical summaries are illustrated using a liver prognostic model to guide transplantation in primary biliary cirrhosis.<\/jats:p>","DOI":"10.1177\/0272989x18801312","type":"journal-article","created":{"date-parts":[[2018,10,15]],"date-time":"2018-10-15T02:27:06Z","timestamp":1539570426000},"page":"904-916","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":64,"title":["A Tutorial on Evaluating the Time-Varying Discrimination Accuracy of Survival Models Used in Dynamic Decision Making"],"prefix":"10.1177","volume":"38","author":[{"given":"Aasthaa","family":"Bansal","sequence":"first","affiliation":[{"name":"The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA (AB)"},{"name":"Department of Biostatistics, University of Washington, Seattle, WA (PJH)"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Patrick J.","family":"Heagerty","sequence":"additional","affiliation":[{"name":"The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, School of Pharmacy, University of Washington, Seattle, WA (AB)"},{"name":"Department of Biostatistics, University of Washington, Seattle, WA (PJH)"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2018,10,14]]},"reference":[{"key":"bibr1-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/hep.1840100102"},{"key":"bibr2-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/hep.1840200120"},{"key":"bibr3-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.3121\/cmr.3.2.87"},{"key":"bibr4-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1111\/j.1600-6143.2006.01276.x"},{"key":"bibr5-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.publhealth.18.1.83"},{"key":"bibr6-0272989X18801312","volume-title":"Statistical Analysis with Missing Data","author":"Little RJA","year":"1982"},{"key":"bibr7-0272989X18801312","unstructured":"Mayo Clinic. Primary biliary cholangitis. Available from: https:\/\/www.mayoclinic.org\/diseases-conditions\/primary-biliary-cirrhosis\/basics\/definition\/con-20029377"},{"key":"bibr8-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1016\/S1665-2681(19)30838-5"},{"key":"bibr9-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1016\/0016-5085(85)90213-6"},{"key":"bibr10-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1056\/NEJM198301063080101"},{"key":"bibr11-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1016\/0140-6736(90)90734-M"},{"key":"bibr12-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.3109\/00365529009107932"},{"key":"bibr13-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1016\/0016-5085(93)91086-W"},{"key":"bibr14-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/hep.510300415"},{"key":"bibr15-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1056\/NEJM198504183121602"},{"key":"bibr16-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4"},{"key":"bibr17-0272989X18801312","volume-title":"Proceedings of the 14th International Joint Conference on Artificial Intelligence","author":"Kohavi R.","year":"1995"},{"key":"bibr18-0272989X18801312","doi-asserted-by":"crossref","volume-title":"Biostatistics: A Methodology for the Health Sciences","author":"van Belle G","DOI":"10.1002\/0471602396"},{"key":"bibr19-0272989X18801312","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1111\/j.2517-6161.1972.tb00899.x","volume":"34","author":"Cox DR.","year":"1972","journal-title":"J R Stat Soc B Met"},{"key":"bibr20-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/9781118032985"},{"key":"bibr21-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1177\/0962280209105024"},{"key":"bibr22-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/sim.4780132307"},{"key":"bibr23-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-0258(19970228)16:4<385::AID-SIM380>3.0.CO;2-3"},{"key":"bibr24-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/bti324"},{"key":"bibr25-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1016\/S0933-3657(03)00033-2"},{"key":"bibr26-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/sim.1593"},{"key":"bibr27-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1214\/08-AOAS169"},{"key":"bibr28-0272989X18801312","volume-title":"Evaluation of Diagnostic Systems: Methods from Signal Detection Theory","author":"Swets JA","year":"1982"},{"key":"bibr29-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1016\/S0001-2998(78)80014-2"},{"key":"bibr30-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780198509844.001.0001"},{"key":"bibr31-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1148\/radiology.143.1.7063747"},{"key":"bibr32-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1111\/j.0006-341X.2000.00337.x"},{"key":"bibr33-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1111\/j.1541-0420.2006.00726.x"},{"key":"bibr34-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1111\/j.0006-341X.2005.030814.x"},{"key":"bibr35-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1093\/biostatistics\/kxs021"},{"key":"bibr36-0272989X18801312","unstructured":"Bansal A, Heagerty PJ, Saha-Chaudhuri P. Dynamic Placement Values: A Basis for Evaluating Prognostic Potential. Unpublished manuscript."},{"key":"bibr37-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1093\/jnci\/djj329"},{"key":"bibr38-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1111\/j.1541-0420.2009.01375.x"},{"key":"bibr39-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.109.192278"},{"key":"bibr40-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1097\/EDE.0b013e3181c30fb2"},{"key":"bibr41-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.105.584102"},{"key":"bibr42-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7"},{"key":"bibr43-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.1970.10488634"},{"key":"bibr44-0272989X18801312","doi-asserted-by":"crossref","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","volume":"58","author":"Tibshirani R.","year":"1996","journal-title":"J R Stat Soc B"},{"key":"bibr45-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/bti724"},{"key":"bibr46-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0084483"},{"key":"bibr47-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/sim.2929"},{"key":"bibr48-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1002\/sim.4085"},{"key":"bibr49-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1111\/biom.12628."},{"key":"bibr50-0272989X18801312","doi-asserted-by":"publisher","DOI":"10.1177\/0962280212451881"}],"container-title":["Medical Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0272989X18801312","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0272989X18801312","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0272989X18801312","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T17:59:29Z","timestamp":1777658369000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0272989X18801312"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,10,14]]},"references-count":50,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2018,11]]}},"alternative-id":["10.1177\/0272989X18801312"],"URL":"https:\/\/doi.org\/10.1177\/0272989x18801312","relation":{},"ISSN":["0272-989X","1552-681X"],"issn-type":[{"value":"0272-989X","type":"print"},{"value":"1552-681X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,10,14]]}}}