{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:52:56Z","timestamp":1760237576493,"version":"build-2065373602"},"reference-count":22,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,5,26]],"date-time":"2020-05-26T00:00:00Z","timestamp":1590451200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The predictive receiver operating characteristic (PROC) curve is a diagrammatic format with application in the statistical evaluation of probabilistic disease forecasts. The PROC curve differs from the more well-known receiver operating characteristic (ROC) curve in that it provides a basis for evaluation using metrics defined conditionally on the outcome of the forecast rather than metrics defined conditionally on the actual disease status. Starting from the binormal ROC curve formulation, an overview of some previously published binormal PROC curves is presented in order to place the PROC curve in the context of other methods used in statistical evaluation of probabilistic disease forecasts based on the analysis of predictive values; in particular, the index of separation (PSEP) and the leaf plot. An information theoretic perspective on evaluation is also outlined. Five straightforward recommendations are made with a view to aiding understanding and interpretation of the sometimes-complex patterns generated by PROC curve analysis. The PROC curve and related analyses augment the perspective provided by traditional ROC curve analysis. Here, the binormal ROC model provides the exemplar for investigation of the PROC curve, but potential application extends to analysis based on other distributional models as well as to empirical analysis.<\/jats:p>","DOI":"10.3390\/e22060593","type":"journal-article","created":{"date-parts":[[2020,5,27]],"date-time":"2020-05-27T05:35:52Z","timestamp":1590557752000},"page":"593","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["On the Binormal Predictive Receiver Operating Characteristic Curve for the Joint Assessment of Positive and Negative Predictive Values"],"prefix":"10.3390","volume":"22","author":[{"given":"Gareth","family":"Hughes","sequence":"first","affiliation":[{"name":"SRUC, Scotland\u2019s Rural College, The King\u2019s Buildings, Edinburgh EH9 3JG, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,5,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2313","DOI":"10.1098\/rsta.2008.0043","article-title":"The predictive receiver operating characteristic curve for the joint assessment of the positive and negative predictive values","volume":"366","author":"Shiu","year":"2008","journal-title":"Phil. Trans. R. Soc. A"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"07TR01","DOI":"10.1088\/1361-6560\/aab4b1","article-title":"Receiver operating characteristic (ROC) curves: Review of methods with applications in diagnostic medicine","volume":"63","author":"Obuchowski","year":"2018","journal-title":"Phys. Med. Biol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2251","DOI":"10.1098\/rsta.2008.0046","article-title":"Preface","volume":"366","author":"Zou","year":"2008","journal-title":"Phil. Trans. R. Soc. A"},{"key":"ref_4","unstructured":"Sackett, D.L., Haynes, R.B., Guyatt, G.H., and Tugwell, P. (1991). Clinical Epidemiology: A Basic Science for Clinical Medicine, Little, Brown Book Group. [2nd ed.]."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Pepe, M.S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford University Press.","DOI":"10.1093\/oso\/9780198509844.001.0001"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1002\/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5","article-title":"What do we mean by validating a prognostic model?","volume":"19","author":"Altman","year":"2000","journal-title":"Stat. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1283","DOI":"10.1007\/s00467-019-04226-6","article-title":"Using urine nitrite sticks to test for urinary tract infection in children aged <2 years: A meta-analysis","volume":"34","author":"Coulthard","year":"2019","journal-title":"Pediatr. Nephrol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"205","DOI":"10.3399\/bjgp19X702113","article-title":"The leaf plot: A novel way of presenting the value of tests","volume":"69","author":"Coulthard","year":"2019","journal-title":"Br. J. Gen. Pract."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1016\/j.jmp.2011.06.003","article-title":"Symmetry of receiver operating characteristic curves and Kullback-Leibler divergences between the signal and noise populations","volume":"5","author":"Bhattacharya","year":"2011","journal-title":"J. Math. Psychol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1342","DOI":"10.3390\/e15041342","article-title":"Symmetry properties of bi-normal and bi-gamma receiver operating characteristic curves are described by Kullback-Leibler divergences","volume":"15","author":"Hughes","year":"2013","journal-title":"Entropy"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.spl.2015.04.003","article-title":"On shape properties of the receiver operating characteristic curve","volume":"103","author":"Bhattacharya","year":"2015","journal-title":"Stat. Probabil. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.acra.2010.09.002","article-title":"Using the mean-to-sigma ratio as a measure of the improperness of binormal ROC curves","volume":"18","author":"Hillis","year":"2011","journal-title":"Acad. Radiol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1200\/JCO.2004.04.148","article-title":"Prospective multicenter study of axillary nodal staging by positron emission tomography in breast cancer: A report of the staging breast cancer with PET study group","volume":"22","author":"Wahl","year":"2004","journal-title":"J. Clin. Oncol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1136","DOI":"10.1094\/PHYTO-01-17-0023-FI","article-title":"Evaluation of probabilistic disease forecasts","volume":"107","author":"Hughes","year":"2017","journal-title":"Phytopathology"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1093\/ije\/28.3.521","article-title":"Selecting diagnostic tests for ruling out or ruling in disease: The use of the Kullback-Leibler distance","volume":"28","author":"Lee","year":"1999","journal-title":"Int. J. Epidemiol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1841","DOI":"10.1177\/0962280218776989","article-title":"Quantifying performance of a diagnostic test as the expected information for discrimination: Relation to the C-statistic","volume":"28","author":"McKeigue","year":"2019","journal-title":"Stat. Methods Med. Res."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hughes, G., Reed, J., and McRoberts, N. (2020). Information graphs incorporating predictive values of disease forecasts. Entropy, 22.","DOI":"10.3390\/e22030361"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"IV","DOI":"10.1055\/s-0038-1627066","article-title":"Information graphs for epidemiological applications of the Kullback-Leibler divergence","volume":"53","author":"Hughes","year":"2014","journal-title":"Meth. Inform. Med."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1148\/radiol.2533091633","article-title":"Receiver operating characteristic analysis for the evaluation of diagnosis and prediction","volume":"253","author":"Gatsonis","year":"2009","journal-title":"Radiology"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1242","DOI":"10.1136\/gut.2009.176867","article-title":"Using CT colonography as a triage technique after a positive faecal occult blood test in colorectal cancer screening","volume":"58","author":"Liedenbaum","year":"2009","journal-title":"Gut"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1097\/TA.0000000000002098","article-title":"Optimal timing of delayed excretory phase computed tomography scan for diagnosis of urinary extravasation after high-grade renal trauma","volume":"86","author":"Keihani","year":"2019","journal-title":"J. Trauma Acute Care Surg."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Oehr, P., and Ecke, T. (2020). Establishment and characterisation of empirical biomarker predictive value ROC curves (PV-ROC) using results of the UBC\u00ae Rapid Test in Bladder Cancer. Entropy, submitted.","DOI":"10.3390\/e22070729"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/6\/593\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:32:32Z","timestamp":1760175152000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/6\/593"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,26]]},"references-count":22,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["e22060593"],"URL":"https:\/\/doi.org\/10.3390\/e22060593","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2020,5,26]]}}}