{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T22:29:09Z","timestamp":1787869749509,"version":"build-2784847793"},"posted":{"date-parts":[[2026,6,29]]},"group-title":"Radiology and Imaging","reference-count":22,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T00:00:00Z","timestamp":1782691200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the Tumor Measurement Initiative through the MD Anderson Strategic Initiative Development Program"},{"name":"the National Cancer Institute of the National Institutes of Health","award":["P30CA016672"],"award-info":[{"award-number":["P30CA016672"]}]},{"name":"the National Cancer Institute of the National Institutes of Health","award":["1R01CA221971"],"award-info":[{"award-number":["1R01CA221971"]}]},{"name":"the National Cancer Institute of the National Institutes of Health","award":["R01CA235564"],"award-info":[{"award-number":["R01CA235564"]}]},{"name":"the National Cancer Institute of the National Institutes of Health","award":["P01CA261669"],"award-info":[{"award-number":["P01CA261669"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,6,29]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:p>\n                  Deep learning models deployed in clinical imaging frequently encounter distribution shifts, yet most out-of-distribution (OOD) detection methods are evaluated only on controlled research datasets. As a result, it is unclear whether existing approaches can reliably identify segmentation failures that arise in real-world clinical practice. We evaluated six OOD detection methods on a deployed liver CT segmentation model (3D nnU-Net) using internal data from 400 patients and external data from 100 patients collected across nearly 70 sites in 7 countries. One method was Pairwise Surface DSC, a surface-based extension of Pairwise DSC, that we introduced. OOD performance was measured using sensitivity, AUROC, and balanced accuracy, with thresholds determined on an independent cohort of 400 patients using the Youden J statistic. Statistical significance was assessed using McNemar tests and stratified bootstraps (\n                  <jats:italic>\u03b1<\/jats:italic>\n                  = 0.05) with Benjamini\u2013Hochberg correction. Pairwise Surface DSC was the top-performing method, with perfect sensitivities (1.00), near-perfect AUROCs (0.97 internal; 1.00 external), and the highest balanced accuracies (0.94 internal; 0.88 external;\n                  <jats:italic>p<\/jats:italic>\n                  &lt; 0.001). These results show that automated failure detection for liver CT segmentation is clinically feasible and that Pairwise Surface DSC is a promising candidate for deployment. Our code is available at\n                  <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/mckellwoodland\/liver_ct_ood_translation\">https:\/\/github.com\/mckellwoodland\/liver_ct_ood_translation<\/jats:ext-link>\n                  .\n                <\/jats:p>","DOI":"10.64898\/2026.06.26.26356692","type":"posted-content","created":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T23:50:19Z","timestamp":1782777019000},"source":"Crossref","is-referenced-by-count":1,"title":["A Real-World Evaluation of Failure Detection for Liver CT Segmentation"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-2546-1524","authenticated-orcid":false,"given":"Jeddy","family":"Bennett","sequence":"first","affiliation":[{"name":"Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston TX 77030, USA"},{"name":"Department of Mathematics, Brigham Young University, Provo UT 84602, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0995-3695","authenticated-orcid":false,"given":"McKell","family":"Woodland","sequence":"additional","affiliation":[{"name":"Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-3015-3570","authenticated-orcid":false,"given":"Austin","family":"Castelo","sequence":"additional","affiliation":[{"name":"Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-9298-6644","authenticated-orcid":false,"given":"Mais","family":"Altaie","sequence":"additional","affiliation":[{"name":"Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2265-8758","authenticated-orcid":false,"given":"Ajith","family":"Anthony","sequence":"additional","affiliation":[{"name":"Department of Interventional Radiology, The University of Texas MD Anderson Cancer Center, Houston TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-9738-6431","authenticated-orcid":false,"given":"Noreen S.","family":"Siddiqi","sequence":"additional","affiliation":[{"name":"Department of Interventional Radiology, The University of Texas MD Anderson Cancer Center, Houston TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5853-5938","authenticated-orcid":false,"given":"James P.","family":"Long","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9364-5040","authenticated-orcid":false,"given":"Kristy K.","family":"Brock","sequence":"additional","affiliation":[{"name":"Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"key":"2026070103450850000_2026.06.26.26356692v1.1","doi-asserted-by":"publisher","DOI":"10.3322\/caac.21834"},{"key":"2026070103450850000_2026.06.26.26356692v1.2","doi-asserted-by":"crossref","unstructured":"Hazhirkarzar, B. , Khoshpouri, P. , Shaghaghi, M. , Ghasabeh, M.A. , Pawlik, T.M. , Kamel, I.R. : Current state of the art imaging approaches for colorectal liver metastasis. Hepatobiliary Surgery and Nutrition 9(1) (2019)","DOI":"10.21037\/hbsn.2019.05.11"},{"key":"2026070103450850000_2026.06.26.26356692v1.3","doi-asserted-by":"publisher","DOI":"10.1016\/S1091-255X(02)00370-0"},{"key":"2026070103450850000_2026.06.26.26356692v1.4","doi-asserted-by":"publisher","DOI":"10.1186\/s41747-023-00383-4"},{"key":"2026070103450850000_2026.06.26.26356692v1.5","doi-asserted-by":"publisher","DOI":"10.1007\/s13244-017-0558-1"},{"key":"2026070103450850000_2026.06.26.26356692v1.6","doi-asserted-by":"publisher","DOI":"10.1016\/S2468-1253(25)00024-X"},{"key":"2026070103450850000_2026.06.26.26356692v1.7","doi-asserted-by":"publisher","DOI":"10.59275\/j.melba.2024-g93a"},{"key":"2026070103450850000_2026.06.26.26356692v1.8","doi-asserted-by":"publisher","DOI":"10.1016\/j.adro.2020.04.023"},{"key":"2026070103450850000_2026.06.26.26356692v1.9","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-024-02117-4"},{"key":"2026070103450850000_2026.06.26.26356692v1.10","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2024.103392"},{"key":"2026070103450850000_2026.06.26.26356692v1.11","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics13040667"},{"key":"2026070103450850000_2026.06.26.26356692v1.12","doi-asserted-by":"publisher","DOI":"10.64898\/2026.02.17.26346486"},{"key":"2026070103450850000_2026.06.26.26356692v1.13","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-020-01008-z"},{"key":"2026070103450850000_2026.06.26.26356692v1.14","unstructured":"Hendrycks, D. , Gimpel, K. : A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: International Conference on Learning Representations (2017)"},{"key":"2026070103450850000_2026.06.26.26356692v1.15","first-page":"21464","volume-title":"Advances in Neural Information Processing Systems","volume":"33","year":"2020"},{"key":"2026070103450850000_2026.06.26.26356692v1.16","first-page":"1321","volume-title":"Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research","volume":"70","year":"2017"},{"key":"2026070103450850000_2026.06.26.26356692v1.17","unstructured":"Lakshminarayanan, B. , Pritzel, A. , Blundell, C. : Simple and scalable predictive uncertainty estimation using deep ensembles. In: Guyon, I. , Luxburg, U.V. , Bengio, S. , Wallach, H. , Fergus, R. , Vishwanathan, S. , Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017)"},{"key":"2026070103450850000_2026.06.26.26356692v1.18","doi-asserted-by":"publisher","DOI":"10.2196\/26151"},{"key":"2026070103450850000_2026.06.26.26356692v1.19","doi-asserted-by":"publisher","DOI":"10.1002\/bimj.200410135"},{"key":"2026070103450850000_2026.06.26.26356692v1.20","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2025.109045"},{"key":"2026070103450850000_2026.06.26.26356692v1.21","first-page":"465","volume-title":"Proceedings of the Third Conference on Medical Imaging with Deep Learning. Proceedings of Machine Learning Research","volume":"121","year":"2020"},{"key":"2026070103450850000_2026.06.26.26356692v1.22","unstructured":"Zhang, O. , Delbrouck, J.B. , Rubin, D.L. : Out of distribution detection for medical images. In: Sudre, C.H. , Licandro, R. , Baumgartner, C. , Melbourne, A. , Dalca, A. , Hutter, J. , Tanno, R. , Abaci Turk, E. , Van Leemput, K. , Torrents Barrena, J. , Wells, W.M. , Macgowan, C. (eds.) Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis. pp. 102\u2013111. Springer International Publishing, Cham (2021)"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.06.26.26356692","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T11:52:55Z","timestamp":1782906775000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.64898\/2026.06.26.26356692"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,29]]},"references-count":22,"URL":"https:\/\/doi.org\/10.64898\/2026.06.26.26356692","relation":{},"subject":[],"published":{"date-parts":[[2026,6,29]]},"subtype":"preprint"}}