{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T13:20:26Z","timestamp":1784640026510,"version":"3.55.0"},"reference-count":38,"publisher":"Wiley","issue":"9","license":[{"start":{"date-parts":[[2025,8,24]],"date-time":"2025-08-24T00:00:00Z","timestamp":1755993600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,8,24]],"date-time":"2025-08-24T00:00:00Z","timestamp":1755993600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["aapm.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J Applied Clin Med Phys"],"published-print":{"date-parts":[[2025,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Deep learning (DL)\u2010based organ segmentation is increasingly used in radiotherapy. While methods exist to generate voxel\u2010wise uncertainty maps from DL\u2010based auto\u2010segmentation models, these maps are rarely presented to clinicians.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>This study aimed to evaluate the impact of DL\u2010generated uncertainty maps on experienced radiation oncologists during the manual correction of DL\u2010based auto\u2010segmentation for prostate radiotherapy.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      Two nnUNet DL models were trained with 10\u2010fold cross\u2010validation on a dataset of 434 patient cases undergoing ultra\u2010hypofractionated MRI\u2010only radiotherapy for prostate cancer. The models performed prostate clinical target volume (CTV) and rectum segmentation. Each cross\u2010validation model was evaluated on an independent test set of 35 patient cases. Segmentation uncertainty was calculated voxel\u2010wise as the SoftMax standard deviation (0\u20130.5,\n                      <jats:italic>n<\/jats:italic>\n                      \u00a0=\u00a010) and visualized as a fixed scale color\u2010coded map.\n                    <\/jats:p>\n                    <jats:p>Four experienced oncologists were asked to:<\/jats:p>\n                    <jats:p>Step 1: Rate the quality of and confidence in the DL segmentations using a four\u2010 and five\u2010point Likert scale, respectively, and edit the segmentations without access to the uncertainty map.<\/jats:p>\n                    <jats:p>Step 2: Repeat step 1 after at least 4 weeks, but this time with the color\u2010coded uncertainty map available.<\/jats:p>\n                    <jats:p>Oncologists were asked to blend the uncertainty map with the DL segmentation and MRI volume.<\/jats:p>\n                    <jats:p>Segmentation edit time was recorded for both steps. In step 2, oncologists also provided free\u2010text feedback on the benefits and drawbacks of using the uncertainty map during segmentation. A histogram analysis was performed to compare the number of voxels edited between step 1 and step 2 for different uncertainty levels (bins with 0.1 intervals).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The DL models achieved high\u2010quality segmentations with a mean Dice coefficient per oncologist of 0.97\u20130.99, calculated between edited and unedited segmentation in step 1 for the prostate CTV and rectum.<\/jats:p>\n                    <jats:p>While the overall quality rating for rectum segmentations decreased slightly on a group level in step 2 compared to step 1, individual responses varied. Some oncologists rated the quality higher for the prostate CTV segmentation with the uncertainty map present, while others rated it lower. Similarly, confidence ratings varied across oncologists for prostate CTV and rectum.<\/jats:p>\n                    <jats:p>Decreased segmentation time was recorded for three oncologists using uncertainty maps, saving 1\u20132 min per patient case, corresponding to 14%\u201333% time reduction. Three oncologists found the uncertainty maps helpful, and one reported benefit was the ability to identify regions of interest more quickly. The histogram analysis had fewer voxel edits in regions of low uncertainty in step 2 compared to step 1. Specifically, 50% fewer voxel edits were recorded for the uncertainty region 0.0\u20130.1, suggesting increased trust in the DL model's prediction in these areas.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Presenting DL uncertainty information to experienced radiation oncologists influences their decision\u2010making, quality perception, and confidence in the DL segmentations. Regions with low uncertainty were less likely to be edited, indicating increased reliance on the model's predictions. Additionally, uncertainty maps can improve efficiency by reducing segmentation time. DL\u2010based segmentation uncertainty can be a valuable tool in clinical practice, enhancing the efficiency of radiotherapy planning.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/acm2.70221","type":"journal-article","created":{"date-parts":[[2025,8,24]],"date-time":"2025-08-24T14:41:02Z","timestamp":1756046462000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Impact of deep learning model uncertainty on manual corrections to MRI\u2010based auto\u2010segmentation in prostate cancer radiotherapy"],"prefix":"10.1002","volume":"26","author":[{"given":"Viktor","family":"Rogowski","sequence":"first","affiliation":[{"name":"Radiation Physics, Department of Hematology, Oncology, and Radiation Physics Sk\u00e5ne University Hospital  Lund Sweden"},{"name":"Department of Medical Radiation Physics Clinical Sciences Lund University  Lund Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8367-1317","authenticated-orcid":false,"given":"Angelica","family":"Svalkvist","sequence":"additional","affiliation":[{"name":"Department of Medical Radiation Sciences, Institute of Clinical Sciences, Sahlgrenska Academy University of Gothenburg  Gothenburg Sweden"},{"name":"Department of Medical Physics and Biomedical Engineering Region V\u00e4stra G\u00f6taland, Sahlgrenska University Hospital  Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0347-3375","authenticated-orcid":false,"given":"Matteo","family":"Maspero","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology Imaging and Cancer Division University Medical Center Utrecht  Utrecht The Netherlands"},{"name":"Computational Imaging Group for MR Diagnostics &amp; Therapy, Center for Image Sciences University Medical Center Utrecht  Utrecht The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2807-8670","authenticated-orcid":false,"given":"Tomas","family":"Janssen","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology The Netherlands Cancer Institute Antoni van Leeuwenhoek  Amsterdam The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Federica Carmen","family":"Maruccio","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology The Netherlands Cancer Institute Antoni van Leeuwenhoek  Amsterdam The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jenny","family":"Gorgisyan","sequence":"additional","affiliation":[{"name":"Radiation Physics, Department of Hematology, Oncology, and Radiation Physics Sk\u00e5ne University Hospital  Lund Sweden"},{"name":"Department of Medical Radiation Physics Clinical Sciences Lund University  Lund Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonas","family":"Scherman","sequence":"additional","affiliation":[{"name":"Radiation Physics, Department of Hematology, Oncology, and Radiation Physics Sk\u00e5ne University Hospital  Lund Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ida","family":"H\u00e4ggstr\u00f6m","sequence":"additional","affiliation":[{"name":"Department of Medical Radiation Sciences, Institute of Clinical Sciences, Sahlgrenska Academy University of Gothenburg  Gothenburg Sweden"},{"name":"Department of Electrical Engineering Chalmers University of Technology  Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Victor","family":"W\u00e5hlstrand","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering Chalmers University of Technology  Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adalsteinn","family":"Gunnlaugsson","sequence":"additional","affiliation":[{"name":"Department of Hematology Oncology, and Radiation Physics, Sk\u00e5ne University Hospital  Lund Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin P","family":"Nilsson","sequence":"additional","affiliation":[{"name":"Department of Hematology Oncology, and Radiation Physics, Sk\u00e5ne University Hospital  Lund Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mathieu","family":"Moreau","sequence":"additional","affiliation":[{"name":"Department of Hematology Oncology, and Radiation Physics, Sk\u00e5ne University Hospital  Lund Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"N\u00e1ndor","family":"Vass","sequence":"additional","affiliation":[{"name":"Department of Hematology Oncology, and Radiation Physics, Sk\u00e5ne University Hospital  Lund Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Niclas","family":"Pettersson","sequence":"additional","affiliation":[{"name":"Department of Medical Radiation Sciences, Institute of Clinical Sciences, Sahlgrenska Academy University of Gothenburg  Gothenburg Sweden"},{"name":"Department of Medical Physics and Biomedical Engineering Region V\u00e4stra G\u00f6taland, Sahlgrenska University Hospital  Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2931-5615","authenticated-orcid":false,"given":"Christian Jamtheim","family":"Gustafsson","sequence":"additional","affiliation":[{"name":"Radiation Physics, Department of Hematology, Oncology, and Radiation Physics Sk\u00e5ne University Hospital  Lund Sweden"},{"name":"Department of Translational Medicine Medical Radiation Physics, Lund University  Malm\u00f6 Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2025,8,24]]},"reference":[{"key":"e_1_2_11_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11684\u2010020\u20100761\u20101"},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12032\u2010020\u201001374\u2010w"},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41571\u2010020\u20100417\u20108"},{"key":"e_1_2_11_5_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.12748"},{"key":"e_1_2_11_6_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13014\u2010020\u201001677\u20102"},{"key":"e_1_2_11_7_1","doi-asserted-by":"publisher","DOI":"10.1080\/0284186X.2021.1975813"},{"key":"e_1_2_11_8_1","doi-asserted-by":"publisher","DOI":"10.1093\/bjr\/tqad018"},{"key":"e_1_2_11_9_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13014\u2010022\u201002088\u20101"},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.phro.2019.12.001"},{"key":"e_1_2_11_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.radonc.2019.10.019"},{"key":"e_1_2_11_12_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000085"},{"key":"e_1_2_11_13_1","doi-asserted-by":"publisher","DOI":"10.1088\/2057\u20101976\/ad6dcd"},{"key":"e_1_2_11_14_1","doi-asserted-by":"publisher","DOI":"10.1038\/s43856\u2010024\u201000528\u20105"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.adro.2021.100658"},{"key":"e_1_2_11_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102101"},{"key":"e_1_2_11_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.radonc.2023.109794"},{"key":"e_1_2_11_18_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361\u20106560\/ad1a26"},{"key":"e_1_2_11_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.radonc.2024.110545"},{"key":"e_1_2_11_20_1","doi-asserted-by":"publisher","DOI":"10.1101\/2024.05.13.24307226"},{"key":"e_1_2_11_21_1","unstructured":"PearceT BrintrupA ZhuJ Understanding Softmax Confidence and Uncertainty.Arxiv preprint arXiv:210604972.2021."},{"key":"e_1_2_11_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2024.108675"},{"key":"e_1_2_11_23_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41592\u2010020\u201001008\u2010z"},{"key":"e_1_2_11_24_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13014\u2010020\u201001513\u20107"},{"key":"e_1_2_11_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.phro.2024.100557"},{"key":"e_1_2_11_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.radonc.2018.01.014"},{"key":"e_1_2_11_27_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41597\u2010025\u201004954\u20105"},{"key":"e_1_2_11_28_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41592\u2010019\u20100686\u20102"},{"key":"e_1_2_11_29_1","doi-asserted-by":"publisher","DOI":"10.1093\/rpd\/ncq007"},{"key":"e_1_2_11_30_1","doi-asserted-by":"publisher","DOI":"10.1259\/bjr\/35012658"},{"key":"e_1_2_11_31_1","doi-asserted-by":"publisher","DOI":"10.1093\/rpd\/ncv542"},{"key":"e_1_2_11_32_1","doi-asserted-by":"publisher","DOI":"10.1093\/rpd\/ncw035"},{"key":"e_1_2_11_33_1","doi-asserted-by":"publisher","DOI":"10.1093\/rpd\/ncab066"},{"key":"e_1_2_11_34_1","unstructured":"CoxRW AshburnerJ BremanH et\u00a0al.A (sort of) new image data format standard: NiFTI\u20101.https:\/\/nifti.nimh.nih.gov\/nifti\u20101\/documentation\/hbm_nifti_2004.pdf"},{"key":"e_1_2_11_35_1","unstructured":"PhilT Sikerdebaard\/dcmrtstruct2nii: v1.0.19. Accessed June 22 2021. doi:10.5281\/zenodo.4037865"},{"key":"e_1_2_11_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10278\u2010017\u20100037\u20108"},{"key":"e_1_2_11_37_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41592\u2010023\u201002150\u20100"},{"key":"e_1_2_11_38_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1976.10481517"},{"key":"e_1_2_11_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.semradonc.2022.06.001"}],"container-title":["Journal of Applied Clinical Medical Physics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/acm2.70221","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/acm2.70221","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/acm2.70221","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,31]],"date-time":"2026-01-31T05:26:13Z","timestamp":1769837173000},"score":1,"resource":{"primary":{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/10.1002\/acm2.70221"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,24]]},"references-count":38,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2025,9]]}},"alternative-id":["10.1002\/acm2.70221"],"URL":"https:\/\/doi.org\/10.1002\/acm2.70221","archive":["Portico"],"relation":{},"ISSN":["1526-9914","1526-9914"],"issn-type":[{"value":"1526-9914","type":"print"},{"value":"1526-9914","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,24]]},"assertion":[{"value":"2025-02-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-14","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70221"}}