{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T13:54:24Z","timestamp":1753883664680,"version":"3.41.2"},"reference-count":21,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"8","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:sec><jats:title>PURPOSE<\/jats:title><jats:p> The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors. <\/jats:p><\/jats:sec><jats:sec><jats:title>METHODS<\/jats:title><jats:p> Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&amp;N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure. <\/jats:p><\/jats:sec><jats:sec><jats:title>RESULTS<\/jats:title><jats:p> Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&amp;N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&amp;N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations. <\/jats:p><\/jats:sec><jats:sec><jats:title>CONCLUSION<\/jats:title><jats:p> Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks. <\/jats:p><\/jats:sec>","DOI":"10.1200\/cci.23.00174","type":"journal-article","created":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T20:25:35Z","timestamp":1718310335000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":0,"title":["Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation"],"prefix":"10.1200","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0503-0175","authenticated-orcid":true,"given":"Kareem A.","family":"Wahid","sequence":"first","affiliation":[{"name":"Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX"},{"name":"Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2191-5659","authenticated-orcid":true,"given":"Onur","family":"Sahin","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Suprateek","family":"Kundu","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Diana","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9722-7580","authenticated-orcid":true,"given":"Anthony","family":"Alanis","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Salik","family":"Tehami","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0046-4337","authenticated-orcid":true,"given":"Serageldin","family":"Kamel","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Simon","family":"Duke","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Cambridge University Hospitals, Cambridge, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4439-7906","authenticated-orcid":true,"given":"Michael V.","family":"Sherer","sequence":"additional","affiliation":[{"name":"Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7853-3531","authenticated-orcid":true,"given":"Mathis","family":"Rasmussen","sequence":"additional","affiliation":[{"name":"Department of Oncology, Aarhus University Hospital, Aarhus, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3523-382X","authenticated-orcid":true,"given":"Stine","family":"Korreman","sequence":"additional","affiliation":[{"name":"Department of Oncology, Aarhus University Hospital, Aarhus, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Fuentes","sequence":"additional","affiliation":[{"name":"Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Cislo","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin E.","family":"Nelms","sequence":"additional","affiliation":[{"name":"Canis Lupus, LLC, Merrimac, WI"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John P.","family":"Christodouleas","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, The University of Pennsylvania Cancer Center, Philadelphia, PA"},{"name":"Elekta, Atlanta, GA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0523-9091","authenticated-orcid":true,"given":"James D.","family":"Murphy","sequence":"additional","affiliation":[{"name":"Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2064-7613","authenticated-orcid":true,"given":"Abdallah S.R.","family":"Mohamed","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Renjie","family":"He","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1020-4966","authenticated-orcid":true,"given":"Mohammed A.","family":"Naser","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1386-1542","authenticated-orcid":true,"given":"Erin F.","family":"Gillespie","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, University of Washington Fred Hutchinson Cancer Center, Seattle, WA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5264-3994","authenticated-orcid":true,"given":"Clifton D.","family":"Fuller","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"233","reference":[{"key":"b1","doi-asserted-by":"publisher","DOI":"10.1016\/j.radonc.2021.05.003"},{"key":"b2","doi-asserted-by":"publisher","DOI":"10.1016\/j.clon.2021.12.003"},{"key":"b3","doi-asserted-by":"publisher","DOI":"10.1117\/1.JMI.10.S1.S11903"},{"key":"b4","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2023.02.048"},{"key":"b5","doi-asserted-by":"publisher","DOI":"10.1002\/cncr.30327"},{"key":"b6","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2015.63.9898"},{"key":"b7","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-023-02062-w"},{"key":"b8","doi-asserted-by":"publisher","DOI":"10.1038\/s41393-018-0169-9"},{"key":"b9","doi-asserted-by":"publisher","DOI":"10.1111\/1754-9485.13423"},{"key":"b10","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2018.10.020"},{"key":"b11","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2004.828354"},{"key":"b12","doi-asserted-by":"publisher","DOI":"10.1016\/j.prro.2021.02.003"},{"key":"b13","doi-asserted-by":"publisher","DOI":"10.2196\/26151"},{"key":"b15","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v035.i04"},{"key":"b16","doi-asserted-by":"publisher","DOI":"10.21105\/joss.01143"},{"key":"b17","doi-asserted-by":"publisher","DOI":"10.21105\/joss.01541"},{"key":"b21","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2022.01.050"},{"key":"b22","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/aa685b"},{"key":"b23","doi-asserted-by":"publisher","DOI":"10.1016\/j.prro.2016.04.005"},{"key":"b24","first-page":"96","volume-title":"Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis","author":"O\u2019Neil AQ","year":"2017"},{"key":"b25","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2020.04.011"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI.23.00174","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T20:25:39Z","timestamp":1718310339000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI.23.00174"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6]]},"references-count":21,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["10.1200\/CCI.23.00174"],"URL":"https:\/\/doi.org\/10.1200\/cci.23.00174","relation":{},"ISSN":["2473-4276"],"issn-type":[{"type":"electronic","value":"2473-4276"}],"subject":[],"published":{"date-parts":[[2024,6]]},"assertion":[{"value":"2023-09-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-08","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-03","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2300174"}}