{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,25]],"date-time":"2026-10-25T01:29:11Z","timestamp":1792891751872,"version":"4.5.0"},"reference-count":55,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T00:00:00Z","timestamp":1679443200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T00:00:00Z","timestamp":1679443200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Data"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine\u00a0data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.<\/jats:p>","DOI":"10.1038\/s41597-023-02062-w","type":"journal-article","created":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T09:06:22Z","timestamp":1679475982000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites"],"prefix":"10.1038","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0503-0175","authenticated-orcid":false,"given":"Kareem A.","family":"Wahid","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Diana","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Onur","family":"Sahin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5880-2802","authenticated-orcid":false,"given":"Michael","family":"Cislo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin E.","family":"Nelms","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Renjie","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohammed A.","family":"Naser","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simon","family":"Duke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael V.","family":"Sherer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John P.","family":"Christodouleas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2064-7613","authenticated-orcid":false,"given":"Abdallah S. R.","family":"Mohamed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James D.","family":"Murphy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5264-3994","authenticated-orcid":false,"given":"Clifton D.","family":"Fuller","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erin F.","family":"Gillespie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,3,22]]},"reference":[{"key":"2062_CR1","doi-asserted-by":"publisher","first-page":"050902","DOI":"10.1118\/1.4871620","volume":"41","author":"G Sharp","year":"2014","unstructured":"Sharp, G. et al. Vision 20\/20: perspectives on automated image segmentation for radiotherapy. Med. Phys. 41, 050902 (2014).","journal-title":"Med. Phys."},{"key":"2062_CR2","doi-asserted-by":"crossref","unstructured":"Segedin, B. & Petric, P. Uncertainties in target volume delineation in radiotherapy \u2013 are they relevant and what can we do about them? Radiology and Oncology 50, 254\u2013262 (2016).","DOI":"10.1515\/raon-2016-0023"},{"key":"2062_CR3","doi-asserted-by":"publisher","first-page":"136","DOI":"10.4103\/0971-6203.44472","volume":"33","author":"CF Njeh","year":"2008","unstructured":"Njeh, C. F. Tumor delineation: The weakest link in the search for accuracy in radiotherapy. J. Med. Phys. 33, 136\u2013140 (2008).","journal-title":"J. Med. Phys."},{"key":"2062_CR4","doi-asserted-by":"crossref","unstructured":"Berry, S. L., Boczkowski, A., Ma, R., Mechalakos, J. & Hunt, M. Interobserver variability in radiation therapy plan output: Results of a single-institution study. Practical Radiation Oncology 6, 442\u2013449 (2016).","DOI":"10.1016\/j.prro.2016.04.005"},{"key":"2062_CR5","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1016\/j.radonc.2021.05.003","volume":"160","author":"MV Sherer","year":"2021","unstructured":"Sherer, M. V. et al. Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review. Radiother. Oncol. 160, 185\u2013191 (2021).","journal-title":"Radiother. Oncol."},{"key":"2062_CR6","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1016\/j.semradonc.2019.02.001","volume":"29","author":"CE Cardenas","year":"2019","unstructured":"Cardenas, C. E., Yang, J., Anderson, B. M., Court, L. E. & Brock, K. B. Advances in Auto-Segmentation. Semin. Radiat. Oncol. 29, 185\u2013197 (2019).","journal-title":"Semin. Radiat. Oncol."},{"key":"2062_CR7","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1016\/j.clon.2021.12.003","volume":"34","author":"K Harrison","year":"2022","unstructured":"Harrison, K. et al. Machine Learning for Auto-Segmentation in Radiotherapy Planning. Clin. Oncol. 34, 74\u201388 (2022).","journal-title":"Clin. Oncol."},{"key":"2062_CR8","doi-asserted-by":"publisher","first-page":"175007","DOI":"10.1088\/1361-6560\/ab99e5","volume":"65","author":"S Yu","year":"2020","unstructured":"Yu, S. et al. Robustness study of noisy annotation in deep learning based medical image segmentation. Phys. Med. Biol. 65, 175007 (2020).","journal-title":"Phys. Med. Biol."},{"key":"2062_CR9","doi-asserted-by":"crossref","unstructured":"Budd, S. et al. Can Non-specialists Provide High Quality Gold Standard Labels in Challenging Modalities? in Domain Adaptation and Representation Transfer, and Affordable Healthcare and AI for Resource Diverse Global Health 251\u2013262 (Springer International Publishing, 2021).","DOI":"10.1007\/978-3-030-87722-4_23"},{"key":"2062_CR10","first-page":"034002","volume":"5","author":"E Heim","year":"2018","unstructured":"Heim, E. et al. Large-scale medical image annotation with crowd-powered algorithms. J Med Imaging (Bellingham) 5, 034002 (2018).","journal-title":"J Med Imaging (Bellingham)"},{"key":"2062_CR11","doi-asserted-by":"crossref","unstructured":"Wesemeyer, T., Jauer, M.-L. & Deserno, T. M. Annotation quality vs. quantity for deep-learned medical image segmentation. in Medical Imaging 2021: Imaging Informatics for Healthcare, Research, and Applications vol. 11601 63\u201376 (SPIE, 2021).","DOI":"10.1117\/12.2582226"},{"key":"2062_CR12","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-021-00900-3","volume":"8","author":"P Afshar","year":"2021","unstructured":"Afshar, P. et al. COVID-CT-MD, COVID-19 computed tomography scan dataset applicable in machine learning and deep learning. Sci Data 8, 121 (2021).","journal-title":"Sci Data"},{"key":"2062_CR13","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2019.3","volume":"6","author":"J Wallner","year":"2019","unstructured":"Wallner, J. & Mischak, I., Jan Egger. Computed tomography data collection of the complete human mandible and valid clinical ground truth models. Sci Data 6, 190003 (2019).","journal-title":"Sci Data"},{"key":"2062_CR14","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-022-01507-y","volume":"9","author":"X Lyu","year":"2022","unstructured":"Lyu, X., Cheng, L. & Zhang, S. The RETA Benchmark for Retinal Vascular Tree Analysis. Sci Data 9, 397 (2022).","journal-title":"Sci Data"},{"key":"2062_CR15","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2015.59","volume":"2","author":"J Kulaga-Yoskovitz","year":"2015","unstructured":"Kulaga-Yoskovitz, J. et al. Multi-contrast submillimetric 3 Tesla hippocampal subfield segmentation protocol and dataset. Sci Data 2, 150059 (2015).","journal-title":"Sci Data"},{"key":"2062_CR16","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-021-00946-3","volume":"8","author":"K Payette","year":"2021","unstructured":"Payette, K. et al. An automatic multi-tissue human fetal brain segmentation benchmark using the Fetal Tissue Annotation Dataset. Sci Data 8, 167 (2021).","journal-title":"Sci Data"},{"key":"2062_CR17","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-022-01564-3","volume":"9","author":"K Jin","year":"2022","unstructured":"Jin, K. et al. FIVES: A Fundus Image Dataset for Artificial Intelligence based Vessel Segmentation. Sci Data 9, 475 (2022).","journal-title":"Sci Data"},{"key":"2062_CR18","doi-asserted-by":"publisher","first-page":"e10","DOI":"10.1016\/j.ijrobp.2021.05.154","volume":"111","author":"D Lin","year":"2021","unstructured":"Lin, D. et al. Contouring Collaborative for Consensus in Radiation Oncology (C3RO): An International Crowdsourcing Challenge to Improve Radiotherapy Contour Delineation. Int. J. Radiat. Oncol. Biol. Phys. 111, e10\u2013e11 (2021).","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"2062_CR19","first-page":"S11903","volume":"10","author":"D Lin","year":"2023","unstructured":"Lin, D. et al. E pluribus unum: prospective acceptability benchmarking from the Contouring Collaborative for Consensus in Radiation Oncology crowdsourced initiative for multiobserver segmentation. J Med Imaging (Bellingham) 10, S11903 (2023).","journal-title":"J Med Imaging (Bellingham)"},{"key":"2062_CR20","doi-asserted-by":"publisher","first-page":"635","DOI":"10.1007\/s10278-012-9462-x","volume":"25","author":"KA Archie","year":"2012","unstructured":"Archie, K. A. & Marcus, D. S. DicomBrowser: software for viewing and modifying DICOM metadata. J. Digit. Imaging 25, 635\u2013645 (2012).","journal-title":"J. Digit. Imaging"},{"key":"2062_CR21","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1038\/s41393-018-0169-9","volume":"56","author":"LA Harvey","year":"2018","unstructured":"Harvey, L. A. REDCap: web-based software for all types of data storage and collection. Spinal Cord 56, 625 (2018).","journal-title":"Spinal Cord"},{"key":"2062_CR22","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.1016\/j.ijrobp.2017.12.013","volume":"100","author":"CS Mayo","year":"2018","unstructured":"Mayo, C. S. et al. American Association of Physicists in Medicine Task Group 263: Standardizing Nomenclatures in Radiation Oncology. Int. J. Radiat. Oncol. Biol. Phys. 100, 1057\u20131066 (2018).","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"2062_CR23","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-30695-9","volume":"13","author":"M Antonelli","year":"2022","unstructured":"Antonelli, M. et al. The Medical Segmentation Decathlon. Nat. Commun. 13, 4128 (2022).","journal-title":"Nat. Commun."},{"key":"2062_CR24","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-26328-2","volume":"12","author":"L Zbinden","year":"2022","unstructured":"Zbinden, L. et al. Convolutional neural network for automated segmentation of the liver and its vessels on non-contrast T1 vibe Dixon acquisitions. Sci. Rep. 12, 22059 (2022).","journal-title":"Sci. Rep."},{"key":"2062_CR25","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-022-01555-4","volume":"9","author":"R Chitalia","year":"2022","unstructured":"Chitalia, R. et al. Expert tumor annotations and radiomics for locally advanced breast cancer in DCE-MRI for ACRIN 6657\/I-SPY1. Sci Data 9, 440 (2022).","journal-title":"Sci Data"},{"key":"2062_CR26","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-022-01587-w","volume":"9","author":"KA Wahid","year":"2022","unstructured":"Wahid, K. A. et al. Muscle and adipose tissue segmentations at the third cervical vertebral level in patients with head and neck cancer. Sci Data 9, 470 (2022).","journal-title":"Sci Data"},{"key":"2062_CR27","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-90599-4","volume":"11","author":"T-W Chiu","year":"2021","unstructured":"Chiu, T.-W., Tsai, Y.-L. & Su, S.-F. Automatic detect lung node with deep learning in segmentation and imbalance data labeling. Sci. Rep. 11, 11174 (2021).","journal-title":"Sci. Rep."},{"key":"2062_CR28","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/j.jneumeth.2016.03.001","volume":"264","author":"X Li","year":"2016","unstructured":"Li, X., Morgan, P. S., Ashburner, J., Smith, J. & Rorden, C. The first step for neuroimaging data analysis: DICOM to NIfTI conversion. J. Neurosci. Methods 264, 47\u201356 (2016).","journal-title":"J. Neurosci. Methods"},{"key":"2062_CR29","unstructured":"Van Rossum, G. & Drake, F. L. Python 3 Reference Manual. (CreateSpace, 2009)."},{"key":"2062_CR30","doi-asserted-by":"publisher","first-page":"226","DOI":"10.1016\/j.prro.2021.02.003","volume":"11","author":"BM Anderson","year":"2021","unstructured":"Anderson, B. M., Wahid, K. A. & Brock, K. K. Simple Python Module for Conversions Between DICOM Images and Radiation Therapy Structures, Masks, and Prediction Arrays. Pract. Radiat. Oncol. 11, 226\u2013229 (2021).","journal-title":"Pract. Radiat. Oncol."},{"key":"2062_CR31","doi-asserted-by":"publisher","first-page":"290","DOI":"10.1007\/s10278-017-0037-8","volume":"31","author":"Z Yaniv","year":"2018","unstructured":"Yaniv, Z., Lowekamp, B. C., Johnson, H. J. & Beare, R. SimpleITK Image-Analysis Notebooks: a Collaborative Environment for Education and Reproducible Research. J. Digit. Imaging 31, 290\u2013303 (2018).","journal-title":"J. Digit. Imaging"},{"key":"2062_CR32","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/j.ctro.2022.06.007","volume":"36","author":"N Taku","year":"2022","unstructured":"Taku, N. et al. Auto-detection and segmentation of involved lymph nodes in HPV-associated oropharyngeal cancer using a convolutional deep learning neural network. Clinical and Translational Radiation Oncology 36, 47\u201355 (2022).","journal-title":"Clinical and Translational Radiation Oncology"},{"key":"2062_CR33","doi-asserted-by":"crossref","unstructured":"Naser, M. A. et al. Head and Neck Cancer Primary Tumor Auto Segmentation Using Model Ensembling of Deep Learning in PET\/CT Images. in Head and Neck Tumor Segmentation and Outcome Prediction 121\u2013133 (Springer International Publishing, 2022).","DOI":"10.1007\/978-3-030-98253-9_11"},{"key":"2062_CR34","doi-asserted-by":"publisher","unstructured":"McDonald, B. A. et al. Investigation of autosegmentation techniques on T2-weighted MRI for off-line dose reconstruction in MR-linac Adapt to Position workflow for head and neck cancers. bioRxiv, https:\/\/doi.org\/10.1101\/2021.09.30.21264327 (2021).","DOI":"10.1101\/2021.09.30.21264327"},{"key":"2062_CR35","doi-asserted-by":"publisher","first-page":"426","DOI":"10.1016\/j.ijrobp.2022.01.050","volume":"113","author":"CE Cardenas","year":"2022","unstructured":"Cardenas, C. E. et al. Comprehensive Quantitative Evaluation of Variability in Magnetic Resonance-Guided Delineation of Oropharyngeal Gross Tumor Volumes and High-Risk Clinical Target Volumes: An R-IDEAL Stage 0 Prospective Study. Int. J. Radiat. Oncol. Biol. Phys. 113, 426\u2013436 (2022).","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"2062_CR36","doi-asserted-by":"publisher","first-page":"903","DOI":"10.1109\/TMI.2004.828354","volume":"23","author":"SK Warfield","year":"2004","unstructured":"Warfield, S. K., Zou, K. H. & Wells, W. M. Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans. Med. Imaging 23, 903\u2013921 (2004).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"2062_CR37","doi-asserted-by":"publisher","DOI":"10.6084\/m9.figshare.21074182.v3","author":"Contouring Collaborative for Consensus in Radiation Oncology","year":"2022","unstructured":"Contouring Collaborative for Consensus in Radiation Oncology. Large-scale crowdsourced radiotherapy segmentations across a variety of cancer sites. Figshare https:\/\/doi.org\/10.6084\/m9.figshare.21074182.v3 (2022)."},{"key":"2062_CR38","doi-asserted-by":"publisher","first-page":"e26151","DOI":"10.2196\/26151","volume":"23","author":"S Nikolov","year":"2021","unstructured":"Nikolov, S. et al. Clinically Applicable Segmentation of Head and Neck Anatomy for Radiotherapy: Deep Learning Algorithm Development and Validation Study. J. Med. Internet Res. 23, e26151 (2021).","journal-title":"J. Med. Internet Res."},{"key":"2062_CR39","unstructured":"Livne, M., Hughes, C., Hawkins, P., Deason, L. & Dudovitch, G. surface-distance: Library to compute surface distance based performance metrics for segmentation tasks. GitHub https:\/\/github.com\/deepmind\/surface-distance (2018)."},{"key":"2062_CR40","doi-asserted-by":"publisher","first-page":"996","DOI":"10.3332\/ecancer.2020.996","volume":"14","author":"S Arculeo","year":"2020","unstructured":"Arculeo, S. et al. The emerging role of radiation therapists in the contouring of organs at risk in radiotherapy: analysis of inter-observer variability with radiation oncologists for the chest and upper abdomen. Ecancermedicalscience 14, 996 (2020).","journal-title":"Ecancermedicalscience"},{"key":"2062_CR41","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1590\/0100-3984.2020.0028","volume":"54","author":"FCF Dionisio","year":"2021","unstructured":"Dionisio, F. C. F. et al. Manual versus semiautomatic segmentation of soft-tissue sarcomas on magnetic resonance imaging: evaluation of similarity and comparison of segmentation times. Radiol Bras 54, 155\u2013164 (2021).","journal-title":"Radiol Bras"},{"key":"2062_CR42","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.radonc.2019.04.006","volume":"137","author":"J van der Veen","year":"2019","unstructured":"van der Veen, J., Gulyban, A. & Nuyts, S. Interobserver variability in delineation of target volumes in head and neck cancer. Radiother. Oncol. 137, 9\u201315 (2019).","journal-title":"Radiother. Oncol."},{"key":"2062_CR43","doi-asserted-by":"publisher","DOI":"10.1186\/s13014-020-01677-2","volume":"16","author":"J van der Veen","year":"2021","unstructured":"van der Veen, J., Gulyban, A., Willems, S., Maes, F. & Nuyts, S. Interobserver variability in organ at risk delineation in head and neck cancer. Radiat. Oncol. 16, 120 (2021).","journal-title":"Radiat. Oncol."},{"key":"2062_CR44","doi-asserted-by":"publisher","first-page":"4479","DOI":"10.1088\/1361-6560\/aa685b","volume":"62","author":"B Aklan","year":"2017","unstructured":"Aklan, B. et al. Regional deep hyperthermia: impact of observer variability in CT-based manual tissue segmentation on simulated temperature distribution. Phys. Med. Biol. 62, 4479\u20134495 (2017).","journal-title":"Phys. Med. Biol."},{"key":"2062_CR45","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.tipsro.2022.08.004","volume":"23","author":"MJA Rasing","year":"2022","unstructured":"Rasing, M. J. A. et al. Online adaptive MR-guided radiotherapy: Conformity of contour adaptation for prostate cancer, rectal cancer and lymph node oligometastases among radiation therapists and radiation oncologists. Tech Innov Patient Support Radiat Oncol 23, 33\u201340 (2022).","journal-title":"Tech Innov Patient Support Radiat Oncol"},{"key":"2062_CR46","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.ctro.2018.06.002","volume":"11","author":"P Franco","year":"2018","unstructured":"Franco, P. et al. Variability of clinical target volume delineation for rectal cancer patients planned for neoadjuvant radiotherapy with the aid of the platform Anatom-e. Clin Transl Radiat Oncol 11, 33\u201339 (2018).","journal-title":"Clin Transl Radiat Oncol"},{"key":"2062_CR47","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1200\/CCI.19.00041","volume":"3","author":"MV Sherer","year":"2019","unstructured":"Sherer, M. V. et al. Development and Usage of eContour, a Novel, Three-Dimensional, Image-Based Web Site to Facilitate Access to Contouring Guidelines at the Point of Care. JCO Clin Cancer Inform 3, 1\u20139 (2019).","journal-title":"JCO Clin Cancer Inform"},{"key":"2062_CR48","doi-asserted-by":"publisher","DOI":"10.1101\/2022.09.23.22280295","author":"D Lin","year":"2022","unstructured":"Lin, D. et al. \u201cE pluribus unum\u201d: Prospective acceptability benchmarking from the contouring collaborative for consensus in Radiation Oncology (C3RO) crowdsourced initiative for multi-observer segmentation. medRxiv https:\/\/doi.org\/10.1101\/2022.09.23.22280295 (2022).","journal-title":"medRxiv"},{"key":"2062_CR49","doi-asserted-by":"publisher","DOI":"10.1186\/1748-717X-7-32","volume":"7","author":"CL Brouwer","year":"2012","unstructured":"Brouwer, C. L. et al. 3D Variation in delineation of head and neck organs at risk. Radiat. Oncol. 7, 32 (2012).","journal-title":"Radiat. Oncol."},{"key":"2062_CR50","doi-asserted-by":"crossref","unstructured":"O\u2019Neil, A. Q., Murchison, J. T., van Beek, E. J. R. & Goatman, K. A. Crowdsourcing Labels for Pathological Patterns in CT Lung Scans: Can Non-experts Contribute Expert-Quality Ground Truth? in Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis 96\u2013105 (Springer International Publishing, 2017).","DOI":"10.1007\/978-3-319-67534-3_11"},{"key":"2062_CR51","unstructured":"Abramoff, Magalh\u00e3es & Ram. Image processing with ImageJ. Biophotonics int."},{"key":"2062_CR52","doi-asserted-by":"publisher","first-page":"3245","DOI":"10.1118\/1.3468652","volume":"37","author":"A Panchal","year":"2010","unstructured":"Panchal, A. & Keyes, R. SU-GG-T-260: dicompyler: an open source radiation therapy research platform with a plugin architecture. Med. Phys. 37, 3245\u20133245 (2010).","journal-title":"Med. Phys."},{"key":"2062_CR53","first-page":"3342","volume":"2016","author":"PA Yushkevich","year":"2016","unstructured":"Yushkevich, P. A., Yang, G. & Gerig, G. ITK-SNAP: An interactive tool for semi-automatic segmentation of multi-modality biomedical images. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2016, 3342\u20133345 (2016).","journal-title":"Conf. Proc. IEEE Eng. Med. Biol. Soc."},{"key":"2062_CR54","doi-asserted-by":"crossref","unstructured":"Pieper, S., Halle, M. & Kikinis, R. 3D Slicer. In 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821) 632\u2013635 Vol. 1 (2004).","DOI":"10.1109\/ISBI.2004.1398617"},{"key":"2062_CR55","doi-asserted-by":"publisher","first-page":"6332","DOI":"10.1118\/1.4754659","volume":"39","author":"C Pinter","year":"2012","unstructured":"Pinter, C., Lasso, A., Wang, A., Jaffray, D. & Fichtinger, G. SlicerRT: radiation therapy research toolkit for 3D Slicer. Med. Phys. 39, 6332\u20136338 (2012).","journal-title":"Med. Phys."}],"container-title":["Scientific Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41597-023-02062-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41597-023-02062-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41597-023-02062-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,16]],"date-time":"2024-10-16T17:09:54Z","timestamp":1729098594000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41597-023-02062-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,22]]},"references-count":55,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2062"],"URL":"https:\/\/doi.org\/10.1038\/s41597-023-02062-w","relation":{"references":[{"id-type":"doi","id":"10.6084\/m9.figshare.21074182.v3","asserted-by":"subject"}],"has-preprint":[{"id-type":"doi","id":"10.1101\/2022.10.05.22280672","asserted-by":"object"}]},"ISSN":["2052-4463"],"issn-type":[{"value":"2052-4463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,22]]},"assertion":[{"value":"14 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 March 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"D.L. is in a research fellowship funded by grants for research and education related to eContour.org. B.E.N. is the founder of ProKnow and advisor to Elekta AB. J.P.C. is an employee of Elekta AB. C.D.F. has received direct industry grant support, speaking honoraria, and travel funding from Elekta AB. E.F.G. is a co-founder of the educational website eContour.org. The other authors have no conflicts of interest to disclose.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"161"}}