{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T12:01:27Z","timestamp":1790251287935,"version":"4.1.0"},"reference-count":70,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,1,13]],"date-time":"2025-01-13T00:00:00Z","timestamp":1736726400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,13]],"date-time":"2025-01-13T00:00:00Z","timestamp":1736726400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100015539","name":"Australian Government","doi-asserted-by":"publisher","award":["1208"],"award-info":[{"award-number":["1208"]}],"id":[{"id":"10.13039\/100015539","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001801","name":"University of Western Australia","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001801","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Phys Eng Sci Med"],"published-print":{"date-parts":[[2025,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Artificial Intelligence (AI) based auto-segmentation has demonstrated numerous benefits to clinical radiotherapy workflows. However, the rapidly changing regulatory, research, and market environment presents challenges around selecting and evaluating the most suitable solution. To support the clinical adoption of AI auto-segmentation systems, Selection Criteria recommendations were developed to enable a holistic evaluation of vendors, considering not only raw performance but associated risks uniquely related to the clinical deployment of AI. In-house experience and key bodies of work on ethics, standards, and best practices for AI in Radiation Oncology were reviewed to inform selection criteria and evaluation strategies. A retrospective analysis using the criteria was performed across six vendors, including a quantitative assessment using five metrics (Dice, Hausdorff Distance, Average Surface Distance, Surface Dice, Added Path Length) across 20 head and neck, 20 thoracic, and 19 male pelvis patients for AI models as of March 2023. A total of 47 selection criteria were identified across seven categories. A retrospective analysis showed that overall no vendor performed exceedingly well, with systematically poor performance in Data Security &amp; Responsibility, Vendor Support Tools, and Transparency &amp; Ethics. In terms of raw performance, vendors varied widely from excellent to poor. As new regulations come into force and the scope of AI auto-segmentation systems adapt to clinical needs, continued interest in ensuring safe, fair, and transparent AI will persist. The selection and evaluation framework provided herein aims to promote user confidence by exploring the breadth of clinically relevant factors to support informed decision-making.<\/jats:p>","DOI":"10.1007\/s13246-024-01513-x","type":"journal-article","created":{"date-parts":[[2025,1,13]],"date-time":"2025-01-13T07:24:14Z","timestamp":1736753054000},"page":"301-316","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Guidance on selecting and evaluating AI auto-segmentation systems in clinical radiotherapy: insights from a six-vendor analysis"],"prefix":"10.1007","volume":"48","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3804-5111","authenticated-orcid":false,"given":"Branimir","family":"Rusanov","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin A.","family":"Ebert","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mahsheed","family":"Sabet","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pejman","family":"Rowshanfarzad","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nathaniel","family":"Barry","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jake","family":"Kendrick","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zaid","family":"Alkhatib","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Suki","family":"Gill","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joshua","family":"Dass","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas","family":"Bucknell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeremy","family":"Croker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Colin","family":"Tang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rohen","family":"White","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sean","family":"Bydder","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mandy","family":"Taylor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luke","family":"Slama","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Godfrey","family":"Mukwada","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,1,13]]},"reference":[{"issue":"2","key":"1513_CR1","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/j.radonc.2016.09.009","volume":"121","author":"SK Vinod","year":"2016","unstructured":"Vinod SK et al (2016) Uncertainties in volume delineation in radiation oncology: a systematic review and recommendations for future studies. Radiother Oncol 121(2):169\u2013179","journal-title":"Radiother Oncol"},{"issue":"6","key":"1513_CR2","doi-asserted-by":"crossref","first-page":"442","DOI":"10.1016\/j.prro.2016.04.005","volume":"6","author":"SL Berry","year":"2016","unstructured":"Berry SL et al (2016) Interobserver variability in radiation therapy plan output: results of a single-institution study. Pract Radiat Oncol 6(6):442\u2013449","journal-title":"Pract Radiat Oncol"},{"issue":"3","key":"1513_CR3","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1515\/raon-2016-0023","volume":"50","author":"B Segedin","year":"2016","unstructured":"Segedin B, Petric P (2016) Uncertainties in target volume delineation in radiotherapy\u2013are they relevant and what can we do about them? Radiol Oncol 50(3):254","journal-title":"Radiol Oncol"},{"issue":"1","key":"1513_CR4","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.meddos.2019.06.002","volume":"45","author":"NTC Fung","year":"2020","unstructured":"Fung NTC et al (2020) Automatic segmentation for adaptive planning in nasopharyngeal carcinoma IMRT: time, geometrical, and dosimetric analysis. Med Dosim 45(1):60\u201365","journal-title":"Med Dosim"},{"key":"1513_CR5","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.radonc.2021.05.003","volume":"160","author":"MV Sherer","year":"2021","unstructured":"Sherer MV et al (2021) Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: a critical review. Radiother Oncol 160:185\u2013191","journal-title":"Radiother Oncol"},{"issue":"1","key":"1513_CR6","doi-asserted-by":"crossref","DOI":"10.1002\/acm2.13873","volume":"24","author":"J Roper","year":"2023","unstructured":"Roper J, Lin M-H, Rong Y (2023) Extensive upfront validation and testing are needed prior to the clinical implementation of AI-based auto-segmentation tools. J Appl Clin Med Phys 24(1):e13873","journal-title":"J Appl Clin Med Phys"},{"issue":"4","key":"1513_CR7","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/j.clon.2023.01.014","volume":"35","author":"S Hindocha","year":"2023","unstructured":"Hindocha S et al (2023) Artificial intelligence for radiotherapy auto-contouring: current use, perceptions of and barriers to implementation. Clin Oncol 35(4):219\u2013226","journal-title":"Clin Oncol"},{"issue":"2","key":"1513_CR8","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.1002\/mp.16188","volume":"50","author":"L Hadjiiski","year":"2023","unstructured":"Hadjiiski L et al (2023) AAPM task group report 273: recommendations on best practices for AI and machine learning for computer-aided diagnosis in medical imaging. Med Phys 50(2):e1\u2013e24","journal-title":"Med Phys"},{"key":"1513_CR9","unstructured":"Artificial Intelligence in Medical Physics. 2023, Vienna: INTERNATIONAL ATOMIC ENERGY AGENCY."},{"issue":"4","key":"1513_CR10","doi-asserted-by":"crossref","DOI":"10.1016\/j.jmir.2024.101733","volume":"55","author":"M Chau","year":"2024","unstructured":"Chau M (2024) Ethical, legal, and regulatory landscape of artificial intelligence in Australian healthcare and ethical integration in radiography: a narrative review. J Med Imaging Radiation Sci 55(4):101733","journal-title":"J Med Imaging Radiation Sci"},{"issue":"2","key":"1513_CR11","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1148\/radiol.2019191586","volume":"293","author":"JR Geis","year":"2019","unstructured":"Geis JR et al (2019) Ethics of artificial intelligence in radiology: summary of the joint European and North American multisociety statement. Radiology 293(2):436\u2013440","journal-title":"Radiology"},{"key":"1513_CR12","doi-asserted-by":"crossref","DOI":"10.1016\/j.radonc.2024.110345","volume":"197","author":"C Hurkmans","year":"2024","unstructured":"Hurkmans C et al (2024) A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy. Radiother Oncol 197:110345","journal-title":"Radiother Oncol"},{"issue":"5","key":"1513_CR13","doi-asserted-by":"crossref","first-page":"486","DOI":"10.1111\/1754-9485.13289","volume":"65","author":"LM Kenny","year":"2021","unstructured":"Kenny LM, Nevin M, Fitzpatrick K (2021) Ethics and standards in the use of artificial intelligence in medicine on behalf of the Royal Australian and New Zealand College of Radiologists. J Med Imaging Radiat Oncol 65(5):486\u2013494","journal-title":"J Med Imaging Radiat Oncol"},{"issue":"4","key":"1513_CR14","doi-asserted-by":"crossref","first-page":"e240225","DOI":"10.1148\/ryai.240225","volume":"6","author":"MG Linguraru","year":"2024","unstructured":"Linguraru MG et al (2024) Clinical, cultural, computational, and regulatory considerations to deploy AI in radiology: perspectives of RSNA and MICCAI experts. Radiol Artif Intell 6(4):e240225","journal-title":"Radiol Artif Intell"},{"key":"1513_CR15","unstructured":"RANZCR. Ethical Principles for Artificial Intelligence in Medicine. Available from: https:\/\/www.ranzcr.com\/college\/document-library\/ethical-principles-for-ai-in-medicine."},{"key":"1513_CR16","unstructured":"RANZCR. Standards of Practice for AI in Radiation Oncology - Consultation Draft. Available from: https:\/\/www.ranzcr.com\/search\/standards-of-practice-for-ai-in-radiation-oncology-consultation-draft."},{"key":"1513_CR17","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/978-3-031-64049-0_9","volume-title":"Artificial intelligence in medical imaging technology: an introduction","author":"E Seeram","year":"2024","unstructured":"Seeram E, Kanade V (2024) Ethical and regulatory considerations. Artificial intelligence in medical imaging technology: an introduction. Springer, pp 151\u2013167"},{"issue":"4","key":"1513_CR18","volume":"55","author":"N Stogiannos","year":"2024","unstructured":"Stogiannos N et al (2024) A multidisciplinary team and multiagency approach for AI implementation: a commentary for medical imaging and radiotherapy key stakeholders. J Med Imaging Radiation Sci 55(4):101717","journal-title":"J Med Imaging Radiation Sci"},{"key":"1513_CR19","doi-asserted-by":"crossref","unstructured":"Theriault-Lauzier P et al (2024) A responsible framework for applying artificial intelligence on medical images and signals at the point-of-care: the PACS-AI platform. Can J Cardiol 40(10):1828\u20131840","DOI":"10.1016\/j.cjca.2024.05.025"},{"issue":"2","key":"1513_CR20","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1177\/08465371231222229","volume":"75","author":"AP Brady","year":"2024","unstructured":"Brady AP et al (2024) Developing, purchasing, implementing and monitoring AI tools in radiology: practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA. Can Assoc Radiol J 75(2):226\u2013244","journal-title":"Can Assoc Radiol J"},{"key":"1513_CR21","doi-asserted-by":"crossref","first-page":"3786","DOI":"10.1007\/s00330-020-07684-x","volume":"31","author":"P Omoumi","year":"2021","unstructured":"Omoumi P et al (2021) To buy or not to buy\u2014evaluating commercial AI solutions in radiology (the ECLAIR guidelines). Eur Radiol 31:3786\u20133796","journal-title":"Eur Radiol"},{"issue":"4","key":"1513_CR22","doi-asserted-by":"crossref","first-page":"827","DOI":"10.1016\/j.ijrobp.2020.04.011","volume":"107","author":"D Lin","year":"2020","unstructured":"Lin D et al (2020) A systematic review of contouring guidelines in radiation oncology: analysis of frequency, methodology, and delivery of consensus recommendations. Int J Radiat Oncol Biol Phys 107(4):827\u2013835","journal-title":"Int J Radiat Oncol Biol Phys"},{"issue":"1","key":"1513_CR23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12880-015-0042-7","volume":"15","author":"AA Taha","year":"2015","unstructured":"Taha AA, Hanbury A (2015) Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging 15(1):1\u201328","journal-title":"BMC Med Imaging"},{"issue":"9","key":"1513_CR24","doi-asserted-by":"crossref","first-page":"e929","DOI":"10.1002\/mp.14320","volume":"47","author":"T Vrtovec","year":"2020","unstructured":"Vrtovec T et al (2020) Auto-segmentation of organs at risk for head and neck radiotherapy planning: from atlas-based to deep learning methods. Med Phys 47(9):e929\u2013e950","journal-title":"Med Phys"},{"key":"1513_CR25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.phro.2019.12.001","volume":"13","author":"F Vaassen","year":"2020","unstructured":"Vaassen F et al (2020) Evaluation of measures for assessing time-saving of automatic organ-at-risk segmentation in radiotherapy. Phys Imaging Radiation Oncol 13:1\u20136","journal-title":"Phys Imaging Radiation Oncol"},{"key":"1513_CR26","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1007\/s10278-021-00460-3","volume":"34","author":"KJ Kiser","year":"2021","unstructured":"Kiser KJ et al (2021) Novel autosegmentation spatial similarity metrics capture the time required to correct segmentations better than traditional metrics in a thoracic cavity segmentation workflow. J Digit Imaging 34:541\u2013553","journal-title":"J Digit Imaging"},{"issue":"7","key":"1513_CR27","doi-asserted-by":"crossref","DOI":"10.2196\/26151","volume":"23","author":"S Nikolov","year":"2021","unstructured":"Nikolov S et al (2021) Clinically applicable segmentation of head and neck anatomy for radiotherapy: deep learning algorithm development and validation study. J Med Internet Res 23(7):e26151","journal-title":"J Med Internet Res"},{"issue":"4","key":"1513_CR28","doi-asserted-by":"crossref","first-page":"667","DOI":"10.3390\/diagnostics13040667","volume":"13","author":"H Baroudi","year":"2023","unstructured":"Baroudi H et al (2023) Automated contouring and planning in radiation therapy: what is \u2018Clinically Acceptable\u2019? Diagnostics 13(4):667","journal-title":"Diagnostics"},{"issue":"4","key":"1513_CR29","doi-asserted-by":"crossref","first-page":"2570","DOI":"10.1002\/mp.15525","volume":"49","author":"J Duan","year":"2022","unstructured":"Duan J et al (2022) Evaluating the clinical acceptability of deep learning contours of prostate and organs-at-risk in an automated prostate treatment planning process. Med Phys 49(4):2570\u20132581","journal-title":"Med Phys"},{"issue":"14","key":"1513_CR30","doi-asserted-by":"crossref","DOI":"10.1088\/1361-6560\/aacb65","volume":"63","author":"JP Kieselmann","year":"2018","unstructured":"Kieselmann JP et al (2018) Geometric and dosimetric evaluations of atlas-based segmentation methods of MR images in the head and neck region. Phys Med Biol 63(14):145007","journal-title":"Phys Med Biol"},{"key":"1513_CR31","doi-asserted-by":"crossref","unstructured":"Mackay K et al (2023) A review of the metrics used to assess auto-contouring systems in radiotherapy. Clin Oncol 35(6):354\u2013369","DOI":"10.1016\/j.clon.2023.01.016"},{"issue":"11","key":"1513_CR32","doi-asserted-by":"crossref","first-page":"5105","DOI":"10.1002\/mp.13200","volume":"45","author":"MJ Gooding","year":"2018","unstructured":"Gooding MJ et al (2018) Comparative evaluation of autocontouring in clinical practice: a practical method using the Turing test. Med Phys 45(11):5105\u20135115","journal-title":"Med Phys"},{"issue":"6","key":"1513_CR33","doi-asserted-by":"crossref","first-page":"061104","DOI":"10.1117\/1.JMI.10.6.061104","volume":"10","author":"K Drukker","year":"2023","unstructured":"Drukker K et al (2023) Toward fairness in artificial intelligence for medical image analysis: identification and mitigation of potential biases in the roadmap from data collection to model deployment. J Med Imaging 10(6):061104\u2013061104","journal-title":"J Med Imaging"},{"key":"1513_CR34","unstructured":"Lexology. Implications of the EU AI Act on medtech companies. 2024 [cited 2024 August 31]; Available from: https:\/\/www.lexology.com\/library\/detail.aspx?g=0964a85c-73d5-49c2-b54b-c201bda52506#:~:text=In%20that%20case%2C%20the%20AI,as%20the%20MDR%20or%20IVDR.&text=As%20noted%20above%2C%20the%20AI,as%20of%202%20August%202026."},{"key":"1513_CR35","unstructured":"Administration, T.G. Artificial Intelligence (AI) and medical device software. 2024 [cited 2024 21 August 2024]; Available from: https:\/\/www.tga.gov.au\/how-we-regulate\/manufacturing\/manufacture-medical-device\/manufacture-specific-types-medical-devices\/software-based-medical-devices\/artificial-intelligence-ai-and-medical-device-software."},{"key":"1513_CR36","unstructured":"IEEE. THE IEEE GLOBAL INITIATIVE 2.0 ON ETHICS OF AUTONOMOUS AND INTELLIGENT SYSTEMS. Available from: https:\/\/standards.ieee.org\/industry-connections\/activities\/ieee-global-initiative\/."},{"key":"1513_CR37","unstructured":"ISO\/IEC. ISO\/IEC JTC 1\/SC 42. 21 August 2024]; Available from: https:\/\/www.iso.org\/committee\/6794475.html."},{"key":"1513_CR38","doi-asserted-by":"crossref","unstructured":"Mayo CS et al (2018) American Association of Physicists in Medicine Task Group 263: standardizing nomenclatures in radiation oncology. Int J Radiation Oncol Biol Phys 100(4):1057\u20131066.","DOI":"10.1016\/j.ijrobp.2017.12.013"},{"key":"1513_CR39","unstructured":"ISO\/IEC. ISO\/IEC 27000 family. 2024 [cited 2024 August 31]; Available from: https:\/\/www.iso.org\/standard\/iso-iec-27000-family."},{"key":"1513_CR40","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1016\/j.phro.2022.11.009","volume":"24","author":"MJ Gooding","year":"2022","unstructured":"Gooding MJ et al (2022) Multicenter comparison of measures for quantitative evaluation of contouring in radiotherapy. Phys Imaging Radiation Oncol 24:152\u2013158","journal-title":"Phys Imaging Radiation Oncol"},{"key":"1513_CR41","doi-asserted-by":"crossref","unstructured":"Trimpl MJ et al (2022) Beyond automatic medical image segmentation\u2014the spectrum between fully manual and fully automatic delineation. Phys Med Biol 67(12): 12TR01.","DOI":"10.1088\/1361-6560\/ac6d9c"},{"key":"1513_CR42","doi-asserted-by":"crossref","unstructured":"Wang G et al (2020) Uncertainty-guided efficient interactive refinement of fetal brain segmentation from stacks of MRI slices. Springer, Cham.","DOI":"10.1007\/978-3-030-59719-1_28"},{"key":"1513_CR43","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.neucom.2019.01.103","volume":"338","author":"G Wang","year":"2019","unstructured":"Wang G et al (2019) Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing 338:34\u201345","journal-title":"Neurocomputing"},{"issue":"8","key":"1513_CR44","volume":"23","author":"DJ Rhee","year":"2022","unstructured":"Rhee DJ et al (2022) Automatic contouring QA method using a deep learning\u2013based autocontouring system. J Appl Clin Med Phys 23(8):e13647","journal-title":"J Appl Clin Med Phys"},{"issue":"11","key":"1513_CR45","doi-asserted-by":"crossref","first-page":"5086","DOI":"10.1002\/mp.13814","volume":"46","author":"DJ Rhee","year":"2019","unstructured":"Rhee DJ et al (2019) Automatic detection of contouring errors using convolutional neural networks. Med Phys 46(11):5086\u20135097","journal-title":"Med Phys"},{"issue":"2","key":"1513_CR46","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/j.ejca.2008.10.026","volume":"45","author":"EA Eisenhauer","year":"2009","unstructured":"Eisenhauer EA et al (2009) New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Euro J Cancer 45(2):228\u2013247","journal-title":"Euro J Cancer"},{"key":"1513_CR47","unstructured":"Holm S (2002) Principles of biomedical ethics, 5th edn.: Beauchamp TL, Childress J F. Oxford University Press, 2001, \u00a3 19.95, pp 454. ISBN 0-19-514332-9. Institute of Medical Ethics."},{"issue":"1","key":"1513_CR48","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1038\/s41746-020-0253-3","volume":"3","author":"MP Sendak","year":"2020","unstructured":"Sendak MP et al (2020) Presenting machine learning model information to clinical end users with model facts labels. NPJ Digit Med 3(1):41","journal-title":"NPJ Digit Med"},{"key":"1513_CR49","unstructured":"Administration, F.a.D. Transparency for machine learning-enabled medical devices: guiding principles. 2024 [cited 2024 27 September]; Available from: https:\/\/www.fda.gov\/medical-devices\/software-medical-device-samd\/transparency-machine-learning-enabled-medical-devices-guiding-principles."},{"key":"1513_CR50","doi-asserted-by":"crossref","unstructured":"Mongan J, Moy L, Kahn Jr CE (2020) Checklist for artificial intelligence in medical imaging (CLAIM): a guide for authors and reviewers. 2020, Radiological Society of North America. p. e200029.","DOI":"10.1148\/ryai.2020200029"},{"issue":"10","key":"1513_CR51","doi-asserted-by":"crossref","first-page":"e537","DOI":"10.1016\/S2589-7500(20)30218-1","volume":"2","author":"X Liu","year":"2020","unstructured":"Liu X et al (2020) Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Lancet Digital Health 2(10):e537\u2013e548","journal-title":"Lancet Digital Health"},{"key":"1513_CR52","doi-asserted-by":"crossref","unstructured":"Vasey B et al (2022) Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ, p. 377.","DOI":"10.1136\/bmj-2022-070904"},{"issue":"10","key":"1513_CR53","doi-asserted-by":"crossref","first-page":"e549","DOI":"10.1016\/S2589-7500(20)30219-3","volume":"2","author":"SC Rivera","year":"2020","unstructured":"Rivera SC et al (2020) Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Lancet Digital Health 2(10):e549\u2013e560","journal-title":"Lancet Digital Health"},{"issue":"2","key":"1513_CR54","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1007\/s10278-021-00573-9","volume":"35","author":"LB van den Oever","year":"2022","unstructured":"van den Oever LB et al (2022) Qualitative evaluation of common quantitative metrics for clinical acceptance of automatic segmentation: a case study on heart contouring from CT images by deep learning algorithms. J Digit Imaging 35(2):240\u2013247","journal-title":"J Digit Imaging"},{"issue":"5","key":"1513_CR55","doi-asserted-by":"crossref","first-page":"1047","DOI":"10.1016\/j.ijrobp.2022.11.021","volume":"115","author":"L Guzene","year":"2023","unstructured":"Guzene L et al (2023) Assessing interobserver variability in the delineation of structures in radiation oncology: a systematic review. Int J Radiation Oncol Biol Phys 115(5):1047\u20131060","journal-title":"Int J Radiation Oncol Biol Phys"},{"issue":"7","key":"1513_CR56","doi-asserted-by":"crossref","first-page":"e43","DOI":"10.1002\/mp.12256","volume":"44","author":"KK Brock","year":"2017","unstructured":"Brock KK et al (2017) Use of image registration and fusion algorithms and techniques in radiotherapy: report of the AAPM Radiation Therapy Committee Task Group No. 132. Medical Physics 44(7):e43\u2013e76","journal-title":"Medical Physics"},{"issue":"7","key":"1513_CR57","doi-asserted-by":"crossref","first-page":"903","DOI":"10.1109\/TMI.2004.828354","volume":"23","author":"SK Warfield","year":"2004","unstructured":"Warfield SK, Zou KH, Wells WM (2004) Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans Med Imaging 23(7):903\u2013921","journal-title":"IEEE Trans Med Imaging"},{"issue":"11","key":"1513_CR58","doi-asserted-by":"crossref","first-page":"e2027426","DOI":"10.1001\/jamanetworkopen.2020.27426","volume":"3","author":"O Oktay","year":"2020","unstructured":"Oktay O et al (2020) Evaluation of deep learning to augment image-guided radiotherapy for head and neck and prostate cancers. JAMA Netw Open 3(11):e2027426\u2013e2027426","journal-title":"JAMA Netw Open"},{"key":"1513_CR59","unstructured":"FDA. Good Machine Learning Practice for Medical Device Development: Guiding Principles. 2021 [cited 2024 31 August]; Available from: https:\/\/www.fda.gov\/medical-devices\/software-medical-device-samd\/good-machine-learning-practice-medical-device-development-guiding-principles."},{"key":"1513_CR60","doi-asserted-by":"crossref","unstructured":"Yang J, Sharp GC, Gooding MJ (2021) Auto-segmentation for radiation oncology: state of the art, 1st edn. Series in Medical Physics and Biomedical Engineering Series. Taylor & Francis Group, Milton.","DOI":"10.1201\/9780429323782-1"},{"key":"1513_CR61","unstructured":"Union, E. The EU Artificial Intelligence Act. 2024 [cited 2024 August 31]; Available from: https:\/\/artificialintelligenceact.eu\/."},{"key":"1513_CR62","doi-asserted-by":"crossref","unstructured":"Zenk M et al (2024) Comparative benchmarking of failure detection methods in medical image segmentation: unveiling the role of confidence aggregation. arXiv preprint arXiv:2406.03323.","DOI":"10.1016\/j.media.2024.103392"},{"key":"1513_CR63","doi-asserted-by":"crossref","first-page":"1213068","DOI":"10.3389\/fonc.2023.1213068","volume":"13","author":"PJ Doolan","year":"2023","unstructured":"Doolan PJ et al (2023) A clinical evaluation of the performance of five commercial artificial intelligence contouring systems for radiotherapy. Front Oncol 13:1213068","journal-title":"Front Oncol"},{"key":"1513_CR64","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.phro.2019.07.005","volume":"11","author":"V Panettieri","year":"2019","unstructured":"Panettieri V et al (2019) Development of a multicentre automated model to reduce planning variability in radiotherapy of prostate cancer. Phys Imaging Radiation Oncol 11:34\u201340","journal-title":"Phys Imaging Radiation Oncol"},{"key":"1513_CR65","doi-asserted-by":"crossref","DOI":"10.1016\/j.radonc.2023.109747","volume":"186","author":"Y McQuinlan","year":"2023","unstructured":"McQuinlan Y et al (2023) An investigation into the risk of population bias in deep learning autocontouring. Radiother Oncol 186:109747","journal-title":"Radiother Oncol"},{"key":"1513_CR66","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.phro.2020.10.008","volume":"16","author":"S Terparia","year":"2020","unstructured":"Terparia S et al (2020) Automatic evaluation of contours in radiotherapy planning utilising conformity indices and machine learning. Phys Imaging Radiation Oncol 16:149\u2013155","journal-title":"Phys Imaging Radiation Oncol"},{"issue":"3","key":"1513_CR67","doi-asserted-by":"crossref","first-page":"297","DOI":"10.2307\/1932409","volume":"26","author":"LR Dice","year":"1945","unstructured":"Dice LR (1945) Measures of the amount of ecologic association between species. Ecology 26(3):297\u2013302","journal-title":"Ecology"},{"issue":"9","key":"1513_CR68","doi-asserted-by":"crossref","first-page":"850","DOI":"10.1109\/34.232073","volume":"15","author":"DP Huttenlocher","year":"1993","unstructured":"Huttenlocher DP, Klanderman GA, Rucklidge WJ (1993) Comparing images using the Hausdorff distance. IEEE Trans Pattern Anal Mach Intell 15(9):850\u2013863","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"1513_CR69","doi-asserted-by":"crossref","first-page":"S282","DOI":"10.1016\/S0167-8140(18)30841-7","volume":"127","author":"M Gooding","year":"2018","unstructured":"Gooding M et al (2018) PV-0531: multi-centre evaluation of atlas-based and deep learning contouring using a modified Turing test. Radiother Oncol 127:S282\u2013S283","journal-title":"Radiother Oncol"},{"key":"1513_CR70","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.radonc.2020.09.008","volume":"153","author":"L Vandewinckele","year":"2020","unstructured":"Vandewinckele L et al (2020) Overview of artificial intelligence-based applications in radiotherapy: recommendations for implementation and quality assurance. Radiother Oncol 153:55\u201366","journal-title":"Radiother Oncol"}],"container-title":["Physical and Engineering Sciences in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13246-024-01513-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13246-024-01513-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13246-024-01513-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,14]],"date-time":"2025-04-14T05:04:50Z","timestamp":1744607090000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s13246-024-01513-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,13]]},"references-count":70,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,3]]}},"alternative-id":["1513"],"URL":"https:\/\/doi.org\/10.1007\/s13246-024-01513-x","relation":{},"ISSN":["2662-4729","2662-4737"],"issn-type":[{"value":"2662-4729","type":"print"},{"value":"2662-4737","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,13]]},"assertion":[{"value":"21 April 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 January 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors declare there are no conflicts of interest related to this research. No financial or personal relationships with any individuals or organizations that could potentially bias the findings or interpretations of the study have been disclosed.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This study was reviewed and ethically approved by the Human Research Ethics Committee (HREC) at Sir Charles Gairdner Hospital (SCGH) in Perth, Western Australia (Protocol Revision Number 4979). This approval allows us to publish the results of the research based on participant data. We also confirm that the research was conducted in accordance with the ethical principles outlined in the Declaration of Helsinki and all relevant local regulations.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}