{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T18:58:50Z","timestamp":1790621930393,"version":"4.1.0"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,6,28]],"date-time":"2021-06-28T00:00:00Z","timestamp":1624838400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,28]],"date-time":"2021-06-28T00:00:00Z","timestamp":1624838400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003130","name":"Fonds Wetenschappelijk Onderzoek","doi-asserted-by":"publisher","award":["SA6419N"],"award-info":[{"award-number":["SA6419N"]}],"id":[{"id":"10.13039\/501100003130","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010648","name":"Universitaire Ziekenhuizen Leuven, KU Leuven","doi-asserted-by":"publisher","award":["C24\/18\/047"],"award-info":[{"award-number":["C24\/18\/047"]}],"id":[{"id":"10.13039\/501100010648","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Radiat Oncol"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>In radiotherapy inaccuracy in organ at risk (OAR) delineation can impact treatment plan optimisation and treatment plan evaluation. Brouwer et al. showed significant interobserver variability (IOV) in OAR delineation in head and neck cancer (HNC) and published international consensus guidelines (ICG) for OAR delineation in 2015. The aim of our study was to evaluate IOV in the presence of these guidelines.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>HNC radiation oncologists (RO) from each Belgian radiotherapy centre were invited to complete a survey and submit contours for 5 HNC cases. Reference contours (OARref) were obtained by a clinically validated artificial intelligence-tool trained using ICG. Dice similarity coefficients (DSC), mean surface distance (MSD) and 95% Hausdorff distances (HD95) were used for comparison.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Fourteen of twenty-two RO (64%) completed the survey and submitted delineations. Thirteen (93%) confirmed the use of delineation guidelines, of which six (43%) used the ICG. The OARs whose delineations agreed best with the OARref were mandible [median DSC 0.9, range (0.8\u20130.9); median MSD 1.1\u00a0mm, range (0.8\u20138.3), median HD95 3.4\u00a0mm, range (1.5\u201338.7)], brainstem [median DSC 0.9 (0.6\u20130.9); median MSD 1.5\u00a0mm (1.1\u20134.0), median HD95 4.0\u00a0mm (2.3\u201315.0)], submandibular glands [median DSC 0.8 (0.5\u20130.9); median MSD 1.2\u00a0mm (0.9\u20132.5), median HD95 3.1\u00a0mm (1.8\u201312.2)] and parotids [median DSC 0.9 (0.6\u20130.9); median MSD 1.9\u00a0mm (1.2\u20134.2), median HD95 5.1\u00a0mm (3.1\u201319.2)]. Oral cavity, cochleas, PCMs, supraglottic larynx and glottic area showed more variation. RO who used the consensus guidelines showed significantly less IOV (p\u2009=\u20090.008).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>Although ICG for delineation of OARs in HNC exist, they are only implemented by about half of RO participating in this study, which partly explains the delineation variability. However, this study highlights that guidelines alone do not suffice to eliminate IOV and that more effort needs to be done to accomplish further treatment standardisation, for example with artificial intelligence.\n<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s13014-020-01677-2","type":"journal-article","created":{"date-parts":[[2021,6,28]],"date-time":"2021-06-28T20:02:27Z","timestamp":1624910547000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":72,"title":["Interobserver variability in organ at risk delineation in head and neck cancer"],"prefix":"10.1186","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1934-4845","authenticated-orcid":false,"given":"J.","family":"van der Veen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"A.","family":"Gulyban","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"S.","family":"Willems","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"F.","family":"Maes","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"S.","family":"Nuyts","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,6,28]]},"reference":[{"issue":"15","key":"1677_CR1","doi-asserted-by":"publisher","first-page":"6015","DOI":"10.1200\/JCO.2017.35.15_suppl.6015","volume":"35","author":"J Bourhis","year":"2017","unstructured":"Bourhis J, Auperin A, Alfonsi M, Sunxu S, Rives M, Pointreau Y, et al. Dose escalation of radiotherapy (RT) for locally advanced head and neck carcinomas treated with concomitant chemotherapy (CT) and RT: Results of the GORTEC 2004\u201301 randomized trial. J Clin Oncol. 2017;35(15):6015\u20136015. https:\/\/doi.org\/10.1200\/JCO.2017.35.15_suppl.6015.","journal-title":"J Clin Oncol."},{"issue":"1","key":"1677_CR2","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1016\/j.radonc.2009.04.014","volume":"92","author":"J-P Pignon","year":"2009","unstructured":"Pignon J-P, le Ma\u00eetre A, Maillard E, Bourhis J. Meta-analysis of chemotherapy in head and neck cancer (MACH-NC): An update on 93 randomised trials and 17,346 patients. Radiother Oncol. 2009;92(1):4\u201314.","journal-title":"Radiother Oncol"},{"issue":"4","key":"1677_CR3","doi-asserted-by":"publisher","first-page":"1088","DOI":"10.1016\/j.ijrobp.2008.05.042","volume":"73","author":"S Nuyts","year":"2009","unstructured":"Nuyts S, Dirix P, Clement PMJ, Vander PV, Delaere P, Schoenaers J, et al. Impact of adding concomitant chemotherapy to hyperfractionated accelerated radiotherapy for advanced head-and-neck squamous cell carcinoma. Int J Radiat Oncol. 2009;73(4):1088\u201395.","journal-title":"Int J Radiat Oncol."},{"issue":"3","key":"1677_CR4","doi-asserted-by":"publisher","first-page":"360","DOI":"10.1016\/j.radonc.2014.06.001","volume":"111","author":"AK Due","year":"2014","unstructured":"Due AK, Vogelius IR, Aznar MC, Bentzen SM, Berthelsen AK, Korreman SS, et al. Recurrences after intensity modulated radiotherapy for head and neck squamous cell carcinoma more likely to originate from regions with high baseline [18F]-FDG uptake. Radiother Oncol. 2014;111(3):360\u20135.","journal-title":"Radiother Oncol"},{"issue":"10","key":"1677_CR5","doi-asserted-by":"publisher","first-page":"636","DOI":"10.1016\/j.clon.2014.05.001","volume":"26","author":"E Bayman","year":"2014","unstructured":"Bayman E, Prestwich RJD, Speight R, Aspin L, Garratt L, Wilson S, et al. Patterns of failure after intensity-modulated radiotherapy in head and neck squamous cell carcinoma using compartmental clinical target volume delineation. Clin Oncol [Internet]. 2014;26(10):636\u201342.","journal-title":"Clin Oncol [Internet]"},{"issue":"29","key":"1677_CR6","doi-asserted-by":"publisher","first-page":"3277","DOI":"10.1200\/JCO.2015.61.2994","volume":"33","author":"V Gr\u00e9goire","year":"2015","unstructured":"Gr\u00e9goire V, Langendijk JA, Nuyts S. Advances in radiotherapy for head and neck cancer. J Clin Oncol. 2015;33(29):3277\u201384.","journal-title":"J Clin Oncol"},{"issue":"2","key":"1677_CR7","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/S1470-2045(10)70290-4","volume":"12","author":"CM Nutting","year":"2011","unstructured":"Nutting CM, Morden JP, Harrington KJ, Urbano TG, Bhide SA, Clark C, et al. Parotid-sparing intensity modulated versus conventional radiotherapy in head and neck cancer (PARSPORT): a phase 3 multicentre randomised controlled trial. Lancet Oncol. 2011;12(2):127\u201336.","journal-title":"Lancet Oncol."},{"issue":"S1","key":"1677_CR8","doi-asserted-by":"publisher","first-page":"E1481","DOI":"10.1002\/hed.24263","volume":"38","author":"S Ghosh-Laskar","year":"2015","unstructured":"Ghosh-Laskar S, Yathiraj PH, Dutta D, Rangarajan V, Purandare N, Gupta T, et al. Prospective randomized controlled trial to compare 3-dimensional conformal radiotherapy to intensity-modulated radiotherapy in head and neck squamous cell carcinoma: long-term results. Head Neck. 2015;38(S1):E1481\u2013E14871487.","journal-title":"Head Neck."},{"issue":"3","key":"1677_CR9","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1016\/j.radonc.2012.07.001","volume":"104","author":"T Gupta","year":"2012","unstructured":"Gupta T, Agarwal J, Jain S, Phurailatpam R, Kannan S, Ghosh-Laskar S, et al. Three-dimensional conformal radiotherapy (3D-CRT) versus intensity modulated radiation therapy (IMRT) in squamous cell carcinoma of the head and neck: a randomized controlled trial. Radiother Oncol. 2012;104(3):343\u20138.","journal-title":"Radiother Oncol"},{"issue":"6","key":"1677_CR10","doi-asserted-by":"publisher","first-page":"634","DOI":"10.1016\/j.oraloncology.2013.02.013","volume":"49","author":"S Rathod","year":"2013","unstructured":"Rathod S, Gupta T, Ghosh-Laskar S, Murthy V, Budrukkar A, Agarwal J. Quality-of-life (QOL) outcomes in patients with head and neck squamous cell carcinoma (HNSCC) treated with intensity-modulated radiation therapy (IMRT) compared to three-dimensional conformal radiotherapy (3D-CRT): evidence from a prospective randomized s. Oral Oncol. 2013;49(6):634\u201342.","journal-title":"Oral Oncol"},{"issue":"1016","key":"1677_CR11","doi-asserted-by":"publisher","first-page":"e530","DOI":"10.1259\/bjr\/66693547","volume":"85","author":"M Mukesh","year":"2012","unstructured":"Mukesh M, Benson R, Jena R, Hoole A, Roques T, Scrase C, et al. Interobserver variation in clinical target volume and organs at risk segmentation in post-parotidectomy radiotherapy: can segmentation protocols help? Br J Radiol. 2012;85(1016):e530\u2013e536536.","journal-title":"Br J Radiol"},{"issue":"4","key":"1677_CR12","doi-asserted-by":"publisher","first-page":"972","DOI":"10.1016\/j.ijrobp.2006.10.029","volume":"67","author":"JS Cooper","year":"2007","unstructured":"Cooper JS, Mukherji SK, Toledano AY, Beldon C, Schmalfuss IM, Amdur R, et al. An evaluation of the variability of tumor-shape definition derived by experienced observers from CT images of supraglottic carcinomas (ACRIN protocol 6658). Int J Radiat Oncol. 2007;67(4):972\u20135.","journal-title":"Int J Radiat Oncol"},{"issue":"1","key":"1677_CR13","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1016\/S0360-3016(01)01751-5","volume":"52","author":"C Rasch","year":"2002","unstructured":"Rasch C, Eisbruch A, Remeijer P, Bos L, Hoogeman M, van Herk M, et al. Irradiation of paranasal sinus tumors, a delineation and dose comparison study. Int J Radiat Oncol. 2002;52(1):120\u20137.","journal-title":"Int J Radiat Oncol"},{"issue":"3","key":"1677_CR14","doi-asserted-by":"publisher","first-page":"726","DOI":"10.1016\/j.ijrobp.2006.01.014","volume":"65","author":"AC Riegel","year":"2006","unstructured":"Riegel AC, Berson AM, Destian S, Ng T, Tena LB, Mitnick RJ, et al. Variability of gross tumor volume delineation in head-and-neck cancer using CT and PET\/CT fusion. Int J Radiat Oncol. 2006;65(3):726\u201332.","journal-title":"Int J Radiat Oncol"},{"issue":"3","key":"1677_CR15","doi-asserted-by":"publisher","first-page":"553","DOI":"10.1016\/S0360-3016(97)00853-5","volume":"40","author":"R Hermans","year":"1998","unstructured":"Hermans R, Feron M, Bellon E, Dupont P, Van den Bogaert W, Baert AL. Laryngeal tumor volume measurements determined with CT: a study on intra- and interobserver variability. Int J Radiat Oncol. 1998;40(3):553\u20137.","journal-title":"Int J Radiat Oncol"},{"key":"1677_CR16","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.radonc.2019.04.006","volume":"137","author":"J van der Veen","year":"2019","unstructured":"van der Veen J, Gulyban A, Nuyts S. Interobserver variability in delineation of target volumes in head and neck cancer. Radiother Oncol. 2019;137:9\u201315. https:\/\/doi.org\/10.1016\/j.radonc.2019.04.006.","journal-title":"Radiother Oncol."},{"issue":"1","key":"1677_CR17","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1186\/1748-717X-7-32","volume":"7","author":"CL Brouwer","year":"2012","unstructured":"Brouwer CL, Steenbakkers RJ, van den Heuvel E, Duppen JC, Navran A, Bijl HP, et al. 3D Variation in delineation of head and neck organs at risk. Radiat Oncol. 2012;7(1):32.","journal-title":"Radiat Oncol"},{"issue":"1","key":"1677_CR18","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1016\/j.radonc.2015.07.041","volume":"117","author":"CL Brouwer","year":"2015","unstructured":"Brouwer CL, Steenbakkers RJHM, Bourhis J, Budach W, Grau C, Gr\u00e9goire V, et al. CT-based delineation of organs at risk in the head and neck region: DAHANCA, EORTC, GORTEC, HKNPCSG, NCIC CTG, NCRI, NRG oncology and TROG consensus guidelines. Radiother Oncol. 2015;117(1):83\u201390.","journal-title":"Radiother Oncol"},{"key":"1677_CR19","doi-asserted-by":"crossref","unstructured":"Willems S, Crijns W, La Greca Saint-Esteven A, Van Der Veen J, Robben D, Depuydt T, et al. Clinical implementation of deepvoxnet for auto-delineation of organs at risk in head and neck cancer patients in radiotherapy. Vol. 11041 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2018.","DOI":"10.1007\/978-3-030-01201-4_24"},{"key":"1677_CR20","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1016\/j.radonc.2019.05.010","volume":"138","author":"J van der Veen","year":"2019","unstructured":"van der Veen J, Willems S, Deschuymer S, Robben D, Crijns W, Maes F, et al. Benefits of deep learning for delineation of organs at risk in head and neck cancer. Radiother Oncol. 2019;138:68\u201374.","journal-title":"Radiother Oncol"},{"issue":"14","key":"1677_CR21","doi-asserted-by":"publisher","first-page":"4557","DOI":"10.1088\/0031-9155\/56\/14\/021","volume":"56","author":"MA Deeley","year":"2011","unstructured":"Deeley MA, Chen A, Datteri R, Noble JH, Cmelak AJ, Donnelly EF, et al. Comparison of manual and automatic segmentation methods for brain structures in the presence of space-occupying lesions: a multi-expert study. Phys Med Biol. 2011;56(14):4557\u201377.","journal-title":"Phys Med Biol"},{"issue":"3","key":"1677_CR22","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1016\/j.radonc.2011.05.015","volume":"101","author":"Christianen MEMC, Langendijk JA, Westerlaan HE, Water TA Van De, Bijl HP","year":"2011","unstructured":"Christianen MEMC, Langendijk JA, Westerlaan HE, Water TA Van De, Bijl HP. Delineation of organs at risk involved in swallowing for radiotherapy treatment planning. Radiother Oncol. 2011;101(3):394\u2013402. https:\/\/doi.org\/10.1016\/j.radonc.2011.05.015.","journal-title":"Radiother Oncol."},{"issue":"4","key":"1677_CR23","doi-asserted-by":"publisher","first-page":"352","DOI":"10.1007\/s11547-014-0448-2","volume":"120","author":"D Genovesi","year":"2015","unstructured":"Genovesi D, Perrotti F, Trignani M, Di Pilla A, Vinciguerra A, Augurio A, et al. Delineating brachial plexus, cochlea, pharyngeal constrictor muscles and optic chiasm in head and neck radiotherapy: a CT-based model atlas. Radiol Medica. 2015;120(4):352\u201360.","journal-title":"Radiol Medica."},{"issue":"1","key":"1677_CR24","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1016\/j.radonc.2005.04.010","volume":"77","author":"X Geets","year":"2005","unstructured":"Geets X, Daisne J-F, Arcangeli S, Coche E, De PM, Duprez T, et al. Inter-observer variability in the delineation of pharyngo-laryngeal tumor, parotid glands and cervical spinal cord: comparison between CT-scan and MRI. Radiother Oncol. 2005;77(1):25\u201331.","journal-title":"Radiother Oncol"},{"issue":"1","key":"1677_CR25","doi-asserted-by":"publisher","first-page":"368","DOI":"10.1016\/j.ijrobp.2010.10.019","volume":"82","author":"BE Nelms","year":"2012","unstructured":"Nelms BE, Tom\u00e9 WA, Robinson G, Wheeler J. Variations in the contouring of organs at risk: test case from a patient with oropharyngeal cancer. Int J Radiat Oncol. 2012;82(1):368\u201378.","journal-title":"Int J Radiat Oncol."}],"container-title":["Radiation Oncology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13014-020-01677-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13014-020-01677-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13014-020-01677-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,28]],"date-time":"2021-06-28T20:07:19Z","timestamp":1624910839000},"score":1,"resource":{"primary":{"URL":"https:\/\/ro-journal.biomedcentral.com\/articles\/10.1186\/s13014-020-01677-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,28]]},"references-count":25,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1677"],"URL":"https:\/\/doi.org\/10.1186\/s13014-020-01677-2","relation":{},"ISSN":["1748-717X"],"issn-type":[{"value":"1748-717X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,28]]},"assertion":[{"value":"1 July 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 September 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 June 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"120"}}