{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T05:27:19Z","timestamp":1783661239116,"version":"3.55.0"},"reference-count":19,"publisher":"Wiley","issue":"8","license":[{"start":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T00:00:00Z","timestamp":1652745600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["aapm.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J Applied Clin Med Phys"],"published-print":{"date-parts":[[2022,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Purpose<\/jats:title><jats:p>To determine the most accurate similarity metric when using an independent system to verify automatically generated contours.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>A reference autocontouring system (primary system to create clinical contours) and a verification autocontouring system (secondary system to test the primary contours) were used to generate a pair of 6 female pelvic structures (UteroCervix [uterus\u00a0+\u00a0cervix], CTVn [nodal clinical target volume (CTV)], PAN [para\u2010aortic lymph nodes], bladder, rectum, and kidneys) on 49 CT scans from our institution and 38 from other institutions. Additionally, clinically acceptable and unacceptable contours were manually generated using the 49 internal CT scans. Eleven similarity metrics (volumetric Dice similarity coefficient (DSC), Hausdorff distance, 95% Hausdorff distance, mean surface distance, and surface DSC with tolerances from 1 to 10\u00a0mm) were calculated between the reference and the verification autocontours, and between the manually generated and the verification autocontours. A support vector machine (SVM) was used to determine the threshold that separates clinically acceptable and unacceptable contours for each structure. The 11 metrics were investigated individually and in certain combinations. Linear, radial basis function, sigmoid, and polynomial kernels were tested using the combinations of metrics as inputs for the SVM.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The highest contouring error detection accuracies were 0.91 for the UteroCervix, 0.90 for the CTVn, 0.89 for the PAN, 0.92 for the bladder, 0.95 for the rectum, and 0.97 for the kidneys and were achieved using surface DSCs with a thickness of 1, 2, or 3\u00a0mm. The linear kernel was the most accurate and consistent when a combination of metrics was used as an input for the SVM. However, the best model accuracy from the combinations of metrics was not better than the best model accuracy from a surface DSC as an input.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>We distinguished clinically acceptable contours from clinically unacceptable contours with an accuracy higher than 0.9 for the targets and critical structures in patients with cervical cancer; the most accurate similarity metric was surface DSC with a thickness of 1, 2, or 3\u00a0mm.<\/jats:p><\/jats:sec>","DOI":"10.1002\/acm2.13647","type":"journal-article","created":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T17:14:39Z","timestamp":1652807679000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["Automatic contouring QA method using a deep learning\u2013based autocontouring system"],"prefix":"10.1002","volume":"23","author":[{"given":"Dong Joo","family":"Rhee","sequence":"first","affiliation":[{"name":"The University of Texas Graduate School of Biomedical Sciences at Houston  Houston Texas USA"},{"name":"Department of Radiation Physics Division of Radiation Oncology The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chidinma P. Anakwenze","family":"Akinfenwa","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bastien","family":"Rigaud","sequence":"additional","affiliation":[{"name":"Department of Imaging Physics The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anuja","family":"Jhingran","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carlos E.","family":"Cardenas","sequence":"additional","affiliation":[{"name":"Department of Radiation Physics Division of Radiation Oncology The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lifei","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Radiation Physics Division of Radiation Oncology The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Surendra","family":"Prajapati","sequence":"additional","affiliation":[{"name":"Department of Radiation Physics Division of Radiation Oncology The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stephen F.","family":"Kry","sequence":"additional","affiliation":[{"name":"Department of Radiation Physics Division of Radiation Oncology The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristy K.","family":"Brock","sequence":"additional","affiliation":[{"name":"Department of Imaging Physics The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beth M.","family":"Beadle","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology Stanford University School of Medicine  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William","family":"Shaw","sequence":"additional","affiliation":[{"name":"Department of Medical Physics (G68) University of the Free State  Bloemfontein South Africa"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Frederika","family":"O'Reilly","sequence":"additional","affiliation":[{"name":"Department of Medical Physics (G68) University of the Free State  Bloemfontein South Africa"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeannette","family":"Parkes","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology and Medical Physics University of Cape Town and Groote Schuur Hospital  Cape Town South Africa"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hester","family":"Burger","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology and Medical Physics University of Cape Town and Groote Schuur Hospital  Cape Town South Africa"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nazia","family":"Fakie","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology and Medical Physics University of Cape Town and Groote Schuur Hospital  Cape Town South Africa"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chris","family":"Trauernicht","sequence":"additional","affiliation":[{"name":"Division of Medical Physics Stellenbosch University Tygerberg Academic Hospital  Cape Town South Africa"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hannah","family":"Simonds","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology Stellenbosch University Tygerberg Academic Hospital  Cape Town South Africa"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laurence E.","family":"Court","sequence":"additional","affiliation":[{"name":"Department of Radiation Physics Division of Radiation Oncology The University of Texas MD Anderson Cancer Center  Houston Texas USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2022,5,17]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.14030"},{"key":"e_1_2_9_3_1","doi-asserted-by":"publisher","DOI":"10.1118\/1.4906197"},{"key":"e_1_2_9_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2013.2251421"},{"key":"e_1_2_9_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2017.06.1932"},{"key":"e_1_2_9_6_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.13814"},{"key":"e_1_2_9_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.semradonc.2019.02.001"},{"key":"e_1_2_9_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compmedimag.2021.101907"},{"key":"e_1_2_9_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.14467"},{"key":"e_1_2_9_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2020.10.038"},{"key":"e_1_2_9_11_1","doi-asserted-by":"crossref","unstructured":"MilletariF NavabN AhmadiSA.V\u2010Net: fully convolutional neural networks for volumetric medical image segmentation.Proc \u2013 2016 4th Int Conf 3D Vision 3DV 2016.2016:565\u2010571.https:\/\/doi.org\/10.1109\/3DV.2016.79","DOI":"10.1109\/3DV.2016.79"},{"key":"e_1_2_9_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2572683"},{"key":"e_1_2_9_13_1","volume-title":"A Method of Establishing Groups of Equal Amplitude in Plant Sociology Based on Similarity of Species Content and Its Application to Analyses of the Vegetation on Danish Commons","author":"S\u00f8rensen TJ","year":"1948"},{"key":"e_1_2_9_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01999507"},{"key":"e_1_2_9_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2009.2013851"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.2196\/26151"},{"key":"e_1_2_9_17_1","doi-asserted-by":"crossref","unstructured":"BoserBE GuyonIM VapnikVN.A training algorithm for optimal margin classifiers.Proceedings of the Fifth Annual Workshop on Computational Learning Theory. COLT \u201992 New York NY USA.Association for Computing Machinery;1992:144\u2010152.https:\/\/doi.org\/10.1145\/130385.130401","DOI":"10.1145\/130385.130401"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00994018"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/5254.708428"},{"key":"e_1_2_9_20_1","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2020.01762"}],"container-title":["Journal of Applied Clinical Medical Physics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/acm2.13647","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/acm2.13647","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/acm2.13647","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,23]],"date-time":"2023-08-23T01:25:18Z","timestamp":1692753918000},"score":1,"resource":{"primary":{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/10.1002\/acm2.13647"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,17]]},"references-count":19,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2022,8]]}},"alternative-id":["10.1002\/acm2.13647"],"URL":"https:\/\/doi.org\/10.1002\/acm2.13647","archive":["Portico"],"relation":{},"ISSN":["1526-9914","1526-9914"],"issn-type":[{"value":"1526-9914","type":"print"},{"value":"1526-9914","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,17]]},"assertion":[{"value":"2022-01-12","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-04-28","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e13647"}}