{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T11:11:31Z","timestamp":1785409891742,"version":"3.56.0"},"reference-count":36,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T00:00:00Z","timestamp":1785283200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NIH\/NCI Cancer Center","award":["P30 CA008748"],"award-info":[{"award-number":["P30 CA008748"]}]},{"award":["P30 CA008748"],"award-info":[{"award-number":["P30 CA008748"]}],"id":[{"id":"https:\/\/ror.org\/00g1d7b60","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Cancers"],"abstract":"<jats:p>Background\/Objectives: Inter-reader variation can alter unidimensional tumor measurements used in RECIST-oriented response assessment. Unlike systems that automate segmentation or target selection, this study aimed to predict lesion-specific measurement uncertainty itself. Methods: The development cohort comprised 463 lung, liver, and lymph-node lesions from 280 patients in Vol-PACT, with four segmentations per lesion. Inter-reader variability was defined as (maximum longest diameter\u2013minimum longest diameter)\/minimum longest diameter. A Swin Transformer was trained to regress this continuous score. A value &gt; 0.20 was used as a pragmatic high-variability alert threshold, with high variability designated as the positive class; this threshold is not equivalent to RECIST progressive disease. Preliminary external evaluation used 21 NSCLC lesions with five reader contours, providing one reader mask at a time. Results: The validation and internal test AUCs were 0.74 (95% CI, 0.61\u20130.85) and 0.89 (95% CI, 0.80\u20130.95), respectively. On the internal test set, MAE was 0.10 (95% CI, 0.08\u20130.12), accuracy was 82% (95% CI, 74\u201390%), sensitivity 72% (95% CI, 59\u201385%), and specificity for lower-variability lesions was 92% (95% CI, 82\u2013100%). Across the five external readers, AUCs ranged from 0.77 to 0.92; the wide confidence intervals reflect the small external sample. Conclusions: This proof-of-concept model may provide a measurement-quality signal that supports target selection or adjudication. Larger multi-institutional studies, patient-level resampling, calibration, ablation testing, and workflow validation are required before clinical use.<\/jats:p>","DOI":"10.3390\/cancers18152438","type":"journal-article","created":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T08:16:52Z","timestamp":1785313012000},"page":"2438","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Deep Learning Quality Control for RECIST-Oriented Assessment: A Vision Transformer Predicts Inter-Reader Variability of Unidimensional Lesion Measurements on Contrast CT"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4138-5755","authenticated-orcid":false,"given":"Abdalla","family":"Ibrahim","sequence":"first","affiliation":[{"name":"Department of Radiology, Memorial Sloan Kettering Cancer Center, 1275 York Ave, New York, NY 10065, USA"},{"name":"Department of Radiation Oncology, William Beaumont University Hospital-Corewell Health, 3601 W 13 Mile Rd, Royal Oak, MI 48084, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amer","family":"Hammad","sequence":"additional","affiliation":[{"name":"The D-Lab, Department of Precision Medicine, GROW\u2014Research Institute for Oncology and Reproduction, Maastricht University, Universiteitsingel 40, 6229 ER Maastricht, The Netherlands"},{"name":"Department of Medicine, College of Medicine, University of Arizona\u2014Tucson, 1501 N Campbell Ave, Tucson, AZ 85724, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lucy","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Medicine, New York Medical College, 40 Sunshine Cottage Rd, Valhalla, NY 10595, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Radiology, Memorial Sloan Kettering Cancer Center, 1275 York Ave, New York, NY 10065, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lawrence H.","family":"Schwartz","sequence":"additional","affiliation":[{"name":"Department of Radiology, Memorial Sloan Kettering Cancer Center, 1275 York Ave, New York, NY 10065, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Binsheng","family":"Zhao","sequence":"additional","affiliation":[{"name":"Department of Radiology, Memorial Sloan Kettering Cancer Center, 1275 York Ave, New York, NY 10065, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,7,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.critrevonc.2008.07.012","article-title":"Imaging techniques to evaluate the response to treatment in oncology: Current standards and perspectives","volume":"72","author":"Marcus","year":"2009","journal-title":"Crit. Rev. Oncol. Hematol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1016\/j.ejca.2008.10.027","article-title":"Individual patient data analysis to assess modifications to the RECIST criteria","volume":"45","author":"Bogaerts","year":"2009","journal-title":"Eur. J. Cancer"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1456","DOI":"10.1007\/s00330-009-1685-y","article-title":"RECIST revised: Implications for the radiologist. A review article on the modified RECIST guideline","volume":"20","author":"Gelderblom","year":"2010","journal-title":"Eur. Radiol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/j.ejca.2016.03.081","article-title":"RECIST 1.1-Update and clarification: From the RECIST committee","volume":"62","author":"Schwartz","year":"2016","journal-title":"Eur. J. Cancer"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.ejca.2015.10.014","article-title":"Observer variability in RECIST-based tumour burden measurements: A meta-analysis","volume":"53","author":"Yoon","year":"2016","journal-title":"Eur. J. Cancer"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1097\/RLI.0000000000000048","article-title":"Target lesion selection: An important factor causing variability of response classification in the Response Evaluation Criteria for Solid Tumors 1.1","volume":"49","author":"Keil","year":"2014","journal-title":"Investig. Radiol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2574","DOI":"10.1200\/JCO.2003.01.144","article-title":"Interobserver and intraobserver variability in measurement of non-small-cell carcinoma lung lesions: Implications for assessment of tumor response","volume":"21","author":"Erasmus","year":"2003","journal-title":"J. Clin. Oncol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1148\/radiol.13122665","article-title":"Intra- and interobserver variability in CT measurements in oncology","volume":"269","author":"McErlean","year":"2013","journal-title":"Radiology"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"W221","DOI":"10.2214\/AJR.09.3928","article-title":"New Response Evaluation Criteria in Solid Tumors (RECIST) guidelines for advanced non-small cell lung cancer: Comparison with original RECIST and impact on assessment of tumor response to targeted therapy","volume":"195","author":"Nishino","year":"2010","journal-title":"Am. J. Roentgenol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhao, B. (2021). Understanding Sources of Variation to Improve the Reproducibility of Radiomics. Front. Oncol., 11.","DOI":"10.3389\/fonc.2021.633176"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3114","DOI":"10.1200\/JCO.2010.33.7071","article-title":"Variability of Lung Tumor Measurements on Repeat Computed Tomography Scans Taken Within 15 Minutes","volume":"29","author":"Oxnard","year":"2011","journal-title":"J. Clin. Oncol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"989","DOI":"10.2214\/AJR.04.1821","article-title":"Inherent variability of CT lung nodule measurements in vivo using semiautomated volumetric measurements","volume":"186","author":"Goodman","year":"2006","journal-title":"Am. J. Roentgenol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ibrahim, A., Barufaldi, B., Refaee, T., Filho, T.M.S., Acciavatti, R.J., Salahuddin, Z., Hustinx, R., Mottaghy, F.M., Maidment, A.D.A., and Lambin, P. (2022). MaasPenn radiomics reproducibility score: A novel quantitative measure for evaluating the reproducibility of CT-based handcrafted radiomic features. Cancers, 14.","DOI":"10.3390\/cancers14071599"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1831","DOI":"10.1016\/j.ejrad.2012.05.018","article-title":"Size determination and response assessment of liver metastases with computed tomography--comparison of RECIST and volumetric algorithms","volume":"82","author":"Rothe","year":"2013","journal-title":"Eur. J. Radiol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"306","DOI":"10.2214\/AJR.05.1063","article-title":"Pulmonary nodule volumetric measurement variability as a function of CT slice thickness and nodule morphology","volume":"188","author":"Petrou","year":"2007","journal-title":"Am. J. Roentgenol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1007\/s00330-008-1229-x","article-title":"A comparison of six software packages for evaluation of solid lung nodules using semi-automated volumetry: What is the minimum increase in size to detect growth in repeated CT examinations?","volume":"19","author":"Gietema","year":"2009","journal-title":"Eur. Radiol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1148\/rg.282075068","article-title":"Radiologic measurements of tumor response to treatment: Practical approaches and limitations","volume":"28","author":"Suzuki","year":"2008","journal-title":"Radiographics"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1148\/radiol.2018180648","article-title":"Validity of RECIST Version 1.1 for Response Assessment in Metastatic Cancer: A Prospective, Multireader Study","volume":"290","author":"Kuhl","year":"2019","journal-title":"Radiology"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Guha, S., Ibrahim, A., Geng, P., Wu, Q., Chou, Y., Akin, O., Schwartz, L.H., Xie, C.-M., and Zhao, B. (2025). Variability of HCC Tumor Diameter and Density Measurements on Dynamic Contrast-Enhanced Computed Tomography. Tomography, 11.","DOI":"10.3390\/tomography11030036"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1007\/s00330-003-2132-0","article-title":"Volumetric measurements of pulmonary nodules at multi-row detector CT: In vivo reproducibility","volume":"14","author":"Wormanns","year":"2004","journal-title":"Eur. Radiol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3852","DOI":"10.1038\/s41598-020-60868-9","article-title":"Reliability of CT radiomic features reflecting tumour heterogeneity according to image quality and image processing parameters","volume":"10","author":"Park","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1427","DOI":"10.1148\/rg.2021200210","article-title":"Deep Learning: An Update for Radiologists","volume":"41","author":"Cheng","year":"2021","journal-title":"Radiographics"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yip, S.S.F., Parmar, C., Blezek, D., Estepar, R.S.J., Pieper, S., Kim, J., and Aerts, H. (2017). Application of the 3D slicer chest imaging platform segmentation algorithm for large lung nodule delineation. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0178944"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1007\/s00330-005-0036-x","article-title":"Inadequacy of manual measurements compared to automated CT volumetry in assessment of treatment response of pulmonary metastases using RECIST criteria","volume":"16","author":"Marten","year":"2006","journal-title":"Eur. Radiol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1973","DOI":"10.1056\/NEJMoa2202170","article-title":"Neoadjuvant nivolumab plus chemotherapy in resectable lung cancer","volume":"386","author":"Forde","year":"2022","journal-title":"N. Engl. J. Med."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1558","DOI":"10.1200\/JCO.19.00201","article-title":"Local consolidative therapy vs. maintenance therapy or observation for patients with oligometastatic non-small-cell lung cancer: Long-term results of a multi-institutional, phase II, randomized study","volume":"37","author":"Gomez","year":"2019","journal-title":"J. Clin. Oncol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"108978","DOI":"10.1016\/j.ejrad.2020.108978","article-title":"Can high-risk CT features suggest local recurrence after stereotactic body radiation therapy for lung cancer? A systematic review and meta-analysis","volume":"127","author":"Kim","year":"2020","journal-title":"Eur. J. Radiol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Expert Panel on Thoracic Imaging, Madan, R., El Alam, R.H., Walker, C.M., Bang, T.J., Bartel, T.B., Batra, K., Brixey, A.G., Christensen, J.D., and Cox, C.W. (2025). ACR Appropriateness Criteria\u00ae Lung Cancer-Surveillance After Therapy. J. Am. Coll. Radiol., 22, S319\u2013S342.","DOI":"10.1016\/j.jacr.2025.02.025"},{"key":"ref_30","first-page":"1","article-title":"The Evolving Paradigm in Managing Pulmonary Ground-glass Nodules: From Aggressive Intervention to Precision Surveillance","volume":"51","author":"Wu","year":"2026","journal-title":"J. Radiol. Sci."},{"key":"ref_31","unstructured":"American College of Radiology Committee on Lung-RADS\u00ae (2022). Lung-RADS Assessment Categories 2022, American College of Radiology. Available online: https:\/\/www.massgeneral.org\/assets\/MGH\/pdf\/imaging\/lung-screening\/lung-rads-assessment-categories.pdf."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1200\/CCI.17.00137","article-title":"Vol-PACT: A foundation for the NIH public-private partnership that supports sharing of clinical trial data for the development of improved imaging biomarkers in oncology","volume":"2","author":"Dercle","year":"2018","journal-title":"JCO Clin. Cancer Inform."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"umaf001","DOI":"10.1093\/radadv\/umaf001","article-title":"Comparing quantitative imaging biomarker alliance volumetric CT classifications with RECIST response categories","volume":"2","author":"Zhao","year":"2025","journal-title":"Radiol. Adv."},{"key":"ref_34","unstructured":"Wee, L., Aerts, H., Kalendralis, P., and Dekker, A. (2019). Data from NSCLC-Radiomics-Interobserver1 [Data Set], The Cancer Imaging Archive."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"100562","DOI":"10.1016\/j.ejro.2024.100562","article-title":"Reproducing RECIST lesion selection via machine learning: Insights into intra and inter-radiologist variation","volume":"12","author":"Petrychenko","year":"2024","journal-title":"Eur. J. Radiol. Open"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1186\/s13244-024-01774-w","article-title":"RECIST 1.1 assessments variability: A systematic pictorial review of blinded double reads","volume":"15","author":"Iannessi","year":"2024","journal-title":"Insights Imaging"}],"container-title":["Cancers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-6694\/18\/15\/2438\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T08:29:40Z","timestamp":1785313780000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-6694\/18\/15\/2438"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,29]]},"references-count":36,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["cancers18152438"],"URL":"https:\/\/doi.org\/10.3390\/cancers18152438","relation":{},"ISSN":["2072-6694"],"issn-type":[{"value":"2072-6694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,29]]}}}