{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:31:14Z","timestamp":1760146274175,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,10,24]],"date-time":"2024-10-24T00:00:00Z","timestamp":1729728000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>This study investigates Intraobserver Features Variability (IFV) in radiomics studies and assesses the effectiveness of the ComBat harmonization method in mitigating these effects. Methods: This study utilizes data from the NSCLC-Radiomics-Interobserver1 dataset, comprising CT scans of 22 Non-Small Cell Lung Cancer (NSCLC) patients, with multiple Gross Tumor Volume (GTV) delineations performed by five radiation oncologists. Segmentation was completed manually (\u201cvis\u201d) or by autosegmentation with manual editing (\u201cauto\u201d). A total of 1229 radiomic features were extracted from each GTV, segmentation method, and oncologist. Features extracted included first order, shape, GLCM, GLRLM, GLSZM, and GLDM from original, wavelet-filtered, and LoG-filtered images. Results: Before implementing ComBat harmonization, 83% of features exhibited p-values below 0.05 in the \u201cvis\u201d approach; this percentage decreased to 34% post-harmonization. Similarly, for the \u201cauto\u201d approach, 75% of features demonstrated statistical significance prior to ComBat, but this figure declined to 33% after its application. Among a subset of three expert radiation oncologists, percentages changed from 77% to 25% for \u201cvis\u201d contouring and from 64% to 23% for \u201cauto\u201d contouring. This study demonstrates that ComBat harmonization could effectively reduce IFV, enhancing the feasibility of multicenter radiomics studies. It also highlights the significant impact of physician experience on radiomics analysis outcomes.<\/jats:p>","DOI":"10.3390\/jimaging10110270","type":"journal-article","created":{"date-parts":[[2024,10,25]],"date-time":"2024-10-25T03:46:04Z","timestamp":1729827964000},"page":"270","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Mitigating Interobserver Variability in Radiomics with ComBat: A Feasibility Study"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-4488-3170","authenticated-orcid":false,"given":"Alessia","family":"D\u2019Anna","sequence":"first","affiliation":[{"name":"Department of Physics and Astronomy \u201cE. Majorana\u201d, University of Catania, Via Santa Sofia 64, 95123 Catania, Italy"}]},{"given":"Giuseppe","family":"Stella","sequence":"additional","affiliation":[{"name":"Department of Physics and Astronomy \u201cE. Majorana\u201d, University of Catania, Via Santa Sofia 64, 95123 Catania, Italy"}]},{"given":"Anna Maria","family":"Gueli","sequence":"additional","affiliation":[{"name":"Department of Physics and Astronomy \u201cE. Majorana\u201d, University of Catania, Via Santa Sofia 64, 95123 Catania, Italy"}]},{"given":"Carmelo","family":"Marino","sequence":"additional","affiliation":[{"name":"Department of Medical Phyisics, Humanitas Istituto Clinico Catanese (H-ICC), Contrada Cubba S.P. 54 n.11, 95045 Misterbianco, Italy"}]},{"given":"Alfredo","family":"Pulvirenti","sequence":"additional","affiliation":[{"name":"Department of Clinical and Experimental Medicine, University of Catania, Via Santa Sofia 64, 95123 Catania, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1016\/j.ejca.2011.11.036","article-title":"Radiomics: Extracting More Information from Medical Images Using Advanced Feature Analysis","volume":"48","author":"Lambin","year":"2012","journal-title":"Eur. J. Cancer"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1016\/j.mri.2012.06.010","article-title":"Radiomics: The Process and the Challenges","volume":"30","author":"Kumar","year":"2012","journal-title":"Magn. Reason. Imaging"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"563","DOI":"10.1148\/radiol.2015151169","article-title":"Radiomics: Images Are More than Pictures, They Are Data","volume":"278","author":"Gillies","year":"2016","journal-title":"Radiology"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"20160665","DOI":"10.1259\/bjr.20160665","article-title":"Quantitative Radiomics Studies for Tissue Characterization: A Review of Technology and Methodological Procedures","volume":"90","author":"Larue","year":"2017","journal-title":"Br. J. Radiol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1038\/nrclinonc.2017.141","article-title":"Radiomics: The Bridge between Medical Imaging and Personalized Medicine","volume":"14","author":"Lambin","year":"2017","journal-title":"Nat. Rev. Clin. Oncol."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Da-Ano, R., Visvikis, D., and Hatt, M. (2020). Harmonization Strategies for Multicenter Radiomics Investigations. Phys. Med. Biol., 65.","DOI":"10.1088\/1361-6560\/aba798"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s00259-016-3427-0","article-title":"Characterization of PET\/CT Images Using Texture Analysis: The Past, the Present\u2026 Any Future?","volume":"44","author":"Hatt","year":"2017","journal-title":"Eur. J. Nucl. Med. Mol. Imaging"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1038\/nrclinonc.2016.162","article-title":"Imaging Biomarker Roadmap for Cancer Studies","volume":"14","author":"Aboagye","year":"2017","journal-title":"Nat. Rev. Clin. Oncol."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sheller, M.J., Edwards, B., Reina, G.A., Martin, J., Pati, S., Kotrotsou, A., Milchenko, M., Xu, W., Marcus, D., and Colen, R.R. (2020). Federated Learning in Medicine: Facilitating Multi-Institutional Collaborations without Sharing Patient Data. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-69250-1"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1038\/s41746-020-00323-1","article-title":"The Future of Digital Health with Federated Learning","volume":"3","author":"Rieke","year":"2020","journal-title":"NPJ Digit. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1012","DOI":"10.3109\/0284186X.2010.498437","article-title":"Variability of Textural Features in FDG PET Images Due to Different Acquisition Modes and Reconstruction Parameters","volume":"49","author":"Galavis","year":"2010","journal-title":"Acta Oncol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1667","DOI":"10.2967\/jnumed.115.156927","article-title":"Impact of Image Reconstruction Settings on Texture Features in 18 F-FDG PET","volume":"56","author":"Yan","year":"2015","journal-title":"J. Nucl. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2638","DOI":"10.1007\/s00259-019-04391-8","article-title":"Radiomics in Nuclear Medicine: Robustness, Reproducibility, Standardization, and How to Avoid Data Analysis Traps and Replication Crisis","volume":"46","author":"Zwanenburg","year":"2019","journal-title":"Eur. J. Nucl. Med. Mol. Imaging"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1016\/j.radonc.2018.03.004","article-title":"Why Validation of Prognostic Models Matters?","volume":"127","author":"Zwanenburg","year":"2018","journal-title":"Radiother. Oncol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1016\/j.radonc.2016.10.002","article-title":"Distributed Learning: Developing a Predictive Model Based on Data from Multiple Hospitals without Data Leaving the Hospital\u2014A Real Life Proof of Concept","volume":"121","author":"Jochems","year":"2016","journal-title":"Radiother. Oncol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Horng, H., Singh, A., Yousefi, B., Cohen, E.A., Haghighi, B., Katz, S., No\u00ebl, P.B., Kontos, D., and Shinohara, R.T. (2022). Improved Generalized ComBat Methods for Harmonization of Radiomic Features. Sci. Rep., 12.","DOI":"10.1038\/s41598-022-23328-0"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1093\/biostatistics\/kxj037","article-title":"Adjusting Batch Effects in Microarray Expression Data Using Empirical Bayes Methods","volume":"8","author":"Johnson","year":"2007","journal-title":"Biostatistics"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chen, C., Grennan, K., Badner, J., Zhang, D., Gershon, E., Jin, L., and Liu, C. (2011). Removing Batch Effects in Analysis of Expression Microarray Data: An Evaluation of Six Batch Adjustment Methods. PLoS ONE, 6.","DOI":"10.1371\/journal.pone.0017238"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.neuroimage.2017.08.047","article-title":"Harmonization of Multi-Site Diffusion Tensor Imaging Data","volume":"161","author":"Fortin","year":"2017","journal-title":"Neuroimage"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"172","DOI":"10.2967\/jnumed.121.262464","article-title":"A Guide to ComBat Harmonization of Imaging Biomarkers in Multicenter Studies","volume":"63","author":"Orlhac","year":"2022","journal-title":"J. Nucl. Med."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2272","DOI":"10.1007\/s00330-020-07284-9","article-title":"How Can We Combat Multicenter Variability in MR Radiomics? Validation of a Correction Procedure","volume":"31","author":"Orlhac","year":"2021","journal-title":"Eur. Radiol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1148\/radiol.2019182023","article-title":"Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics","volume":"291","author":"Orlhac","year":"2019","journal-title":"Radiology"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1016\/j.neuroimage.2017.11.024","article-title":"Harmonization of Cortical Thickness Measurements across Scanners and Sites","volume":"167","author":"Fortin","year":"2018","journal-title":"Neuroimage"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Foy, J.J., Al-Hallaq, H.A., Grekoski, V., Tran, T., Guruvadoo, K., Armato, S.G., and Sensakovic, W.F. (2020). Harmonization of Radiomic Feature Variability Resulting from Differences in CT Image Acquisition and Reconstruction: Assessment in a Cadaveric Liver. Phys. Med. Biol., 65.","DOI":"10.1088\/1361-6560\/abb172"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1007\/s10334-020-00892-y","article-title":"A Radiomics Pipeline Dedicated to Breast MRI: Validation on a Multi-Scanner Phantom Study","volume":"34","author":"Orlhac","year":"2021","journal-title":"Magn. Reson. Mater. Phys. Biol. Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1186\/s13244-020-00887-2","article-title":"Radiomics in Medical Imaging\u2014\u201cHow-to\u201d Guide and Critical Reflection","volume":"11","author":"Cester","year":"2020","journal-title":"Insights Imaging"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.semradonc.2019.02.001","article-title":"Advances in Auto-Segmentation","volume":"29","author":"Cardenas","year":"2019","journal-title":"Semin. Radiat. Oncol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1016\/j.radonc.2015.10.007","article-title":"Variability of Clinical Target Volume Delineation for Definitive Radiotherapy in Cervix Cancer","volume":"117","author":"Eminowicz","year":"2015","journal-title":"Radiother. Oncol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.radonc.2012.02.010","article-title":"Heterogeneity in Head and Neck IMRT Target Design and Clinical Practice","volume":"103","author":"Hong","year":"2012","journal-title":"Radiother. Oncol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"944","DOI":"10.1016\/j.ijrobp.2008.10.034","article-title":"Variability of Target and Normal Structure Delineation for Breast Cancer Radiotherapy: An RTOG Multi-Institutional and Multiobserver Study","volume":"73","author":"Li","year":"2009","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1612","DOI":"10.1016\/j.neuroimage.2006.07.050","article-title":"Optimum Template Selection for Atlas-Based Segmentation","volume":"34","author":"Wu","year":"2007","journal-title":"Neuroimage"},{"key":"ref_32","unstructured":"Wee, L., Aerts, H.J.L., Kalendralis, P., and Dekker, A. (2019). Data From NSCLC-Radiomics-Interobserver1 [Data Set]. Cancer Imaging Arch., 10."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1007\/s10278-013-9622-7","article-title":"The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository","volume":"26","author":"Clark","year":"2013","journal-title":"J. Digit. Imaging"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4006","DOI":"10.1038\/ncomms5006","article-title":"Decoding Tumour Phenotype by Noninvasive Imaging Using a Quantitative Radiomics Approach","volume":"5","author":"Aerts","year":"2014","journal-title":"Nat. Commun."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"5931","DOI":"10.1002\/mp.14322","article-title":"FAIR-compliant Clinical, Radiomics and DICOM Metadata of RIDER, Interobserver, Lung1 and Head-Neck1 TCIA Collections","volume":"47","author":"Kalendralis","year":"2020","journal-title":"Med. Phys."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Primakov, S., Lavrova, E., Salahuddin, Z., Woodruff, H.C., and Lambin, P. (2022). Precision-Medicine-Toolbox: An Open-Source Python Package for Facilitation of Quantitative Medical Imaging and Radiomics Analysis. arXiv.","DOI":"10.1016\/j.simpa.2023.100508"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1148\/radiol.2020191145","article-title":"The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-Based Phenotyping","volume":"295","author":"Zwanenburg","year":"2020","journal-title":"Radiology"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kothari, G., Woon, B., Patrick, C.J., Korte, J., Wee, L., Hanna, G.G., Kron, T., Hardcastle, N., and Siva, S. (2022). The Impact of Inter-Observer Variation in Delineation on Robustness of Radiomics Features in Non-Small Cell Lung Cancer. Sci. Rep., 12.","DOI":"10.1038\/s41598-022-16520-9"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/10\/11\/270\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:19:54Z","timestamp":1760113194000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/10\/11\/270"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,24]]},"references-count":38,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["jimaging10110270"],"URL":"https:\/\/doi.org\/10.3390\/jimaging10110270","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2024,10,24]]}}}