{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T14:32:07Z","timestamp":1781533927700,"version":"3.54.5"},"reference-count":26,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T00:00:00Z","timestamp":1775606400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100008109","name":"Wuxi Municipal Bureau on Science and Technology","doi-asserted-by":"publisher","award":["Y20242206"],"award-info":[{"award-number":["Y20242206"]}],"id":[{"id":"10.13039\/501100008109","id-type":"DOI","asserted-by":"publisher"}]},{"award":["Y20242206"],"award-info":[{"award-number":["Y20242206"]}],"id":[{"id":"https:\/\/ror.org\/01zzv0338","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Radiation pneumonitis (RP), one of the most common and severe complications in locally advanced non-small cell lung cancer (LA-NSCLC) patients following thoracic radiotherapy, presents significant challenges in prediction due to the complexity of clinical risk factors, incomplete multimodal data, and unavailable slice-level annotations in pre-radiotherapy CT images. To address these challenges, we propose a multimodal fusion framework based on Dual-Layer Attention-Based Adaptive Bag Embedding Multiple-Instance Learning (DAAE-MIL) for accurate RP prediction. This study retrospectively collected data from 995 LA-NSCLC patients who received thoracic radiotherapy between November 2018 and April 2025. After screening, Subject datasets (n = 670) were allocated for training (n = 535), and the remaining samples (n = 135) were reserved for an independent test set. The proposed framework first extracts pre-radiotherapy CT image features using a fine-tuned C3D network, followed by the DAAE-MIL module to screen critical instances and generate bag-level representations, thereby enhancing the accuracy of deep feature extraction. Subsequently, clinical data, radiomics features, and CT-derived deep features are integrated to construct a multimodal prediction model. The proposed model demonstrates promising RP prediction performance across multiple evaluation metrics, outperforming both state-of-the-art and unimodal RP prediction approaches. On the test set, it achieves an accuracy (ACC) of 0.93 and an area under the curve (AUC) of 0.97. This study validates that the proposed method effectively addresses the limitations of single-modal prediction and the unknown key features in pre-radiotherapy CT images while providing significant clinical value for RP risk assessment.<\/jats:p>","DOI":"10.3390\/jimaging12040158","type":"journal-article","created":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T11:41:18Z","timestamp":1775648478000},"page":"158","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Multimodal Fusion Prediction of Radiation Pneumonitis via Key Pre-Radiotherapy Imaging Feature Selection Based on Dual-Layer Attention Multiple-Instance Learning"],"prefix":"10.3390","volume":"12","author":[{"given":"Hao","family":"Wang","sequence":"first","affiliation":[{"name":"School of Automation and Intelligent Sciences, Jiangnan University, Wuxi 214122, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dinghui","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Automation and Intelligent Sciences, Jiangnan University, Wuxi 214122, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuguang","family":"Han","sequence":"additional","affiliation":[{"name":"Department of Respiratory and Critical Care Medicine, Wuxi School of Medicine, Jiangnan University Medical Center, Wuxi 214122, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingli","family":"Tang","sequence":"additional","affiliation":[{"name":"School of Automation and Intelligent Sciences, Jiangnan University, Wuxi 214122, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenlong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Respiratory and Critical Care Medicine, Wuxi School of Medicine, Jiangnan University Medical Center, Wuxi 214122, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,4,8]]},"reference":[{"key":"ref_1","first-page":"229","article-title":"Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"74","author":"Bray","year":"2024","journal-title":"CA Cancer J. Clin."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1016\/j.semradonc.2017.04.009","article-title":"Thoracic radiation normal tissue injury","volume":"27","author":"Simone","year":"2017","journal-title":"Semin. Radiat. Oncol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1097\/MCP.0000000000000682","article-title":"A contemporary review of radiation pneumonitis","volume":"26","author":"Ullah","year":"2020","journal-title":"Curr. Opin. Pulm. Med."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Li, P., Zhang, J., and Tian, X. (2023). Radiation Pneumonia. Radiology of Infectious and Inflammatory Diseases-Volume 3: Heart and Chest, Springer.","DOI":"10.1007\/978-981-99-4614-3_23"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1016\/j.ijrobp.2013.09.037","article-title":"Modeling pathologic response of esophageal cancer to chemoradiation therapy using spatial-temporal 18F-FDG PET features, clinical parameters, and demographics","volume":"88","author":"Zhang","year":"2014","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"191","DOI":"10.3390\/automation4030012","article-title":"Automation Radiomics in Predicting Radiation Pneumonitis (RP)","volume":"4","author":"Raptis","year":"2023","journal-title":"Automation"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"15330338241254060","DOI":"10.1177\/15330338241254060","article-title":"Deep-Learning Model Prediction of Radiation Pneumonitis Using Pretreatment Chest Computed Tomography and Clinical Factors","volume":"23","author":"Lee","year":"2024","journal-title":"Technol. Cancer Res. Treat."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"109581","DOI":"10.1016\/j.radonc.2023.109581","article-title":"Computed tomography and radiation dose images-based deep-learning model for predicting radiation pneumonitis in lung cancer patients after radiation therapy","volume":"182","author":"Zhang","year":"2023","journal-title":"Radiother. Oncol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1007\/s13246-023-01249-0","article-title":"Image masking using convolutional networks improves performance classification of radiation pneumonitis for non-small cell lung cancer","volume":"46","author":"Kawahara","year":"2023","journal-title":"Phys. Eng. Sci. Med."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ai, Y., Ni, W., Su, W., Jin, X., Shen, Y., Huang, W., Xiang, Z., Yu, X., Xie, C., and Jin, X. (2025). Integrating deep learning and multi-omics features in radiation pneumonitis prediction for lung cancer patients using PET\/CT. BMC Med. Imaging, 25.","DOI":"10.1186\/s12880-025-01971-z"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1007\/s00066-024-02221-x","article-title":"Enhancing the prediction of symptomatic radiation pneumonitis for locally advanced non-small-cell lung cancer by combining 3D deep learning-derived imaging features with dose\u2013volume metrics: A two-center study","volume":"201","author":"Kong","year":"2025","journal-title":"Strahlenther. Onkol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1931","DOI":"10.1109\/TPAMI.2006.248","article-title":"MILES: Multiple-instance learning via embedded instance selection","volume":"28","author":"Chen","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Raykar, V., Krishnapuram, B., Bi, J., Dundar, M., and Rao, R. (2008). Bayesian multiple instance learning: Automatic feature selection and inductive transfer. Proceedings of the 25th International Conference on Machine Learning, Association for Computing Machinery.","DOI":"10.1145\/1390156.1390258"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1016\/j.compmedimag.2014.11.010","article-title":"Computer-aided diagnosis from weak supervision: A benchmarking study","volume":"42","author":"Kandemir","year":"2015","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/S0004-3702(96)00034-3","article-title":"Solving the multiple instance problem with axis-parallel rectangles","volume":"89","author":"Dietterich","year":"1997","journal-title":"Artif. Intell."},{"key":"ref_16","first-page":"1073","article-title":"EM-DD: An improved multiple-instance learning technique","volume":"14","author":"Zhang","year":"2001","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_17","first-page":"577","article-title":"Support vector machines for multiple-instance learning","volume":"15","author":"Andrews","year":"2002","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1109\/FSKD.2008.54","article-title":"Multi-class multi-instance learning for lung cancer image classification based on bag feature selection","volume":"Volume 2","author":"Zhu","year":"2008","journal-title":"Proceedings of the 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery, Jinan, China"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1016\/j.media.2014.01.010","article-title":"Weakly supervised histopathology cancer image segmentation and classification","volume":"18","author":"Xu","year":"2014","journal-title":"Med. Image Anal."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1016\/j.jbi.2015.08.017","article-title":"Multiple instance learning for computer aided detection and diagnosis of gastric cancer with dual-energy CT imaging","volume":"57","author":"Li","year":"2015","journal-title":"J. Biomed. Inform."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, Z., Yuan, L., Xu, H., Cheng, R., and Wen, X. (2020). Deep multi-instance learning with induced self-attention for medical image classification. Proceedings of the 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Seoul, Republic of Korea, IEEE.","DOI":"10.1109\/BIBM49941.2020.9313518"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1016\/j.patcog.2014.07.022","article-title":"Multiple instance learning with bag dissimilarities","volume":"48","author":"Cheplygina","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_23","first-page":"1500","article-title":"Predictors of pneumonitis in patients with locally advanced non-small cell lung cancer treated with definitive chemoradiation followed by consolidative durvalumab","volume":"8","author":"McDonald","year":"2023","journal-title":"Adv. Radiat. Oncol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"20424","DOI":"10.1038\/s41598-020-77552-7","article-title":"Radiomic prediction of radiation pneumonitis on pretreatment planning computed tomography images prior to lung cancer","volume":"10","author":"Hirose","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.radonc.2019.01.003","article-title":"Predicting radiation pneumonitis in locally advanced stage II\u2013III non-small cell lung cancer using machine learning","volume":"133","author":"Luna","year":"2019","journal-title":"Radiother. Oncol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"110047","DOI":"10.1016\/j.radonc.2023.110047","article-title":"Integration of dosimetric parameters, clinical factors, and radiomics to predict symptomatic radiation pneumonitis in lung cancer patients undergoing combined immunotherapy and radiotherapy","volume":"190","author":"Nie","year":"2024","journal-title":"Radiother. Oncol."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/12\/4\/158\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T12:09:10Z","timestamp":1775650150000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/12\/4\/158"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,8]]},"references-count":26,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4]]}},"alternative-id":["jimaging12040158"],"URL":"https:\/\/doi.org\/10.3390\/jimaging12040158","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,8]]}}}