{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T06:49:08Z","timestamp":1770965348175,"version":"3.50.1"},"reference-count":20,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,6,15]],"date-time":"2023-06-15T00:00:00Z","timestamp":1686787200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"FCT-Foundation for Science and Technology Portugal","award":["LA\/P\/0063\/2020"],"award-info":[{"award-number":["LA\/P\/0063\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In a clinical context, physicians usually take into account information from more than one data modality when making decisions regarding cancer diagnosis and treatment planning. Artificial intelligence-based methods should mimic the clinical method and take into consideration different sources of data that allow a more comprehensive analysis of the patient and, as a consequence, a more accurate diagnosis. Lung cancer evaluation, in particular, can benefit from this approach since this pathology presents high mortality rates due to its late diagnosis. However, many related works make use of a single data source, namely imaging data. Therefore, this work aims to study the prediction of lung cancer when using more than one data modality. The National Lung Screening Trial dataset that contains data from different sources, specifically, computed tomography (CT) scans and clinical data, was used for the study, the development and comparison of single-modality and multimodality models, that may explore the predictive capability of these two types of data to their full potential. A ResNet18 network was trained to classify 3D CT nodule regions of interest (ROI), whereas a random forest algorithm was used to classify the clinical data, with the former achieving an area under the ROC curve (AUC) of 0.7897 and the latter 0.5241. Regarding the multimodality approaches, three strategies, based on intermediate and late fusion, were implemented to combine the information from the 3D CT nodule ROIs and the clinical data. From those, the best model\u2014a fully connected layer that receives as input a combination of clinical data and deep imaging features, given by a ResNet18 inference model\u2014presented an AUC of 0.8021. Lung cancer is a complex disease, characterized by a multitude of biological and physiological phenomena and influenced by multiple factors. It is thus imperative that the models are capable of responding to that need. The results obtained showed that the combination of different types may have the potential to produce more comprehensive analyses of the disease by the models.<\/jats:p>","DOI":"10.3390\/s23125597","type":"journal-article","created":{"date-parts":[[2023,6,15]],"date-time":"2023-06-15T03:18:46Z","timestamp":1686799126000},"page":"5597","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Single Modality vs. Multimodality: What Works Best for Lung Cancer Screening?"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1563-2978","authenticated-orcid":false,"given":"Joana Vale","family":"Sousa","sequence":"first","affiliation":[{"name":"Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal"},{"name":"Faculty of Engineering (FEUP), University of Porto, 4200-465 Porto, Portugal"}]},{"given":"Pedro","family":"Matos","sequence":"additional","affiliation":[{"name":"Faculty of Engineering (FEUP), University of Porto, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3069-2282","authenticated-orcid":false,"given":"Francisco","family":"Silva","sequence":"additional","affiliation":[{"name":"Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal"},{"name":"Faculty of Science (FCUP), University of Porto, 4169-007 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4588-5880","authenticated-orcid":false,"given":"Pedro","family":"Freitas","sequence":"additional","affiliation":[{"name":"Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal"},{"name":"Faculty of Engineering (FEUP), University of Porto, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6193-8540","authenticated-orcid":false,"given":"H\u00e9lder P.","family":"Oliveira","sequence":"additional","affiliation":[{"name":"Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal"},{"name":"Faculty of Science (FCUP), University of Porto, 4169-007 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1681-2436","authenticated-orcid":false,"given":"Tania","family":"Pereira","sequence":"additional","affiliation":[{"name":"Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"209","DOI":"10.3322\/caac.21660","article-title":"Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"71","author":"Sung","year":"2021","journal-title":"CA Cancer J. Clin."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Silva, F., Pereira, T., Neves, I., Morgado, J., Freitas, C., Malafaia, M., Sousa, J., Fonseca, J., Negr\u00e3o, E., and Flor de Lima, B. (2022). Towards Machine Learning-Aided Lung Cancer Clinical Routines: Approaches and Open Challenges. J. Pers. Med., 12.","DOI":"10.3390\/jpm12030480"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"889","DOI":"10.1183\/13993003.00359-2016","article-title":"Risk factors for lung cancer worldwide","volume":"48","author":"Malhotra","year":"2016","journal-title":"Eur. Respir. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"39","DOI":"10.5644\/ama2006-124.213","article-title":"Lung Cancer: Preventable Disease","volume":"47","author":"Srkalovic","year":"2018","journal-title":"Acta Medica Acad."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1007\/s11547-019-01130-9","article-title":"Classification of lung nodules based on CT images using squeeze-and-excitation network and aggregated residual transformations","volume":"125","author":"Zhang","year":"2020","journal-title":"La Radiol. Medica"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1038\/s41746-020-00341-z","article-title":"Fusion of medical imaging and electronic health records using deep learning: A systematic review and implementation guidelines","volume":"3","author":"Huang","year":"2020","journal-title":"NPJ Digit. Med."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lyu, J., Bi, X., and Ling, S.H. (2020). Multi-Level Cross Residual Network for Lung Nodule Classification. Sensors, 20.","DOI":"10.3390\/s20102837"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1118\/1.3528204","article-title":"The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans","volume":"38","author":"McLennan","year":"2011","journal-title":"Med. Phys."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"798","DOI":"10.1007\/s10278-021-00453-2","article-title":"The Effects of Perinodular Features on Solid Lung Nodule Classification","volume":"34","author":"Calheiros","year":"2021","journal-title":"J. Digit. Imaging"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1319","DOI":"10.1007\/s11548-021-02415-z","article-title":"3D axial-attention for lung nodule classification","volume":"16","author":"Shak","year":"2021","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"457","DOI":"10.31557\/APJCP.2019.20.2.457","article-title":"A Novel Hybrid Feature Extraction Model for Classification on Pulmonary Nodules","volume":"20","author":"Kailasam","year":"2019","journal-title":"Asian Pac. J. Cancer Prev."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1007\/s11548-022-02576-5","article-title":"Res-trans networks for lung nodule classification","volume":"17","author":"Liu","year":"2022","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_13","first-page":"154","article-title":"Computed Tomography Features of Lung Structure Have Utility for Differentiating Malignant and Benign Pulmonary Nodules","volume":"9","author":"Uthoff","year":"2022","journal-title":"Chronic Obstr. Pulm. Dis. J. COPD Found."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"32","DOI":"10.3109\/15412550903499522","article-title":"Genetic Epidemiology of COPD (COPDGene) Study Design","volume":"7","author":"Regan","year":"2011","journal-title":"COPD J. Chronic Obstr. Pulm. Dis."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1158\/1055-9965.EPI-16-0176","article-title":"Risk of Lung Cancer Associated with COPD Phenotype Based on Quantitative Image Analysis","volume":"25","author":"Schwartz","year":"2016","journal-title":"Cancer Epidemiol. Biomark. Prev."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"National Lung Screening Trial Team, Aberle, D.R., Berg, C.D., Black, W.C., Church, T.R., Fagerstrom, R.M., Galen, B., Gareen, I.F., Gatsonis, C., and Goldin, J. (2011). The National Lung Screening Trial: Overview and Study Design. Radiology, 258, 243\u2013253.","DOI":"10.1148\/radiol.10091808"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"5472","DOI":"10.1002\/mp.13237","article-title":"Fusion of quantitative imaging features and serum biomarkers to improve performance of computer-aided diagnosis scheme for lung cancer: A preliminary study","volume":"45","author":"Gong","year":"2018","journal-title":"Med. Phys."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"bbab569","DOI":"10.1093\/bib\/bbab569","article-title":"Multimodal deep learning for biomedical data fusion: A review","volume":"23","author":"Stahlschmidt","year":"2022","journal-title":"Briefings Bioinform."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1109\/TKDE.2005.50","article-title":"Using AUC and accuracy in evaluating learning algorithms","volume":"17","author":"Huang","year":"2005","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2939","DOI":"10.1007\/s00371-021-02166-7","article-title":"A survey on deep multimodal learning for computer vision: Advances, trends, applications, and datasets","volume":"38","author":"Bayoudh","year":"2022","journal-title":"Vis. Comput."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/12\/5597\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:55:22Z","timestamp":1760126122000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/12\/5597"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,15]]},"references-count":20,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["s23125597"],"URL":"https:\/\/doi.org\/10.3390\/s23125597","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,15]]}}}